In this literature review, my objective is to introduce the deep
learning technology. Very less is heard of deep learning method to solve
engineering problems, this report will answer about exploring Deep learning
method to facilitate big data analysis and its applications to solve the engineering
problems. We will also learn about the
technologies where deep learning method was used and its results.
Daily an enormous amount of sensing devices collects and generate large
amount of data over a period for a wide range of fields and applications. Based
on the nature of the application, these devices will result in real-time data
streams. To analyze this data and transforming it into useful data to do continuous
improvement in science & technology is
the key, which we will understand through this literature survey.
I started my literature review with reading articles on deep
learning and big data by various University such as Obuda University( Budapest
Hungary) and Institute of Structural mechanics ( Weimar, Germany). These
articles helped me in understanding the applications of deep learning in
various fields like image processing, biometrics and also in medicine. It is important to know deep learning first
which will make it easier to understand its application on solving engineering
problems. These articles and various other articles are available on the
internet as an opensource articles and has free access. I have mentioned the
applications of deep learning which are present in all the articles. Also I
have shown how the publications on deep learning is increasing day by day with
help of data from science journals.
My aim in this literature review is to answer the following
is deep learning?
are problems that have been solved using deep learning?
successful is deep learning while applying to various problems and what are its
The research methods that I used are:
Literature Study(Library): It is important to know
the subject thoroughly before writing a report on it. This method help me
understand more in detail about Deep learning.
Scenario(Stepping Stones): Since it’s a latest technology development, current scenario and
applications are useful to demonstrate advantages of deep learning.
Concept, Requirement List and Risk analysis(Stepping Stones):By using this methods of research we can learn:
what was the
need of developing deep learning method, its advantages and risks of misusing
the big data as it involves artificial intelligence.
Highly confidential data can also be misused
while evaluating and understanding sensory data. Risk analysis and protecting
privacy & security is also important to ensure ethical use of this
In most of the articles, it can be seen the efforts to make the
machine learning more advanced from designing control system to a level that it
can make its own decisions i.e. artificial intelligence.
Main body of text
Artificial intelligence (AI) is the intelligence demonstrated by
machines which is an effective approach to human learning and reasoning 1. In
1950, “The Turing Test” was proposed as an good explanation of how a computer
could perform a human cognitive reasoning 2. AI can be divided into specific
research sub-fields. For example: Natural Language Processing (NLP) 3 can
enhance the writing experience in various applications 4,5. The most important
part of NLP is machine translation, which is the translation between languages.
Machine translation algorithms aides applications which can consider grammar
structure as well as spelling mistakes. Also computer suggests words to writer
or editor to make changes 6. Figure 1 shows how AI covers seven subfields of
Recent studies in big data and machine learning analyzes
multiple possibilities of characterization of databases 7. For many the
years, databases are collected for statistical purposes. Statistical curves can
describe past, and present in order to predict future behaviors. However the
techniques developed to process this data by algorithms is very recent, an
optimization of those algorithms could lead on an effective self–learning 8. Decision
making by robots can be implemented based on existing values, multiple criteria
and statistics advanced methods.
Figure 1. Artificial intelligence
and its subfields.
3.2 What is deep learning?
Deep learning is an emerging area of machine learning research. It
comprises of multiple hidden layers of artificial neural networks. The deep
learning methodology applies nonlinear transformations and model abstractions
of high level in large databases. The recent development in deep learning
architectures within numerous fields have already provided significant
contributions in artificial intelligence1.
Deep learning method’s ultimate goal is to give rise to artificial
intelligence with prime focus on
mathematical and computational principles to learn from examples to
Deep learning first appeared in the year 2006 and
it was known as hierarchical learning which is usually used in the fields of
pattern recognition 2. Deep learning consists of mainly two aspects 4:
processing in multi-layers.
and Unsupervised learning.
Nonlinear processing in multiple layers means an algorithm where
the current layer takes the output of the previous layer as an input. Hierarchy
is developed among the layers of data
based on importance criteria. On the other hand, supervised and unsupervised learning is related
with the class target label, its availability means a supervised system,
whereas its absence means an unsupervised system.
Applications of Deep Learning in engineering problems:
2003, experiments were done by applying particle filtering and Bayesian –
belief propagation. The main concept of
this experiment was that if human can detect the face of person by watching
only half cropped photo, a computer can also reconstruct the photo from half
2006, greed algorithm and hierarchy was developed to process handwritten
Convolutional Neural Networks for iris recognitions increases the accuracy up
to 99.35% 11.
location recognition nowadays allows the user to know a determined address
based on a picture. A Supervised Semantics – Preserving Deep Hashing (SSPDH)
algorithm has proved a considerable improvement in comparison with Visual Hash
(VHB) and Space – Saliency Fingerprint
Selection (SSFS). The accuracy of SSPDH is even 70% more efficient 12.
Facebook and Microsoft all have developed face detection as security lock
system for the cell phones this digital image processing is done by the use of
deep learning method.
2009, automatic speech recognition application was carried out to reduce the
Phone Error Rate by using two different architectures of deep belief networks
was employed to speed up the developing and optimization of FaceSentinel face
recognition devices. There devices could expand the face recognition process
from one -to-one to one -to- many in nine months 14. Without deep learning it
would have taken 10 years. These devices are used at Heathrow airport in London
and have the potential to be used as time and attendance and in banking sector 3.
Natural Language Processing:
uses large end-to- end long short-term memory network. It translates the whole sentences
rather than some part of the sentence. It supports over 100 languages 15.
Several important applications of deep learning in the field of speech
recognition and image processing are summarized in the Table -1 16.
Table 1: Deep learning applications 2003-2017
Publications in Deep learning
Figure 2 shows database of springer’s publications in deep learning.
We can see that there is a huge increase in publications which shows the keen interests
of researchers 16.
2: Data of number of publications in deep learning from
Springer database :2006 to 2017.
learning is one of the fastest growing applications of machine learning. Large number
of publications prove that a lot of research is going on in this subject.
of layers and supervision in learning are the two key aspects to develop a good
application of deep learning. Hierarchy is essential for data classification whereas
supervision checks for the importance of the data in the process.
learning has given concrete results in the field of image processing and speech
learning is also an important method for designing security tools as it can do facial
recognitions which is already a feature in today’s smartphones.
Abdel, O. : Applying convolutional neural networks
concepts to hybrid NN-HMM model for speech recognition. Acoustics, Speech and
Signal Processing 7(2012).
2 Mosavi A. :Varkonyi-Koczy
A. R. : Integration of Machine Learning and Optimization for Robot Learning.
Advances in Intelligent Systems and Computing (2017).
3 Bannister A.: Biometrics
and AI: how FaceSentinel evolves 13 times faster thanks to deep learning
4 Bengio, Y.: Learning deep
architectures for AI. Foundations and trends in Machine Learning 2 (2009).
Liu W Deep learning hashing for mobile visual search.
EURASIP Journal on Image and Video Processing 17, (2017).
6 Mosavi, A.,
Varkonyi-Koczy, A. R., Fullsack, M.: Combination of Machine Learning and
Optimization for Automated Decision-Making. MCDM (2015).
Hinton G E, Simon O, Yee-Whye T A fast learning
algorithm for deep belief nets. Neural computation 18, 1527-1554 (2006).
8 Miotto R et al (2017)
Deep learning for healthcare: review, opportunities and challenges. Briefings
9 Bengio Y.: Fundamentals
of deep learning of representations, Tel Aviv university (2014).
10 Mosavi, A., Vaezipour, A.: Reactive Search
Optimization; Application to Multiobjective Optimization Problems. Applied
Mathematics 3, 1572-1582 (2012)
11 Lee T.: David M
Hierarchical Bayesian inference in the visual cortex. JOSA 20, 1434-1448
12 Lee J-G (2017) Deep
Learning in Medical Imaging: General Overview. Korean Journal of Radiology
13 Marra F.: A Deep Learning
Approach for Iris Sensor Model Identification. Pattern Recognition Letters
14 Mosavi, A. Rabczuk, T.:
Learning and Intelligent Optimization for Computational Materials Design
Innovation, Learning and Intelligent Optimization, Springer-Verlag, (2017)
15 Schuster, Mike; Johnson,
Melvin; Thorat, Nikhil “Zero-Shot Translation with Google’s Multilingual
Neural Machine Translation System” (2016).
16 R. Vargas, A.
Mosavi ,L. Ruiz : Deep Learning: A Review, Advances in Intelligent Systems and Computing, (2017).