1.

Introduction:

In this literature review, my objective is to introduce the deep

learning technology. Very less is heard of deep learning method to solve

engineering problems, this report will answer about exploring Deep learning

method to facilitate big data analysis and its applications to solve the engineering

problems. We will also learn about the

technologies where deep learning method was used and its results.

Daily an enormous amount of sensing devices collects and generate large

amount of data over a period for a wide range of fields and applications. Based

on the nature of the application, these devices will result in real-time data

streams. To analyze this data and transforming it into useful data to do continuous

improvement in science & technology is

the key, which we will understand through this literature survey.

2.

Research methods

I started my literature review with reading articles on deep

learning and big data by various University such as Obuda University( Budapest

Hungary) and Institute of Structural mechanics ( Weimar, Germany). These

articles helped me in understanding the applications of deep learning in

various fields like image processing, biometrics and also in medicine. It is important to know deep learning first

which will make it easier to understand its application on solving engineering

problems. These articles and various other articles are available on the

internet as an opensource articles and has free access. I have mentioned the

applications of deep learning which are present in all the articles. Also I

have shown how the publications on deep learning is increasing day by day with

help of data from science journals.

My aim in this literature review is to answer the following

questions:

1.

What

is deep learning?

2.

What

are problems that have been solved using deep learning?

3.

How

successful is deep learning while applying to various problems and what are its

advantages ?

The research methods that I used are:

1.

Literature Study(Library): It is important to know

the subject thoroughly before writing a report on it. This method help me

understand more in detail about Deep learning.

2.

Scenario(Stepping Stones): Since it’s a latest technology development, current scenario and

applications are useful to demonstrate advantages of deep learning.

3.

Concept, Requirement List and Risk analysis(Stepping Stones):By using this methods of research we can learn:

what was the

need of developing deep learning method, its advantages and risks of misusing

the big data as it involves artificial intelligence.

Highly confidential data can also be misused

while evaluating and understanding sensory data. Risk analysis and protecting

privacy & security is also important to ensure ethical use of this

technology.

In most of the articles, it can be seen the efforts to make the

machine learning more advanced from designing control system to a level that it

can make its own decisions i.e. artificial intelligence.

3.

Main body of text

3.1 Introduction:

Artificial intelligence (AI) is the intelligence demonstrated by

machines which is an effective approach to human learning and reasoning 1. In

1950, “The Turing Test” was proposed as an good explanation of how a computer

could perform a human cognitive reasoning 2. AI can be divided into specific

research sub-fields. For example: Natural Language Processing (NLP) 3 can

enhance the writing experience in various applications 4,5. The most important

part of NLP is machine translation, which is the translation between languages.

Machine translation algorithms aides applications which can consider grammar

structure as well as spelling mistakes. Also computer suggests words to writer

or editor to make changes 6. Figure 1 shows how AI covers seven subfields of

computer sciences.

Recent studies in big data and machine learning analyzes

multiple possibilities of characterization of databases 7. For many the

years, databases are collected for statistical purposes. Statistical curves can

describe past, and present in order to predict future behaviors. However the

techniques developed to process this data by algorithms is very recent, an

optimization of those algorithms could lead on an effective self–learning 8. Decision

making by robots can be implemented based on existing values, multiple criteria

and statistics advanced methods.

Figure 1. Artificial intelligence

and its subfields.

3.2 What is deep learning?

Deep learning is an emerging area of machine learning research. It

comprises of multiple hidden layers of artificial neural networks. The deep

learning methodology applies nonlinear transformations and model abstractions

of high level in large databases. The recent development in deep learning

architectures within numerous fields have already provided significant

contributions in artificial intelligence1.

Deep learning method’s ultimate goal is to give rise to artificial

intelligence with prime focus on

mathematical and computational principles to learn from examples to

acquire knowledge9.

Deep learning first appeared in the year 2006 and

it was known as hierarchical learning which is usually used in the fields of

pattern recognition 2. Deep learning consists of mainly two aspects 4:

1.

Nonlinear

processing in multi-layers.

2.

Supervised

and Unsupervised learning.

Nonlinear processing in multiple layers means an algorithm where

the current layer takes the output of the previous layer as an input. Hierarchy

is developed among the layers of data

based on importance criteria. On the other hand, supervised and unsupervised learning is related

with the class target label, its availability means a supervised system,

whereas its absence means an unsupervised system.

3.3

Applications of Deep Learning in engineering problems:

1.

Image Processing:

·

In

2003, experiments were done by applying particle filtering and Bayesian –

belief propagation. The main concept of

this experiment was that if human can detect the face of person by watching

only half cropped photo, a computer can also reconstruct the photo from half

cropped photo10.

·

In

2006, greed algorithm and hierarchy was developed to process handwritten

digits.

·

Applying

Convolutional Neural Networks for iris recognitions increases the accuracy up

to 99.35% 11.

·

Mobile

location recognition nowadays allows the user to know a determined address

based on a picture. A Supervised Semantics – Preserving Deep Hashing (SSPDH)

algorithm has proved a considerable improvement in comparison with Visual Hash

Bit

(VHB) and Space – Saliency Fingerprint

Selection (SSFS). The accuracy of SSPDH is even 70% more efficient 12.

·

Google,

Facebook and Microsoft all have developed face detection as security lock

system for the cell phones this digital image processing is done by the use of

deep learning method.

2.

Biometrics:

·

In

2009, automatic speech recognition application was carried out to reduce the

Phone Error Rate by using two different architectures of deep belief networks

13.

·

DL

was employed to speed up the developing and optimization of FaceSentinel face

recognition devices. There devices could expand the face recognition process

from one -to-one to one -to- many in nine months 14. Without deep learning it

would have taken 10 years. These devices are used at Heathrow airport in London

and have the potential to be used as time and attendance and in banking sector 3.

3.

Natural Language Processing:

Google translate

uses large end-to- end long short-term memory network. It translates the whole sentences

rather than some part of the sentence. It supports over 100 languages 15.

Several important applications of deep learning in the field of speech

recognition and image processing are summarized in the Table -1 16.

Table 1: Deep learning applications 2003-2017

.

3.4

Publications in Deep learning

Figure 2 shows database of springer’s publications in deep learning.

We can see that there is a huge increase in publications which shows the keen interests

of researchers 16.

Figure

2: Data of number of publications in deep learning from

Springer database :2006 to 2017.

3.5

Conclusions:

·

Deep

learning is one of the fastest growing applications of machine learning. Large number

of publications prove that a lot of research is going on in this subject.

·

Hierarchy

of layers and supervision in learning are the two key aspects to develop a good

application of deep learning. Hierarchy is essential for data classification whereas

supervision checks for the importance of the data in the process.

·

Deep

learning has given concrete results in the field of image processing and speech

recognition.

·

Deep

learning is also an important method for designing security tools as it can do facial

recognitions which is already a feature in today’s smartphones.

4.

References

1

Abdel, O. : Applying convolutional neural networks

concepts to hybrid NN-HMM model for speech recognition. Acoustics, Speech and

Signal Processing 7(2012).

2 Mosavi A. :Varkonyi-Koczy

A. R. : Integration of Machine Learning and Optimization for Robot Learning.

Advances in Intelligent Systems and Computing (2017).

3 Bannister A.: Biometrics

and AI: how FaceSentinel evolves 13 times faster thanks to deep learning

(2016).

4 Bengio, Y.: Learning deep

architectures for AI. Foundations and trends in Machine Learning 2 (2009).

5

Liu W Deep learning hashing for mobile visual search.

EURASIP Journal on Image and Video Processing 17, (2017).

6 Mosavi, A.,

Varkonyi-Koczy, A. R., Fullsack, M.: Combination of Machine Learning and

Optimization for Automated Decision-Making. MCDM (2015).

7

Hinton G E, Simon O, Yee-Whye T A fast learning

algorithm for deep belief nets. Neural computation 18, 1527-1554 (2006).

8 Miotto R et al (2017)

Deep learning for healthcare: review, opportunities and challenges. Briefings

in Bioinformatics.

9 Bengio Y.: Fundamentals

of deep learning of representations, Tel Aviv university (2014).

10 Mosavi, A., Vaezipour, A.: Reactive Search

Optimization; Application to Multiobjective Optimization Problems. Applied

Mathematics 3, 1572-1582 (2012)

11 Lee T.: David M

Hierarchical Bayesian inference in the visual cortex. JOSA 20, 1434-1448

(2003).

12 Lee J-G (2017) Deep

Learning in Medical Imaging: General Overview. Korean Journal of Radiology

18(4):570-584.

13 Marra F.: A Deep Learning

Approach for Iris Sensor Model Identification. Pattern Recognition Letters

(2017).

14 Mosavi, A. Rabczuk, T.:

Learning and Intelligent Optimization for Computational Materials Design

Innovation, Learning and Intelligent Optimization, Springer-Verlag, (2017)

15 Schuster, Mike; Johnson,

Melvin; Thorat, Nikhil “Zero-Shot Translation with Google’s Multilingual

Neural Machine Translation System” (2016).

16 R. Vargas, A.

Mosavi ,L. Ruiz : Deep Learning: A Review, Advances in Intelligent Systems and Computing, (2017).