Digital : An Ingression into Deep Learning_FP

In ____________, information must flow from input to output only in one direction.

Feed-Forward artificial

Hiden

H2O.ai offers built-in integration tools for which platform?

All

tool supports the use of GPU.

turi

Machine Learning is inspired by the structure of the brain.

false

Which of the following is a type of Recurrent Neural Network?

All

Autoencoders are trained without supervision.

true

As the amount of data increases, the performance of Machine Learning algorithm

deIncrease

The following are tools offered by deepnet platforms, except

H2o

Every neuron in the Input Layer represents a/an _______________ variable that influences the
output.

Indepent

There is no exact formula for calculating the number of hidden layers, as well as the number of
neurons in each hidden layer.

True
introduces the concept of memory cell which can process data within time lags.

Autoend

____________ is a technique which helps machines to mimic human behavior.

Deep learning

Machine Learning is a subset o

Deep

Prelude

Have you ever wondered

How google translates an entire web page to a different language in a matter of seconds? Or,
how your phone gallery groups images into different folders based on their locations?

All these are innovations of Deep Learning! But, what exactly is Deep Learning? Explore to
know more!

Deep Learning

The Big Picture!

Deep Learning is a subset of Machine Learning, which in turn, is a subset of Artificial
Intelligence.

1. Artificial Intelligence

Artificial Intelligence (AI) is a technique that helps machines to mimic human behavior.

2. Machine Learning

Machine Learning is an application of AI that allows the system to learn and improve from
experience automatically.

3. Deep Learning

Deep Learning is a type of Machine Learning that is inspired by the structure of the brain. It
is also known as Artificial Neural Network (ANN). It uses complex algorithms and deep
neural networks to train models.

What is Deep Learning?

Definition
Deep Learning involves networks which are capable of learning from data and functions
similar to the human brain.

Why Deep Learning?

Let us explore the reasons which make Deep Learning shine in the industry.

 Processes massive amount of data

Deep Learning can process an enormous amount of both Structured and Unstructured
data.

 Performs Complex Operations

Deep Learning algorithms are capable enough to perform complex operations when
compared to the Machine Learning algorithms.

Why Deep Learning?

 Achieves Best Performance

o As the amount of data increases, the performance of Machine Learning
algorithms decreases.
o On the other hand, Deep Learning maintains the performance of the model.
 Feature Extraction
o Machine Learning algorithms extract patterns from labeled sample data,
while Deep Learning algorithms take large volumes of data as input, analyze
them to extract the features on its own.

Deep Learning vs. Machine Learning

Say, for instance, we develop a machine that differentiates between cherries and tomatoes.

Machine Learning
If done through Machine Learning, we need to specify the features based on which the two
can be differentiated like size and stem, in this case.

Deep Learning

In Deep Learning, the features are picked by the Neural Network without any human
intervention. But, that kind of independence can be achieved by a higher volume of data in
training the machine.

A Curtain Raiser

Watch this video to know more about Deep Learning

 The human brain contains billions of cells called Neurons. The structure of a neuron
is depicted in the above image.
 Neural Networks is a set of algorithms designed to learn the way our brain works.

 The biological neurons inspire the structure and the functions of the neural networks.

Biological and Artificial Neurons - Terminologies

Biological Neuron Artificial Neuron

Dendrites Inputs
Nucleus Nodes
Synapse Weights
Axon Output

A Node is also called a Neuron or Perceptron.

Structure of an Artificial Neural Network

The basic structure of an Artificial Neural Network (ANN) consists of artificial neuron that
are grouped into 3 different layers namely:

There are three different layers in a neural network, namely:

1. Input Layer
2. Hidden Layer
3. Output Layer

Input Layer

 The input layer communicates with the external environment and presents a
pattern to the neural network.
 Every neuron in the input layer represents an independent variable that influences
the output.

Structure of an Artificial Neural Network

Hidden Layer

 A Neural Network consists of several hidden layers, each consisting of a collection

of neurons.
 The hidden layer is an intermediate layer found between the input layer and the
output layer.
 This layer is responsible forextracting the features required from the input.
 There is no exact formula for calculating the number of the hidden layers as well as the
number of neurons in each hidden layer.

Output Layer

 The output layer of the neural network collects and transmits information in the
desired format.

Structure of an Artificial Neural Network

Every single layer is made up of one or more number of neurons. A simple model of an
Artificial neuron is shown in the figure.
Parameters represented are:

1. X1,X2,...,XmX_1,X_2,...,X_mX1,X2,...,Xm are the input signals.

2. W1,W2,...WmW_1,W_2,...W_mW1,W2,...Wm are the weights.
3. ∑\sum∑ is the Summation function
4. σ\sigmaσ is the Activation function.
5. bkb_kbk represents the bias value

Artificial Neural Network - Process Flow

The process flow depicted in the image is explained as follows:

 A node is where the computations happen

 A node combines the Input data X1,X2,...,XmX_1,X_2,...,X_mX1,X2,...,Xm with a set of
Weights W1,W2,...,WmW_1,W_2,...,W_mW1,W2,...,Wm which in turn aids in amplifying
or diminishing the input with respect to the polarity attached to it.
 On multiplying the inputs with weights, a specific significance is assigned to the inputs with
regard to the task the algorithm is learning. It helps in answering the question Which
input is the most helpful in learning data without error?

 The input-weight products are summed up using the Summing function, and the output
yielded is passed to the Activation function to decide whether and to what extent the
signal should progress to have an effect in the final output.

Summation and Activation Functions

Summation Function

 Various inputs are multiplied with their respective connection weights and summed up
together with the bias value.

∑j=1NXjWj+bk\sum_{j=1}^{N}X_jW_j+b_k∑j=1NXjWj+bk

where:

XjX_jXj represents the inputs

WjW_jWj represents the weights

Summation and Activation Functions

Activation Function

 Activation Function aids in deriving the output.

 It is also known as the Transfer Function.

 It maps the resulting values between 0 to 1 or -1 to 1.

 There are two types of Activation function, namely:

1. Linear Activation Function

2. Non-Linear Activation Function
 Non-Linear Activation Function is the most commonly used Activation function
in Neural Networks.

Non-Linear Activation Functions

Watch this video to know some of the most commonly used Activation Functions in Neural
Networks.

The default Activation Function is ReLU.

Working of a Neural Network

Watch this video to know the working of a Neural Network.

In January 2018, Google's CEO Sundar Pichai claimed that Artificial intelligence (AI) would be more
transformative to humanity than electricity.

Prelude
Curious about how weight and bias values are decided? This section will help you in
understanding the concepts of the learning process in Neural Network.

Learning Process of a Neural Network

The learning process of a Neural Network includes updating the network architecture and
connecting the weights for the network to perform efficiently.

Designing a Learning Process involves the following:

1. Learning Paradigm

Having a model from the environment in which Neural Network works.

2. Learning Rules

Figuring out the rules that aid in updating the weights.

3. Learning Algorithms

Identifying the procedure to update the weights according to the learning rules.

Learning Paradigm

The following are the various Learning Paradigms in Neural Networks:

 Supervised
 Unsupervised
 Reinforcement

Learning Rules

The four basic types of Learning Rules in Neural Network are:

 Error Correction Rules

 Hebbian
  Boltzmann
 Competitive Learning

Learning Algorithms

Following are a few Deep Learning algorithms:

 Adaptive Resonance Theory

 Kochen Self Organization Map
 ADALINE
 Perceptron
 Backpropagation

DRC-Hubo

View the following video on DRC-Hubo developed by Team KAIST from South Korea which
impresses us by remarkably precise and deliberate motion.

Principles

Principles

A Neural Network works based on two principles, namely:

1. Forward Propagation
2. Backward Propagation

Forward Propagation in Neural Network

Explore this video to know about the Forward Propagation.

Backward Propagation in Neural Network

Explore this video to know about the Backward Propagation.

Prelude

In this section, let us discover the Network Architecture that prevails in Neural Network.

Neural Network Architecture

The architecture of a Neural Network can be broadly classified into two, namely:

1. Feed Forward Artificial Neural Network

2. Recurrent Neural Network

Feed Forward Artificial Neural Network

 The information must flow from input to output only in one direction.
 No Feedback loops must be present.

A few Feed Forward Artificial Neural Networks are:

 Single-layer Feed Forward Network

 Multi-layer Feed Forward Network

Recurrent Neural Network

 Information can be transmitted in both directions.

 Feedback loops are allowed.
A few Recurrent Neural Networks are:

 Hopfield Network
 Elman Network
 Jordan Network

Explore More!

The following video will help you know more about the Feed Forward Networks and
Recurrent Neural Networks in mathematical terms.

MELTANT-α\alphaα is an avatar robot with the world’s most powerful and flexible robotic hand
which directly imitates human hand movements.

Prelude

Neural Networks are employed in various fields. Following are a few types of Neural
Networks that we will explore in this section:

 Radial Basis Function

 Long/Short Term Memory
 Gated Recurrent Unit
 Autoencoder
 Convolutional Neural Networks

Radial Basis Function

Radial Basis Function is similar to Feed Forward Network but uses Logistic function as the
Activation Function.

Check out this video to know more!

Long/Short Term Memory

  Long/Short Term Memory (LSTM) networks are used to classify, process, and make
predictions based on time series data.
 LSTM networks can predict the action in a specific video frame by keeping in mind the
action that occurred in the earlier frames.
 The applications of LSTM include writing, speech recognition, and so on.

Gated Recurrent Unit

 Gated Recurrent Unit (GRU) is LSTM with a forget gate.

 It is used in sound, speech synthesis, and so on.

More on LSTM and GRU

Watch the following video to know more about LSTM and GRU Neural Networks.

Autoencoder

 Autoencoder (AE) is used for classification, clustering, and feature compression.

 It is trained without supervision.

Convolutional Neural Networks

The following video will help you understand the basic concepts of Convolutional Neural
Networks (CNN).

Applications

Image Classification
  Image classification is one of the common applications of deep learning.
 A convolutional neural network can be used to recognize images and label them
automatically.

Object Recognition

Deepnets (an optimized version of Deep Neural Networks) can be trained to recognize different
objects within the same image.

Video Recognition

 Deepnets can also be trained for real-time object recognition with respect to time
and space.
 Some of the famous use cases are driverless cars, robots, and theft detection.

Sentiment Analysis

 Using sentiment analysis, the underlying intent of the text can be extracted.
 With social media channels, it is possible to automate and measure the feelings of the
public on a given news story, topic, brand, or product.

 Positive sentiment can be identified, thereby allowing the marketing of a product, or

understanding which elements of a business strategy are working.

Medical Applications

 Deepnets can be trained to detect cancerous cells, benign and malignant tumors from
MRI and CT scans.
 They are also applied in drug discovery by training nets with molecular structure and
chemical compositions.

Tools

Deepnet Platform
  A deepnet platform is a service that allows you to incorporate deepnet in your
applications without building one from scratch.
 This platform provides a set of tools and interfaces to create a custom deepnet.

Deepnet platforms are of two types, namely:

 Software platform: This platform is available as downloadable packages that need to be

deployed on your hardware.
 Full platform: It is available as an online interactive UI to build and deploy models
without any coding experience.

More on Deepnet Platform

The following video explains what exactly is a deepnet platform.

Deepnet Platforms - Tools

The following are the tools offered by the deepnet platforms:

 Deepnet capability
 Data Munging
 UI/Model Management
 Infrastructure

H2O.ai

 H2O.ai is an open-source machine learning platform.

 Along with many machine learning algorithms, this platform currently provides a
deepnet capability known as Multilayer Perceptron (MLP).

Features

 H2O.aiH_2O.aiH2O.ai supports MLP for a deep learning model, which uses the L-
BFGS algorithm for backpropagation.
 H2O.aiH_2O.aiH2O.ai offers built-in integration tools for platforms like HDFS,
Amazon S3, SQL, and NoSQL.
  Its intuitive UI can be accessed by programming environments like R, Python, and
JSON.
 You can model and analyze data with Tableau, Microsoft Excel, and RStudio.
 Since the package needs to be downloaded and deployed in the hardware,
H2O.aiH_2O.aiH2O.ai comes with an in-memory map-reduce, and columnar
compression to accelerate the training of models.

Turi - A Graph-based Tool

Turi is a software platform that provides two deepnet capabilities (convolutional network
and MLP) along with machine learning and graph algorithms.

Features

 Turi provides built-in support for integration of Amazon S3, SQL DB, HDFS, Spark RDD, and
Pandas data frames.
 It offers a UI interface for model management and also includes a visualization tool called
GraphLab Canvas for visualizing the model results.

 To work with large data, Turi comes with built-in storage support like SFrame, SArray, and
SGraph.
 Unlike H2O.aiH_2O.aiH2O.ai, Turi supports the use of GPUs.

Turi

Watch this video to know more about the Turi platform.

Course Summary

While we've seen great strides in robotic capability recently, there's still a vast gap between
human and robot motor skills. Even at simple motor skills like gripping, machine still have a
very long way to go to suit human ability.

By linking learning with continuous feedback and control, however, we may begin to bridge
that gap, enabling robots to manage the complexities of the real world intelligently and
reliably.

Replicating the human brain at entirety will not just be a Science fiction episode anymore.
The future indeed will be full of surprises. Keep learning!