Scribd
Upload
70 views

Neural

This document provides an overview of artificial neural networks (ANN) and their application to pattern classification tasks. It discusses how ANNs can be used to classify lidar data from NASA into different land cover types like sky, vegetation, etc. in order to more compactly transmit the data. Specific topics covered include: - How biological neural networks inspired ANNs and how they are well-suited for problems without algorithmic solutions. - An example of using ANNs to determine the curve of a road from an image for autonomous vehicle applications. - The basic theory behind ANN layers and neurons. - Unsupervised learning techniques like self-organizing maps (SOM) to find patterns in

Uploaded by

depa
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
70 views

Neural

This document provides an overview of artificial neural networks (ANN) and their application to pattern classification tasks. It discusses how ANNs can be used to classify lidar data from NASA into different land cover types like sky, vegetation, etc. in order to more compactly transmit the data. Specific topics covered include: - How biological neural networks inspired ANNs and how they are well-suited for problems without algorithmic solutions. - An example of using ANNs to determine the curve of a road from an image for autonomous vehicle applications. - The basic theory behind ANN layers and neurons. - Unsupervised learning techniques like self-organizing maps (SOM) to find patterns in

Uploaded by

depa
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 35

Artificial Neural

Network Theory
and Application
Ashish Venugopal
Sriram Gollapalli
Ulas Bardak
Presentation Overview

 Artificial Neural Networks


 NASA profiles as a Pattern
Classification Task
 Implementing Pattern Classification
on the Silicon Recognition Neuron
Board
ANN General Overview

 Inspired by biological neuron models


 Decision are distributed throughout
system
 Components interacts at several
stages
 Components work in unison to solve
one problem
 ANN well suited to problems that don’t
have closed form algorithmic solution
 Black box model – hard to interpret
ANNs Applied

 Driving a car. Task involves


determining the direction the road is
curving based on an image
 Image pixels are input
 Curve angle is output
 Learned non-linear relationship
between pixels and roads
• hard to understand the model
Theory

 Input layer, Hidden layer, Output


Layer
 Input – one “neuron” for each pixel
 Hidden – set of neurons to store what
was learned.
 Output layer – one neuron for each of
5 turn directions – ie straight, slight
left, hard right..
 But what is a neuron?
Neurons

 Math terms: Binary decision makers


 English terms: take in some input, decide
whether is “high” or “low”, and pass this
value on.
 Similar to neurons in the brain propagating
signals.
How is the model learned?

 Once you have training examples


 Pair (feature set, classification)
 Run the back propagation algorithms
 Learns connection weights between
neurons, as well as within neuron
transfer weights.
 Network topology changes affects
learning behavior!
Unsupervised Networks

 So far we have used labeled training


examples to learn relationship
between features and response
variable.
 What if we are just exploring patterns
that exist in the features?
 Unsupervised learning / clustering…
Unsupervised Goal

 Find prototypical points within data set that


can be used to approximate distribution of
the whole data set.
 Compression / Visualization
 Typical methods
 Define a measure of similarity between data
points
 Select initial prototypes
 Move prototypes to best fit the data –
minimize error
Kohonen Net

 Unsupervised technique

 Each output neuron represents a


prototype point, input layer is
presented with data
Self Organizing Map

 Each prototype is pulled towards that


data, and brings its closest prototypes
a little closer as well…
 SOM Demo
How is it relevant to our
problem?
 We will go through:
 Definition of what we are facing
 How we can use neural nets
 How we can improve on this method
LIDAR in Space

 Data from LITE project


 Send a laser beam down from a
shuttle and get reflection readings.
 Data gathered for 53 hours in 1994.
 40GB of data
Input
Problem

 There is too much data to send and


process
 Each pixel is a number to be
transmitted.
 Each reading (one column) contains
3000 pixels.
 Need a way to transmit the
information in a more compact
fashion.
Applying Neural Nets

 Some readings are very similar to


each other.
 Define classes that will contain
multiple readings.
 Define a representative for the class
which is close enough to all the class
members.
Applying Neural Nets (cont.)

 In order to get class


 Train Kohonen Nets on data for
specific number of classes.
 Once trained, just pass in a new
reading (3000 data points) and get the
class id. Just transfer the class id.
 Go from 3000x3000 numbers to be
transferred to 3000 numbers
Example
Result – 8 classes
Result – 16 Classes
Result – 64 Classes
Result – 128 Classes
Result – 256 Classes
Problems

 Takes a while to train


 256 classes took 44 hours on a dual
P3 1GHz computer with 1.5GB of
RAM.
 Not optimal
 Classifying by the complete reading is
wasteful.
• Need one number for each reading
Improving NN
sky
Improving NN (cont.)

 Easier version:
 Look at blocks of 50x50 pixels instead
of just 1 pixel.
 Classify as sky, etc.
 Can preprocess using kNN, basic
filters to get rid of noise in data.
 Classes can be formed from
sequence of 50 pixel blocks.
Improving NN (cont.)

 Harder version:
 Use variable sized mesh
• In parts with just the sky use a big block.
• In detailed parts use smaller blocks.
 Something to think about…
Muren Board

 Silicon Recognition, Inc.


 2 ZISC078 Chips (156 neurons in
parallel)
 1 Mb of memory

 Up to 1 million recognitions/second
ZISC

 ZISC, the cornerstone of the Muren board,


completely overcomes the three major
limitations of software-based pattern
recognition applications:
 ZISC fully handles non-linearity and
fuzziness
 ZISC is a fully massively parallel processing
system. Recognition speed is consistent,
regardless of the number of stored patterns
 ZISC does not use any algorithms for
programming the solution, ZISC is taught
RBF Space mapping
Example

Source picture Result (after processing)


(circuit board) (shows good and bad points)
Process

 Learning
 Classifying

 Training

 Testing with new data


NASA Data
Further investigation

 Can we write directly to the board?


 Ability to preprocess data?
Questions?

You might also like