Systèmes robotisés intelligents

Smart Robotic Systems

Sensors and Variables Estimation

Gilles TAGNE
[email protected]

1 1
Sensors and variables estimation

Summary:

1. Sensors for mobile robots

2. Variables estimation
3. Multi-sensor fusion

2 2
Sensors and variables estimation

Purpose:

Usual problem in robotics:

• Where am I? // In which configuration am I?
• Where should I go? // What configuration
should I reach?
• How to get there?

➔ We need sensors to get data and we need

some algorithms to use them!

3 3
Sensors for mobile robots
We have two class of sensors:

• Proprioceptive sensors: provide information on the

internal state of the robot
➔ Example: get the relative position of the robot.

• Exteroceptive sensors: provide information on the state

of the environment
➔ Example: get the absolute position of the robot.

Sensors can be:

• Passive / Active
• Analog / Digital
• Logic / Smart

4 4
Sensors for mobile robots
Classification example:

5 5
Sensors for mobile robots
Sensors for position measurement:

▪ Odometry sensors:
Use encoders to measure the rotation of the wheels and/or
steering angles.
Pros: proprioceptive sensor → always able to give an
estimation of the position.
Cons: accumulation of error.

▪ Orientation sensors:
Use gyroscopes or accelerometers to measure speed of rotation
or accelerations.
Pros: proprioceptive sensor.
Cons: drift over time (error integration)

6 6
Sensors for mobile robots
Odometry sensors
Measurement of wheel speed rotation
➔ Estimation of the robot's displacement
Most common daily use: dead-reckoning (estimation of the
position of your car in a tunnel into which you can’t receive GPS)

Two types of odometry sensors:

• Relative optical encoder

• Absolute optical encoder

7 7
Sensors for mobile robots
Optical encoders

How does it work?: a ray of light aim at a

photosensor and is periodically interrupted by a
perforated disc that turn with the shaft of the motor.

➔ Measuring the position consists in

counting the number of impulses.
(example: step motor)

8 8
Sensors for mobile robots
Relative optical encoders
Those encoders usually measure the speed of rotation from which we can
easily retrieve the position.

Same principle than the previous example but with two engraved tracks
shifted by 1/4th of the period → allow to get the direction of rotation and to
improve the resolution therefore the precision. (2 tracks = resolution x4)

9 9
Sensors for mobile robots
Relative optical encoders

10 10
Sensors for mobile robots
Relative optical encoders

A compromise has to be made between:

- low resolution → high noise especially at low speed
- high resolution → need high frequency counting electronics

Pros:
- Low cost (but price increase with the resolution)
- Few cables (usually 5)
- Small size

Cons:
- No absolute position → need a recalibration procedure!

11 11
Sensors for mobile robots
Absolute optical encoders
Same use of light through a wheel, but with an engraved binary code.

Each position of the axis is a number on Nbits ➔ absolute position!

12 12
Sensors for mobile robots
Absolute optical encoders
Be careful about which kind of encoder wheel you have!

13 13
Sensors for mobile robots
Absolute optical encoders
Be careful about which kind of encoder wheel you have!

« Natural » binary VS Gray code

14 14
Sensors for mobile robots
Absolute optical encoders
Each position is on Nbits ➔ Sensor resolution = 2N

Each new track on the wheel ➔ Resolution x2 but price ≈x4 !

Pros:
- Absolute position

Cons:
- More expensive (price increasing with resolution)
- Slightly bigger component
- More cables (one for each track + power)

15 15
Sensors for mobile robots
Resume: Optical encoders for odometry
Pros:
- Low cost
- Easy to integrate

Cons:
- We never know exactly the diameter of the wheel (tire
pressure, ground condition…)
- Wheel drift on accelerations/brakings

➔ Cumulative error on the position estimation!

➔ Need the support of complementary sensors to

correct the localization estimation of a mobile robot.
16 16
Sensors for mobile robots
Resume: Optical encoders for odometry

17 17
Sensors for mobile robots
Resume: Optical encoders for odometry

18 18
Sensors for mobile robots

Inertial Measurement Unit (IMU / Compass)

➔ Widely used to measure accelerations and orientation
(phones, GPS, VR headsets…)

19 19
Sensors for mobile robots

Inertial Measurement Unit (IMU / Compass)

Typically, one sensor per type needed on each axis:
● 6DOF = 3-axis accelerometer + 3-axis gyroscope
● 9DOF = 6DOF + 3-axis magnetometer
● 10DOF = 9DOF + barometric pressure sensor (altitude)

➔Additionnal feature: thermometer for self-calibration.

➔Need to be placed as close as possible to the gravity

center of the robot or its position relatively to this point
must be known precisely for the conception (and use) of
the robot model.

20 20
Sensors for mobile robots

Inertial Measurement Unit (IMU / Compass)

Gyroscopes:
➔Gives angular velocity in degrees/s or rad/s.
➔Has constant bias which is affected by temperature.
➔Bias changes over time (bias stability).

21 21
Sensors for mobile robots

Inertial Measurement Unit (IMU / Compass)

Accelerometer:
➔Gives acceleration in m/s2 or g.
➔At rest an accelerometer measures the gravity vector pointing up.
➔Accurate long term (no drift) but not short term (noise).

22 22
Sensors for mobile robots

Inertial Measurement Unit (IMU / Compass)

Gyroscope:
➔Measures magnetic field strength on each axis in Gauss (unit
of magnetic flux) or µT (unit of magnetic field strength).
➔Points generally towards magnetic north.
➔Can be distorted by nearby metals or electronics.

23 23
Sensors for mobile robots

Inertial Measurement Unit (IMU / Compass)

24 24
Sensors for mobile robots
Telemetry:
Principle: To measure the time of flight to get the distance.
Transmitter and receiver are close so we measure the echo.

Can be done using:

- Sound waves (ultrasound)
- Electromagnetic waves (radio)
- Light (mostly infrared)

25 25
Sensors for mobile robots
Sonar / Ultrasonic sensors:
Proximity and distance measurement
Sound waves that reflect on the obstacles Advantages: Precision,
low cost

Drawbacks: Maximum
frequency of
measurements, cone
angle of opening
➔ Check min and max
range!

26 26
Sensors for mobile robots
Sonar / Ultrasonic sensors:

Directivity patterns for two different sonar sensors.

27 27
Sensors for mobile robots
Radar / Doppler / proximity sensor:
Distance and Speed measurement
Similar to ultrasonic telemeter but with electromagnetic waves (radio)

Advantages: Low cost, Highly

industrialized

Drawbacks: Waves carried by air

therefore unusable at high speed,
aerodynamic effects that deteriorate
the signal, absorption of the signal by
certain materials

Main applications: Parking

assistance

28 10
28
Sensors for mobile robots
Infrared telemeter
Proximity and distance optical measurement

Advantages: Measurement acquisition frequency,

precision, short range
Drawbacks: Low distances <5m (Typically <2m), what
about transparent materials?
29 29
Sensors for mobile robots
LIDAR
• Rotating laser with high acquisition
frequency
• 180/360° Precise distance
measurements
• Mainly used for mapping

Example (characteristics):

30 30
Sensors for mobile robots
LIDAR
Can provide 2D or 3D data in an array of
data or in a point cloud
➔ Depends on the need and the
computational power available (need
to apply a transform to each point to
get their absolute position)
➔ Laser can be problematic with
some surfaces (mirrors, windows…)
➔ Depending on the need there is a
choice to be made between the
frequency of acquisition and the
precision (resolution)

31 31
Sensors for mobile robots
LIDAR

32 32
Sensors for mobile robots
Cameras (Vision)
2D Cameras important features:
- Resolution
- RBG // BW or Grayscale // IR (need
infrared light emitters for night vision or a very
sensitive sensor for thermal vision)
- Frame rate (FPS)
- Apperture & Sensitivity
- Fixed / Adjustable focal length
- Rolling or global shutter
- Connectivity (USB? CSI? Other?)
- CMOS / CCD
- Horizontal & Vertical FOV

33 33
Sensors for mobile robots
Cameras (Vision)
2D Cameras important features:

• “Eye to hand” → “Caméra deportée”

• “Eye in hand” → “Caméra embarquée”

≠
34 34
Sensors for mobile robots
Cameras (Vision)
2D Cameras variation: Omnidirectional camera
- Use of optical lenses or shaped mirrors to get a 360° picture.
➔ Nice idea but need to use time-consuming computer vision
algorithm to compensate the optical deformation before any
processing of the image…

35 35
Sensors for mobile robots
Cameras (Vision)
2D Cameras variation: Stereovision
➔ Two 2D cameras used to get precisous depth information
= cheap 3D camera!
➔ Based on how our eyes work with our brain
→ The distance between the cameras must be
known precisely!

36 36
Sensors for mobile robots
Cameras (Vision)
2D Cameras variation: Stereovision

Easy and cheap but results

are not very precise…

37 37
Sensors for mobile robots
Cameras (Vision)
3D Cameras:
Time of Flight (TOF) cameras = TOF sensor matrix
= Infrared telemeter marix
➔ Low resolution, often short range but good precision
➔ The higher the resolution or range, the higher the price!
➔ More and more integration into new smartphones
so it might become cheaper and better in a near future…

38 38
Sensors for mobile robots
Cameras (Vision)
3D Cameras:
Structured IR light = projection of an infrared pattern
➔ The distortion of the pattern show the depth variations
➔ Good precision and resolution
➔ Quite unexpensive
➔ The data fusion with a RGB camera gives a RGB-D camera

39 39
Sensors for mobile robots
Cameras (Vision)
3D Cameras:
Structured IR light = Example of the D415 and D435
Intel RealSense cameras

40 40
Sensors for mobile robots
Cameras (Vision)

41 41
Sensors for mobile robots
Cameras (Vision systems)
Advantages:
• Usually cheap and easy to use
• Allows customed 2D/3D data depending on the need
• Combined with computer vision algorithms it can give a lot of
information about the environment around the robot and allow it to
achieve many tasks (tracking a target, recognize object or faces,
detect patterns…)

Drawbacks:
• The form factor of some camera may not be easy to integrate in
constrained environment
• Useless without computer vision algorithms that need some
computation power to run in real time (if possible)
• Those algorithms can provide false positive detections

42 42
Sensors for mobile robots
GNSS* sensors:
GPS (Global Positioning System)
- Measuring the absolute position of a point in a
fixed landmark (the center of the earth)
- 3 satellites signals needed (4 is more robust)
- Only for outdoor navigation
→ Useful for position recalibration
- Differential GPS (DGPS) → 1-3 cm precision

Advantages: No drift
Drawbacks: Low frequency <5Hz (typically 1Hz),
unavailability of satellite signals (indoor, under water…)
*Global Navigation Satellite System-
43 43
Sensors for mobile robots
Other sensors:
A lot of sensors exist for all your needs !

44 44
Robot components: sensors
What levels of processing for those sensors ?
➔ How much processing power? Which board process the
data?
Low

Electronics: the sensor which detects bumping into obstacles

=> binary data or analog value through ADC.

Signal Processing: microphone sensor for recognizing a voice

=> filtering, clipping…

Computation: a camera to find your best friend at the party

High => algorithm, machine learning…

Low Medium High

45 45
Sensors for mobile robots
Classification example:
• Analog sensors
• Digital sensors
• Logic sensors

Exercise: Build an array

sensors of the following
robots :
• Pepper
• Turtlebot 3 Burger
• Turtlebot 2i
• ISEN Mobile robot

46 46
Variables estimation
What need to be estimated?
• Noise from the data of a sensor (odometry, IMU) or
from any other component (resistance, motor…).

Why?
• Data is too noisy → Impossibility to measure precisely
• Very expensive sensor so we want the best precision
and we need to remove precisely the noise

How to estimate?
• By developing a mathematical estimator
• By filtering the data (Kalman filter)
• Combining or merging many information

47 47
Variables estimation
Mathematical estimator:

We make several measure with the sensor and notice some

fluctuations on the data → there is noise!

Most of the time we use a linear model:

𝑦 =𝑎𝜃+ 𝜔
with: 𝑦 = measured data
𝑎 = known parameter (gain...)
𝜃 = real sensor value
𝜔 = random noise

We will use statistics to characterize the random noise

➔ Hypothesis: 𝜔 is a non zero mean Gaussian white noise

This is a simple example but building an estimator of a system is

a really complex task that needs a lot of data and math skills!
48 48
Variables estimation
Kalman Filter:

Widely used for filtering random noise.

Use the mathematical model of the system and more statistics!
The equations of the Kalman Filter algorithm may seem complex but the most
important is to know how it works and to know how to use it!

Based on 3 simple steps:

1. Prediction: Based on the mathematical model we
estimate the value that has yet to be measured.
2. Measure: We get the raw data read by the sensor.
3. Correction: We calculate the difference between
the estimation and the measurement, and we
update the mathematical model.

The recursive use of this algorithm over time ensure the error to
be null over time as the error is removed at each step.

49 49
Variables estimation
Kalman Filter:

Other existing similar algorithms

depending on your need and your
available computation power:
- Extended Kalman Filter
- Unscented KF
- Particle Filter
…

➔ More to be seen in the chapter

on Localization!

50 50
Multi-sensor data fusion

Purposes of data fusion:

• More robust estimation of the robot state
• Analysis of the robot near environment
• Planning for future actions

Data fusion examples:

• Odometry + GPS = Robust vehicle localization
• Odometry + IMU = Robust simple position for small mobile robots
• Camera + Odometry + IMU + US + Lidar = Development of the
robot dynamic local map (SLAM)

51 51
Multi-sensor data fusion
Data fusion Methods:
• Probability theory
• Theory of possibility
• Theory of fuzzy sets (fuzzy logic)
• Theory of belief functions
• Decision trees
• Rule bases
• Nearest neighbors
• Neural networks
• Bayesian networks
• Markov chains
• EFK
• UKF
• Particle Filtering
• … Kalman Filter!
52 52
Multi-sensor data fusion
Application of the Kalman Filter to data fusion:
The odometry data (red)
accumulate error over
time.

The fusion of the

odometry and IMU data
provide a much better
result but still has a huge
error in the end.

➔ Data fusion is not

magical!
➔ It helps to get more
reliable data from
sensors but absolute
positionning is always
required for precision!
53 53
Multi-sensor data fusion
Application of the Kalman Filter to data fusion:
The same way we considered our sensors data to be noisy,
we can merge two data from different sources by
considering that each one has some statistic error.
➔ A lot of applications: position and LIDAR data fusion

54 54
Multi-sensor data fusion
Application of the Kalman Filter to data fusion:
The same way we considered our sensors data to be noisy,
we can merge two data from different sources by
considering that each one has some statistic error.
➔ A lot of applications: 2D image and Depth data fusion

55 55
Multi-sensor data fusion
Application of the Kalman Filter to data fusion:
➔ A lot of applications: 3D image + position = 3D map

56 56
 Next steps…
• Locomotion
• Localization
• Trajectory planning
• Robot Control
• Power supply
• Practical work

57 57