Read Anytime Anywhere Easy Ebook Downloads at ebookmeta.

com

Robotics Vision and Control Fundamental Algorithms

in MATLAB 3rd Peter Corke

https://ebookmeta.com/product/robotics-vision-and-control-
fundamental-algorithms-in-matlab-3rd-peter-corke/

OR CLICK HERE

DOWLOAD EBOOK

Visit and Get More Ebook Downloads Instantly at https://ebookmeta.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Robotics Vision and Control Fundamental Algorithms in

MATLAB 3rd Peter Corke Witold Jachimczyk Remo Pillat

https://ebookmeta.com/product/robotics-vision-and-control-fundamental-
algorithms-in-matlab-3rd-peter-corke-witold-jachimczyk-remo-pillat/

ebookmeta.com

Robotics and Control: Fundamental Algorithms in MATLAB®

1st Edition Peter Corke

https://ebookmeta.com/product/robotics-and-control-fundamental-
algorithms-in-matlab-1st-edition-peter-corke/

ebookmeta.com

Robotic Vision: Fundamental Algorithms in MATLAB® 1st

Edition Peter Corke

https://ebookmeta.com/product/robotic-vision-fundamental-algorithms-
in-matlab-1st-edition-peter-corke/

ebookmeta.com

Rwanda s Popular Genocide A Perfect Storm 1st Edition Jean

Paul Kimonyo

https://ebookmeta.com/product/rwanda-s-popular-genocide-a-perfect-
storm-1st-edition-jean-paul-kimonyo/

ebookmeta.com
ENT Vivas: a Guide to Passing the Intercollegiate FRCS
(ORL-HNS) Viva Examination 1st Edition Adnan Darr

https://ebookmeta.com/product/ent-vivas-a-guide-to-passing-the-
intercollegiate-frcs-orl-hns-viva-examination-1st-edition-adnan-darr/

ebookmeta.com

Evidence Based Critical Care A Case Study Approach Robert

C. Hyzy

https://ebookmeta.com/product/evidence-based-critical-care-a-case-
study-approach-robert-c-hyzy/

ebookmeta.com

Psychological Type Therapy A Practitioner s Guide to

Strengthening Relationships 1st Edition Brian A. Gerrard

https://ebookmeta.com/product/psychological-type-therapy-a-
practitioner-s-guide-to-strengthening-relationships-1st-edition-brian-
a-gerrard/
ebookmeta.com

Victorian Science and Imagery Representation and Knowledge

in Nineteenth Century Visual Culture Sci Culture in the
Nineteenth Century 1st Edition Nancy Rose Marshall
(Editor)
https://ebookmeta.com/product/victorian-science-and-imagery-
representation-and-knowledge-in-nineteenth-century-visual-culture-sci-
culture-in-the-nineteenth-century-1st-edition-nancy-rose-marshall-
editor/
ebookmeta.com

3D Bioprinting and Nanotechnology in Tissue Engineering

and Regenerative Medicine 2nd Edition Lijie Grace Zhang

https://ebookmeta.com/product/3d-bioprinting-and-nanotechnology-in-
tissue-engineering-and-regenerative-medicine-2nd-edition-lijie-grace-
zhang/
ebookmeta.com
Basics of Pain Management Second Edition Gautam Das Md

https://ebookmeta.com/product/basics-of-pain-management-second-
edition-gautam-das-md/

ebookmeta.com
 Springer Tracts in Advanced Robotics 147

Peter Corke
Witold Jachimczyk
Remo Pillat

Robotics,
Vision
and
Control Third Edition
FUNDAMENTAL
ALGORITHMS
IN MATLAB®
Springer Tracts in Advanced Robotics
Volume 147

Series Editors
Bruno Siciliano, Dipartimento di Ingegneria Elettrica e Tecnologie
dell’Informazione, Università degli Studi di Napoli Federico II,
Napoli, Italy
Oussama Khatib, Artificial Intelligence Laboratory, Department of
Computer Science, Stanford University, Stanford, CA, USA

Advisory Editors
Nancy Amato, Computer Science & Engineering, Texas A&M
University, College Station, TX, USA
Oliver Brock, Fakultät IV, TU Berlin, Berlin, Germany
Herman Bruyninckx, KU Leuven, Heverlee, Belgium
Wolfram Burgard, Institute of Computer Science, University of
Freiburg, Freiburg, Baden-Württemberg, Germany
Raja Chatila, ISIR, Paris cedex 05, France
Francois Chaumette, IRISA/INRIA, Rennes, Ardennes, France
Wan Kyun Chung, Robotics Laboratory, Mechanical Engineering,
POSTECH, Pohang, Korea (Republic of)
Peter Corke, Queensland University of Technology, Brisbane, QLD,
Australia
Paolo Dario, LEM, Scuola Superiore Sant’Anna, Pisa, Italy
Alessandro De Luca, DIAGAR, Sapienza Università di Roma, Roma,
Italy
Rüdiger Dillmann, Humanoids and Intelligence Systems Lab, KIT -
Karlsruher Institut für Technologie, Karlsruhe, Germany
Ken Goldberg, University of California, Berkeley, CA, USA
John Hollerbach, School of Computing, University of Utah, Salt
Lake, UT, USA
Lydia E. Kavraki, Department of Computer Science, Rice University,
Houston, TX, USA
Vijay Kumar, School of Engineering and Applied Mechanics,
University of Pennsylvania, Philadelphia, PA, USA
Bradley J. Nelson, Institute of Robotics and Intelligent Systems,
ETH Zurich, Zürich, Switzerland
Frank Chongwoo Park, Mechanical Engineering Department,
Seoul National University, Seoul, Korea (Republic of)
S. E. Salcudean, The University of British Columbia, Vancouver, BC,
Canada
Roland Siegwart, LEE J205, ETH Zürich, Institute of Robotics &
Autonomous Systems Lab, Zürich, Switzerland
Gaurav S. Sukhatme, Department of Computer Science, University
of Southern California, Los Angeles, CA, USA
The Springer Tracts in Advanced Robotics (STAR) publish new developments and advances in the
fields of robotics research, rapidly and informally but with a high quality. The intent is to cover all the
technical contents, applications, and multidisciplinary aspects of robotics, embedded in the fields of
Mechanical Engineering, Computer Science, Electrical Engineering, Mechatronics, Control, and Life
Sciences, as well as the methodologies behind them. Within the scope of the series are monographs,
lecture notes, selected contributions from specialized conferences and workshops, as well as selected
PhD theses.
Special offer: For all clients with a print standing order we offer free access to the electronic volumes
of the Series published in the current year.
Indexed by SCOPUS, DBLP, EI Compendex, zbMATH, SCImago.
All books published in the series are submitted for consideration in Web of Science.
Peter Corke  Witold Jachimczyk  Remo Pillat

Robotics, Vision
and Control
Fundamental Algorithms in MATLAB®

3rd edition 2023

Peter Corke Remo Pillat
Queensland University of Technology MathWorks
Brisbane, QLD, Australia Natick, MA, USA

Witold Jachimczyk
MathWorks
Natick, MA, USA

ISSN 1610-7438 ISSN 1610-742X (electronic)

Springer Tracts in Advanced Robotics
ISBN 978-3-031-07261-1 ISBN 978-3-031-07262-8 (eBook)
https://doi.org/10.1007/978-3-031-07262-8

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzer-
land AG 2011, 2017, 2023
Previously published in two volumes
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or
information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar
methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publica-
tion does not imply, even in the absence of a specific statement, that such names are exempt from the
relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or
the editors give a warranty, express or implied, with respect to the material contained herein or for any
errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
 v

To my family Phillipa, Lucy and Madeline for their indulgence

and support;
my parents Margaret and David for kindling my curiosity;
and to Richard (Lou) Paul who planted the seed that became
this book. – PC

To my wife Halina, my daughter Julia and my son Alec for their

encouragement and support;
my mom Janina and my uncle Alfred for inspiring me to be
an engineer. – WJ

To my beloved wife, Kristi, for her enduring patience and tolerance

for new projects;
to my children Tobin and Evabelle for giving up some play time;
and to my parents Lothar and Birgit for allowing me to pursue
my dreams. – RP
Foreword

Once upon a time, a very thick document of a dissertation from a faraway land
came to me for evaluation. Visual robot control was the thesis theme and Peter
Corke was its author. Here, I am reminded of an excerpt of my comments, which
reads, this is a masterful document, a quality of thesis one would like all of one’s
students to strive for, knowing very few could attain – very well considered and
executed.
The connection between robotics and vision has been, for over three decades,
the central thread of Peter Corke’s productive investigations and successful devel-
opments and implementations. In this third edition of the book on Robotics, Vision
and Control he is joined by Witold Jachimczyk and Remo Pillat from MathWorks,
the publishers of MATLAB. In its melding of theory and application, this third edi-
tion has considerably benefited from the authors’ unique and diverse experience in
academia, commercial software development, and real-world applications across
robotics, computer vision and self-driving vehicles.
There have been numerous textbooks in robotics and vision, but few have
reached the level of integration, analysis, dissection, and practical illustrations
evidenced in this book. The discussion is thorough, the narrative is remarkably
informative and accessible, and the overall impression is of a significant contri-
bution for researchers and future investigators in our field. Most every element
that could be considered as relevant to the task seems to have been analyzed and
incorporated, and the effective use of toolbox software echoes this thoroughness.
The reader is taken on a realistic walk through the fundamentals of mobile
robots, navigation, localization, manipulator-arm kinematics, dynamics, and joint-
level control, as well as camera modeling, image processing, feature extraction,
and multi-view geometry. These areas are finally brought together through exten-
sive discussion of visual servo system and large-scale real-world examples. In the
process, the authors provide insights into how complex problems can be decom-
posed and solved using powerful numerical tools and effective software.
The Springer Tracts in Advanced Robotics (STAR) is devoted to bringing to
the research community the latest advances in the robotics field on the basis of
their significance and quality. Through a wide and timely dissemination of criti-
cal research developments in robotics, our objective with this series is to promote
more exchanges and collaborations among the researchers in the community and
contribute to further advancements in this rapidly growing field.
The authors bring a great addition to our STAR series with an authoritative
book, reaching across fields, thoughtfully conceived and brilliantly accomplished.

Oussama Khatib
Stanford, California
March 2023
 vii

Preface

» Tell me and I will forget.

Show me and I will remember.
Involve me and I will understand.
– Chinese proverb

Simple things should be simple,

complex things should be possible.
– Alan Kay

These are exciting times for robotics. Since the first edition of this book was pub-
lished over ten years ago we have seen great progress: the actuality of self-driving
cars on public roads, multiple robots on Mars (including one that flies), robotic as-
teroid and comet sampling, the rise of robot-enabled businesses like Amazon, and
the DARPA Subterranean Challenge where teams of ground and aerial robots au-
tonomously mapped underground spaces. We have witnessed the drone revolution
– flying machines that were once the domain of the aerospace giants can now be
bought for just tens of dollars. All of this has been powered by the ongoing improve-
ment in computer power and tremendous advances in low-cost inertial sensors and
cameras – driven largely by consumer demand for better mobile phones and gam-
ing experiences. It’s getting easier for individuals to create robots – 3D printing is
now very affordable, the Robot Operating System (ROS) is capable and widely used,
and powerful hobby technologies such as the Arduino, Raspberry Pi, and Dynamixel
servo motors are available at low cost. This in turn has driven the growth of the global
maker community, and empowered individuals working at home, and enabled small
startups to do what would once have been done by major corporations.
Robots are machines which acquire data, process it, and take action based on it.
The data comes from a variety of sensors that measure, for example, the velocity
of a wheel, the angle of a robot arm’s joint, or the intensities of millions of pixels
that comprise an image of the world. For many robotic applications the amount of
data that needs to be processed, in real-time, is massive. For a vision sensor it can
be on the order of tens to hundreds of megabytes per second. Progress in robots
and machine vision has been, and continues to be, driven by more effective ways
to process sensory data.
One axis of progress has been driven by the relentless increase in affordable
computational power. 7 Moore’s law predicts that the number of transistors on When the first author started in
a chip will double every two years, and this enables ever-increasing amounts of robotics and vision in the mid 1980s,
memory, and parallel processing with multiple cores and graphical processing units the IBM PC had been recently released
(GPUs). Concomitantly, the size of transistors has shrunk and clock speed has in- – it had a 4.77 MHz 16-bit
creased. microprocessor and 16 kbytes
The other axis of progress is algorithmic, exploiting this abundance of compu- (expandable to 256 k) of memory. In
tation and memory to solve robotics problems. Over decades, the research com- the 1990s it took a rack full of custom
munity has developed many solutions for important problems in perception, lo- electronics to process video data in real
calization, planning, and control. However, for any particular problem there is a time.
wide choice of algorithms, and each of them may have several implementations.
These will be written in a variety of languages, with a variety of API styles and
conventions, and with variable code quality, documentation, support, and licence
conditions. This is a significant challenge for robotics today, and “cobbling to-
gether” disparate pieces of software has become an essential skill for roboticists.
The ROS framework 7 has helped by standardizing interfaces and allowing com- See 7 https://ros.org.
mon functions to be composed to solve a particular problem. Nevertheless, the
software side of robotics is still harder and more time-consuming than it should
be. This unfortunate complexity, and the sheer range of choice, presents a very real
barrier to somebody new entering the field.
viii Preface

. Fig. 1 Once upon a time a lot of equipment was needed to do vision-based robot control. The first
author with a large rack full of real-time image processing and robot control equipment and a PUMA
560 robot (1992). Over the intervening 30 years, the number of transistors on a chip has increased by a
factor of 230=2  30;000 according to Moore’s law

» the software tools used in this book aim to reduce complexity for the reader

The software tools that are used in this book aim to reduce complexity for the reader
by providing a coherent and complete set of functionality. We use MATLAB® ,
a popular mathematical and engineering computing environment and associated
licensed toolboxes that provide functionality for robotics and computer vision. This
makes common algorithms tangible and accessible. You can read much of the code,
you can apply it to your own problems, or you can extend it. It gives you a “leg up”
as you begin your journey into robotics.
» allow the user to work with real problems, not just trivial examples

This book uses that software to illustrate each topic, and this has a number of
benefits. Firstly, the software allows the reader to work with real problems, not just
trivial examples. For real robots, those with more than two links, or real images
with millions of pixels the computation required is beyond unaided human ability.
Secondly, these software tools help us gain insight which can otherwise get lost
in the complexity. We can rapidly and easily experiment, play what if games, and
depict the results graphically using the powerful 2D and 3D graphical display tools
of MATLAB.
» a cohesive narrative that covers robotics and computer vision – both separately and
together

The book takes a conversational approach, weaving text, mathematics, and lines of
code into a cohesive narrative that covers robotics and computer vision – separately,
and together as robotic vision. It shows how complex problems can be decomposed
and solved using just a few simple lines of code. More formally this is an inductive
learning approach, going from specific and concrete examples to the more general.
» show how complex problems can be decomposed and solved

The topics covered in this book are guided by real problems faced by practitioners
of robotics, computer vision, and robotic vision. Consider the book as a grand tast-
ing menu and we hope that by the end of this book you will share our enthusiasm
for these topics.
» consider the book as a grand tasting menu
 ix
Preface

We were particularly motivated to present a solid introduction to computer vision

for roboticists. The treatment of vision in robotics textbooks tends to concentrate
on simple binary vision techniques. In this book we will cover a broad range of
topics including color vision, advanced segmentation techniques, image warping,
image retrieval, stereo vision, pose estimation, deep learning, bundle adjustment,
and visual odometry. We also cover nonperspective imaging using fisheye lenses,
catadioptric optics, and the emerging area of light-field cameras. These topics are
growing in importance for robotics but are not commonly covered. Vision is a
powerful sensing modality, and roboticists should have a solid grounding in mod-
ern fundamentals. The last part of the book shows how to apply vision and robotics
concepts to solving real-world automated driving and UAV applications.
This book is unlike other textbooks, and deliberately so. While there are already
a number of excellent textbooks that cover robotics and computer vision separately
and in depth, there are few that cover both in an integrated fashion. Achieving such
integration was a principal goal of this book.
» software is a first-class citizen in this book

Software is a tangible instantiation of the algorithms described – it can be read and

it can be pulled apart, modified and put back together again – software is a first-
class citizen in this book. There are a number of classic books that use software
in an illustrative fashion and have influenced this approach, for example LaTeX: A
document preparation system (Lamport 1994), Numerical Recipes in C (Press et al.
2007), The Little Lisper (Friedman et al. 1987), and Structure and Interpretation of
Classical Mechanics (Sussman et al. 2001). In this book, over 2000 lines of code
across over 1600 examples illustrate how the software can be used and generally
provide instant gratification in just a couple of lines of MATLAB code.
» instant gratification in just a couple of lines of MATLAB code

Thirdly, building the book around MATLAB and the associated toolboxes that we
are able to tackle more realistic and more complex problems than other books.
» this book provides a complementary approach

A key motivation of this book is to make robotics, vision, and control algorithms
accessible to a wide audience. The mathematics that underpins robotics is in-
escapable, but the theoretical complexity has been minimized and the book as-
sumes no more than an undergraduate-engineering level of mathematical knowl-
edge. The software-based examples help to ground the abstract concepts and make
them tangible. This approach is complementary to the many other excellent text-
books that cover these same topics but which take a stronger, and more traditional,
theoretical approach. This book is best read in conjunction with those other texts,
and the end of each chapter has a section on further reading that provides pointers
to relevant textbooks and key papers.
The fields of robotics and computer vision are underpinned by theories devel-
oped by mathematicians, scientists, and engineers over many hundreds of years.
Some of their names have become adjectives like Coriolis, Gaussian, Laplacian, or
Cartesian; nouns like Jacobian, or units like Newton and Coulomb. They are inter-
esting characters from a distant era when science was a hobby and their day jobs
were as doctors, alchemists, gamblers, astrologers, philosophers, or mercenaries.
To know whose shoulders we are standing on, the book includes small vignettes
about the lives of some of these people – a smattering of history as a backstory.
Many people have helped with critical comments over previous editions –
this edition is the better for their input and we thank: Paul Newman, Daniela
Rus, Cédric Pradalier, Tim Barfoot, Dmitry Bratanov, Duncan Campbell, Donald
Dansereau, Tom Drummond, Malcolm Good, Peter Kujala, Obadiah Lam, Jörn
x Preface

Malzahn, Felipe Nascimento Martins, Ajay Pandey, Dan Richards, Sareh Shi-
razi, Surya Singh, Ryan Smith, Ben Talbot, Dorian Tsai, Ben Upcroft, François
Chaumette, Donald Dansereau, Kevin Lynch, Robert Mahony, and Frank Park.
Thanks also to all those who have submitted errata. We are grateful to our col-
leagues who have provided detailed and insightful feedback on the latest chapter
drafts: Christina Kazantzidou (who helped to polish words and mathematical no-
tation), Tobias Fischer, Will Browne, Jesse Haviland, Feras Dayoub, Dorian Tsai,
Alessandro De Luca, Renaud Detry, Steve Eddins, Qu Cao, Labhansh Atriwal,
Birju Patel, Hannes Daepp, Karsh Tharyani, Jianxin Sun, Brian Fanous, Cameron
Stabile, Akshai Manchana, and Zheng Dong.
We have tried hard to eliminate errors but inevitably some will remain. Please
contact us on 7 https://github.com/petercorke/RVC3-MATLAB with issues or sug-
gestions for improvements and extensions.
We thank our respective employers, Queensland University of Technology and
MathWorks® , for their support of this project. Over all editions, this book has en-
joyed strong support from the MathWorks book program, and from the publisher.
At Springer-Nature, Thomas Ditzinger has supported this project since before the
first edition, and Wilma McHugh and Ellen Blasig have assisted with this edition.
Special thanks also to Yvonne Schlatter and the team at le-tex for their wonderful
support with typesetting.

Notes on the Third Edition

The first edition (2011), the second The first two editions of this book 9 were based on MATLAB® in conjunction with
edition as a single volume (2017) and open-source toolboxes that are now thirty years old – that’s a long time for any
then as a two-volume set (2022). piece of software. Much has happened in the last decade that motivates a change to
the software foundations of the book, and that has led to two third editions:
4 The version you are reading, rewritten with colleagues from MathWorks, is
based on MATLAB, and state-of-the-art toolboxes developed by MathWorks
including: Robotics System Toolbox™ , Navigation Toolbox™ , Computer Vi-
sion Toolbox™ , and Image Processing Toolbox™ .
To run the examples in this book you require appropriate software licenses and
details are given in 7 App. A.
4 The alternative version, is based on Python which is a popular open-source
language with strong third-party support. The old MATLAB-based toolboxes
have been ported to Python (Corke 2021).

In addition to changing the software underpinnings of the book, this third edition
also provides an opportunity to fix errors, improve mathematical notation, and clar-
ify the narrative throughout. Chapters 2 and 7 have been extensively rewritten. This
edition also includes new topics such as graph-based path planning, Dubins and
Reeds-Shepp paths, branched robots, URDF models, collision checking, task-space
control, deep learning for object detection and semantic segmentation, fiducial
markers, and point clouds. Chapter 16, previously advanced visual servoing, has
been replaced with large-scale application examples that showcase advanced fea-
tures of many MathWorks toolboxes.

Peter Corke
Witold Jachimczyk
Remo Pillat
Brisbane, Australia and Boston, USA
July 2022
 xi

Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 A Brief History of Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Types of Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 Definition of a Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4 Robotic Vision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.5 Ethical Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.6 About the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.6.1 MATLAB and the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.6.2 Notation, Conventions, and Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.6.3 Audience and Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.6.4 Learning with the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.6.5 Teaching with the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.6.6 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.6.7 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

I Foundations
2 Representing Position and Orientation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.1 Foundations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.1.1 Relative Pose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.1.2 Coordinate Frames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.1.3 Pose Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.1.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.2 Working in Two Dimensions (2D) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.2.1 Orientation in Two Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.2.2 Pose in Two Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.3 Working in Three Dimensions (3D) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.3.1 Orientation in Three Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.3.2 Pose in Three Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
2.4 Advanced Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
2.4.1 Pre- and Post-Multiplication of Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
2.4.2 Active and Passive Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
2.4.3 Direction Cosine Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
2.4.4 Efficiency of Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
2.4.5 Distance Between Orientations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
2.4.6 Normalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
2.4.7 Understanding the Exponential Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
2.4.8 More About Twists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
2.4.9 Configuration Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
2.5 MATLAB Classes for Pose and Rotation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
2.6 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
2.6.1 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
2.6.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

3 Time and Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

3.1 Time-Varying Pose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.1.1 Rate of Change of Orientation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.1.2 Rate of Change of Pose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
3.1.3 Transforming Spatial Velocities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
3.1.4 Incremental Rotation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
3.1.5 Incremental Rigid-Body Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
xii Contents

3.2 Accelerating Bodies and Reference Frames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

3.2.1 Dynamics of Moving Bodies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
3.2.2 Transforming Forces and Torques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
3.2.3 Inertial Reference Frame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
3.3 Creating Time-Varying Pose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
3.3.1 Smooth One-Dimensional Trajectories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
3.3.2 Multi-Axis Trajectories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
3.3.3 Multi-Segment Trajectories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
3.3.4 Interpolation of Orientation in 3D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
3.3.5 Cartesian Motion in 3D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
3.4 Application: Inertial Navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
3.4.1 Gyroscopes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
3.4.2 Accelerometers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
3.4.3 Magnetometers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
3.4.4 Inertial Sensor Fusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
3.5 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
3.5.1 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
3.5.2 Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
3.5.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

II Mobile Robotics
4 Mobile Robot Vehicles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
4.1 Wheeled Mobile Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
4.1.1 Car-Like Vehicle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
4.1.2 Differentially-Steered Vehicle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
4.1.3 Omnidirectional Vehicle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
4.2 Aerial Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
4.3 Advanced Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
4.3.1 Nonholonomic and Underactuated Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
4.4 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
4.4.1 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
4.4.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

5 Navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
5.1 Introduction to Reactive Navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
5.1.1 Braitenberg Vehicles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
5.1.2 Simple Automata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
5.2 Introduction to Map-Based Navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
5.3 Planning with a Graph-Based Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
5.3.1 Breadth-First Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
5.3.2 Uniform-Cost Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
5.3.3 A Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
5.3.4 Minimum-Time Path Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
5.3.5 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
5.4 Planning with an Occupancy Grid Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
5.4.1 Distance Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
5.4.2 D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
5.5 Planning with Roadmaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
5.5.1 Introduction to Roadmap Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
5.5.2 Probabilistic Roadmap Method (PRM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
5.6 Planning Drivable Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
5.6.1 Dubins Path Planner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
5.6.2 Reeds-Shepp Path Planner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
5.6.3 Lattice Planner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
 xiii
Contents

5.6.4 Clothoids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

5.6.5 Planning in Configuration Space (RRT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
5.7 Advanced Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
5.7.1 A vs Dijkstra Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
5.7.2 Converting Grid Maps to Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
5.7.3 Converting Between Graphs and Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
5.7.4 Local and Global Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
5.8 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
5.8.1 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
5.8.2 Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
5.8.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

6 Localization and Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

6.1 Dead Reckoning Using Odometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
6.1.1 Modeling the Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
6.1.2 Estimating Pose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
6.2 Landmark Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
6.2.1 Localizing in a Landmark Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
6.2.2 Creating a Landmark Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
6.2.3 EKF SLAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
6.2.4 Sequential Monte-Carlo Localization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
6.2.5 Rao-Blackwellized SLAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
6.3 Occupancy Grid Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
6.3.1 Application: Lidar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
6.3.2 Lidar-Based Odometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
6.3.3 Lidar-Based Map Building . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
6.3.4 Lidar-Based Localization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
6.3.5 Simulating Lidar Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
6.4 Pose-Graph SLAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
6.4.1 Pose-Graph Landmark SLAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
6.4.2 Pose-Graph Lidar SLAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
6.5 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
6.5.1 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
6.5.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271

III Robot Manipulators

7 Robot Arm Kinematics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
7.1 Forward Kinematics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
7.1.1 Forward Kinematics from a Pose Graph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278
7.1.2 Forward Kinematics as a Chain of Robot Links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
7.1.3 Branched Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
7.1.4 Unified Robot Description Format (URDF) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
7.1.5 Denavit-Hartenberg Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
7.2 Inverse Kinematics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
7.2.1 2-Dimensional (Planar) Robotic Arms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
7.2.2 3-Dimensional Robotic Arms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
7.2.3 Underactuated Manipulator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
7.2.4 Overactuated (Redundant) Manipulator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
7.3 Trajectories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
7.3.1 Joint-Space Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
7.3.2 Cartesian Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
7.3.3 Kinematics in Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312
7.3.4 Motion Through a Singularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313
xiv Contents

7.4 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315

7.4.1 Writing on a Surface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
7.4.2 A 4-Legged Walking Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316
7.5 Advanced Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320
7.5.1 Creating the Kinematic Model for a Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320
7.5.2 Modified Denavit-Hartenberg Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
7.5.3 Products of Exponentials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
7.5.4 Collision Checking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324
7.6 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
7.6.1 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326
7.6.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327

8 Manipulator Velocity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329

8.1 Manipulator Jacobian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330
8.1.1 Jacobian in the World Coordinate Frame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330
8.1.2 Jacobian in the End Effector Coordinate Frame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
8.1.3 Analytical Jacobian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
8.2 Application: Resolved-Rate Motion Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
8.3 Jacobian Condition and Manipulability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338
8.3.1 Jacobian Singularities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338
8.3.2 Velocity Ellipsoid and Manipulability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339
8.3.3 Dealing with Jacobian Singularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342
8.3.4 Dealing with a Non-Square Jacobian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342
8.4 Force Relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346
8.4.1 Transforming Wrenches to Joint Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347
8.4.2 Force Ellipsoids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348
8.5 Numerical Inverse Kinematics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348
8.6 Advanced Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
8.6.1 Computing the Manipulator Jacobian Using Twists . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
8.6.2 Manipulability, Scaling, and Units . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
8.7 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
8.7.1 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
8.7.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352

9 Dynamics and Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355

9.1 Independent Joint Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356
9.1.1 Actuators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356
9.1.2 Friction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358
9.1.3 Link Mass . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359
9.1.4 Gearbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360
9.1.5 Modeling the Robot Joint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
9.1.6 Velocity Control Loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
9.1.7 Position Control Loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
9.1.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369
9.2 Rigid-Body Equations of Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369
9.2.1 Gravity Term . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372
9.2.2 Inertia (Mass) Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
9.2.3 Coriolis and Centripetal Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
9.2.4 Effect of Payload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
9.2.5 Base Wrench . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376
9.2.6 Dynamic Manipulability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
9.3 Forward Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378
9.4 Rigid-Body Dynamics Compensation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380
9.4.1 Feedforward Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
9.4.2 Computed-Torque Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382
 xv
Contents

9.5 Task-Space Dynamics and Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382

9.6 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
9.6.1 Series-Elastic Actuator (SEA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
9.7 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389
9.7.1 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389
9.7.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391

IV Computer Vision
10 Light and Color . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
10.1 Spectral Representation of Light . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396
10.1.1 Absorption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399
10.1.2 Reflectance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400
10.1.3 Luminance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400
10.2 Color . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
10.2.1 The Human Eye . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402
10.2.2 Camera Sensor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404
10.2.3 Measuring Color . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405
10.2.4 Reproducing Colors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407
10.2.5 Chromaticity Coordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410
10.2.6 Color Names . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414
10.2.7 Other Color and Chromaticity Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
10.2.8 Transforming Between Different Primaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
10.2.9 What Is White? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419
10.3 Advanced Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420
10.3.1 Color Temperature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420
10.3.2 Color Constancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421
10.3.3 White Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422
10.3.4 Color Change Due to Absorption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423
10.3.5 Dichromatic Reflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424
10.3.6 Gamma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425
10.4 Application: Color Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427
10.4.1 Comparing Color Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427
10.4.2 Shadow Removal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428
10.5 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 430
10.5.1 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431
10.5.2 Data Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432
10.5.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433

11 Images and Image Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435

11.1 Obtaining an Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436
11.1.1 Images from Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436
11.1.2 Images from an Attached Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
11.1.3 Images from a Video File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
11.1.4 Images from the Web . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444
11.1.5 Images from Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445
11.2 Image Histograms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447
11.3 Monadic Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 448
11.4 Dyadic Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451
11.4.1 Application: Chroma Keying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452
11.4.2 Application: Motion detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453
11.5 Spatial Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455
11.5.1 Linear Spatial Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456
11.5.2 Template Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 468
11.5.3 Nonlinear Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473
xvi Contents

11.6 Mathematical Morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474

11.6.1 Noise Removal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 477
11.6.2 Boundary Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479
11.6.3 Hit or Miss Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479
11.6.4 Distance Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481
11.7 Shape Changing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482
11.7.1 Cropping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482
11.7.2 Image Resizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483
11.7.3 Image Pyramids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484
11.7.4 Image Warping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485
11.8 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 488
11.8.1 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489
11.8.2 Sources of Image Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489
11.8.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 490

12 Image Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493

12.1 Region Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496
12.1.1 Pixel Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496
12.1.2 Representation: Distinguishing Multiple Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 506
12.1.3 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 510
12.1.4 Object Detection Using Deep Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 519
12.1.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520
12.2 Line Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 521
12.2.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 525
12.3 Point Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 525
12.3.1 Classical Corner Detectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 526
12.3.2 Scale-Space Feature Detectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 531
12.4 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534
12.4.1 Character Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534
12.4.2 Image Retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535
12.5 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539
12.5.1 MATLAB Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539
12.5.2 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539
12.5.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 541

13 Image Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543

13.1 Perspective Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 544
13.1.1 Perspective Projection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 544
13.1.2 Modeling a Perspective Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 548
13.1.3 Discrete Image Plane . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 549
13.1.4 Camera Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 551
13.1.5 Projecting Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553
13.1.6 Lens Distortion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 556
13.2 Camera Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 558
13.2.1 Calibrating with a 3D Target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 558
13.2.2 Decomposing the Camera Calibration Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 560
13.2.3 Pose Estimation with a Calibrated Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 562
13.2.4 Camera Calibration Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563
13.3 Wide Field-of-View Cameras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 566
13.3.1 Fisheye Lens Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 566
13.3.2 Catadioptric Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 569
13.3.3 Spherical Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 572
13.4 Unified Imaging Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574
13.4.1 Mapping Wide-Angle Images to the Sphere . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575
13.4.2 Mapping from the Sphere to a Perspective Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . 577
 xvii
Contents

13.5 Novel Cameras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 578

13.5.1 Multi-Camera Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 578
13.5.2 Light-Field Cameras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 579
13.6 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 581
13.6.1 Fiducial Markers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 581
13.6.2 Planar Homography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 583
13.7 Advanced Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 584
13.7.1 Projecting 3D Lines and Quadrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 584
13.7.2 Nonperspective Cameras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 586
13.8 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 587
13.8.1 Further Reading and Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 588
13.8.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 590

14 Using Multiple Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 593

14.1 Point Feaature Correspondence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 595
14.2 Geometry of Multiple Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 599
14.2.1 The Fundamental Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 601
14.2.2 The Essential Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 603
14.2.3 Estimating the Fundamental Matrix from Real Image Data . . . . . . . . . . . . . . . . . . . . 604
14.2.4 Planar Homography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 609
14.3 Sparse Stereo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 614
14.3.1 3D Triangulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 614
14.3.2 Bundle Adjustment (advanced) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 618
14.4 Dense Stereo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 623
14.4.1 Stereo Failure Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 627
14.4.2 Refinement and Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 629
14.4.3 Stereo Image Rectification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 631
14.5 Anaglyphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 634
14.6 Other Depth Sensing Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 635
14.6.1 Depth from Structured Light . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 635
14.6.2 Depth from Time-Of-Flight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 636
14.7 Point Clouds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 637
14.7.1 Fitting Geometric Objects into a Point Cloud . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 639
14.7.2 Matching Two Sets of Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 641
14.8 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 644
14.8.1 Perspective Correction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 644
14.8.2 Image Mosaicing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 647
14.8.3 Visual Odometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 650
14.9 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 654
14.9.1 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 654
14.9.2 Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 658
14.9.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 659

V Robotics, Vision & Control

15 Vision-Based Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 665
15.1 Position-Based Visual Servoing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 668
15.2 Image-Based Visual Servoing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 670
15.2.1 Camera and Image Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 671
15.2.2 Controlling Feature Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 678
15.2.3 Estimating Feature Depth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 681
15.2.4 Performance Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 683
15.3 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 686
15.3.1 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 687
15.3.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 690
xviii Contents

16 Real-World Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 693

16.1 Lane and Vehicle Detection with a Monocular Camera . . . . . . . . . . . . . . . . . . . . . . 694
16.2 Highway Lane Change Planner and Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 697
16.3 UAV Package Delivery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 699
16.4 Pick-and-Place Workflow in Gazebo Using Point-Cloud Processing and RRT
Path Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 701

Supplementary Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 705

A Software Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 706
B Linear Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 708
C Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 716
D Lie Groups and Algebras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 732
E Linearization, Jacobians, and Hessians . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 737
F Solving Systems of Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 742
G Gaussian Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 751
H Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 754
I Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 760
J Peak Finding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 763
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 767
Index of People . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 783
Index of Functions, Classes, and Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 785
Index of Apps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 793
Index of Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 795
General Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 797
 xix

Nomenclature

The notation used in robotics and computer vision varies considerably across books
and research papers. The symbols used in this book, and their units where appro-
priate, are listed below. Some symbols have multiple meanings and their context
must be used to disambiguate them.

Notation Description
v a vector

vO a unit vector parallel to v

vQ homogeneous representation of the vector v

vi i th element of vector v

vQ i i th element of the homogeneous vector vQ

vx ; vy ; vz elements of the coordinate vector v

A a matrix

ai;j the element of A at row i and column j

0mn an m  n matrix of zeros

1mn an m  n identity matrix

qM a quaternion, qM 2 H

qV a unit quaternion, qV 2 S3

f .x/ a function of x

Fx .x/ the first derivative of f .x/, @f =@x

Fxy .x; y/ the second derivative of f .x; y/, @2 f =@x@y

fF g coordinate frame F

Vectors are generally lower-case Roman or Greek letters, while matrices are gener-
ally upper-case Roman or Greek letters. Various decorators are applied to symbols:

Decorators Description
x desired value of x

xC predicted value of x

x# measured, or observed, value of x

xO estimated value of x, also a unit vector as above

xN mean of x or relative value

x hki k th element of a time series

where x could be a scalar, vector, or matrix.

xx Nomenclature

Symbol Description Unit

B viscous friction coefficient N m s rad1

B magnetic flux density T

C camera matrix, C 2 R 34

P
C.q; q/ manipulator centripetal and Coriolis term kg m2 s1

C configuration space of a robot with N joints: C  RN

e mathematical constant, base of natural logarithms,

e D 2:71828 : : :

E illuminance (in lux) lx

f focal length m

f force N

P
f .q/ manipulator friction torque Nm

g gravitation acceleration, see . Fig. 3.11 ms2

g.q/ manipulator gravity term Nm

H the set of all quaternions (H for Hamilton)

J inertia kg m2

J inertia tensor, J 2 R33 kg m2

J Jacobian matrix
A
JB Jacobian transforming velocities in frame fBg to frame fAg

k; K constant

K camera intrinsic matrix, K 2 R33

Ki amplifier gain (transconductance) A V1

Km motor torque constant Nm A1

L luminance (in nit) nt

m mass kg

M.q/ manipulator inertia tensor kg m2

N the set of natural numbers f1; 2; 3; : : :g

N0 the set of natural numbers including zero f0; 1; 2; : : :g

N.;  2 / a normal (Gaussian) distribution with mean  and standard devia-

tion 

p an image plane point

P a world point

p coordinate vector of an image plane point, p 2 R2

P coordinate vector of a world point, P 2 R3

P2 projective space of all 2D points, elements are a 3-tuple

P 3
projective space of all 3D points, elements are a 4-tuple

q generalized coordinates, robot configuration q 2 C m, rad

Q generalized force Q 2 R N
N, Nm

R an orthonormal rotation matrix, R 2 SO.2/ or SO.3/

R set of real numbers

R>0 set of positive real numbers

 xxi
Nomenclature

Symbol Description Unit

R0 set of non-negative real numbers

R 2
set of all 2D points, elements are a 2-tuple

R 3
set of all 3D points, elements are a 3-tuple

s distance along a path or trajectory, s 2 Œ0; 1

s Laplace transform operator

S1 set of points on the unit circle  set of angles Œ0; 2 /

Sn unit sphere embedded in RnC1

se.n/ Lie algebra for SE.n/, a vector space of R.nC1/.nC1/ augmented

skew-symmetric matrices

so.n/ Lie algebra for SO.n/, a vector space of Rnn skew-symmetric

matrices

SE.n/ special Euclidean group of matrices, T 2 SE.n/; T  R.nC1/.nC1/ ,

represents pose in n dimensions, aka homogeneous transforma-
tion matrix, aka rigid-body transformation

SO.n/ special orthogonal group of matrices, R 2 SO.n/; R  Rnn ; R> D

R1 ; det.R/ D 1, represents orientation in n dimensions, aka
rotation matrix

S twist in 2 or 3 dimensions, S 2 R3 or R6

t time s

T task space of robot, T  SE.3/ K

T sample interval s

T temperature K

T optical transmission m1

T a homogeneous transformation matrix T 2 SE.2/ or SE.3/

A
TB a homogeneous transformation matrix representing the pose
of frame fBg with respect to frame fAg. If A is not given then
assumed relative to world coordinate frame f0g. Note that
A
TB D . B TA /1

u; v coordinates of point on a camera’s image plane pixels

u0 ; v 0 coordinates of a camera’s principal point pixels

N vN
u; normalized image plane coordinates, relative to the principal pixels
point

v velocity m s1

v velocity vector m s1

w wrench, a vector of forces and moments .mx ; my ; mz ; fx ; fy ; fz / N, Nm

2 R6

O y,
x, O zO unit vectors aligned with the x-, y- and z-axes, 3D basis vectors

X; Y; Z Cartesian coordinates of a point

N yN
x; retinal image-plane coordinates m

Z set of all integers f: : : ; 2; 1; 0; 1; 2; : : :g

¿ null motion

˛ roll angle rad

ˇ pitch angle rad

xxii Nomenclature

Symbol Description Unit

ı an increment

 an error or residual, ideally zero

 3-angle representation of rotation,  2 R3 rad

 body torque,  2 R 3
Nm

 an angle, first Euler angle, yaw angle, heading angle rad

 wavelength m

 an eigenvalue

 innovation

 spatial velocity, .!x ; !y ; !z ; vx ; vy ; vz / 2 R6 m s1

rad s1

 abstract representation of pose (pronounced ksigh)

A
B abstract representation of relative pose, frame fBg with respect to
frame fAg or rigid-body motion from frame fAg to fBg

 ti .d / abstract pose that is pure translation of d along axis i 2 fx; y; zg

ri
 . / abstract pose that is pure rotation a pure rotation of  about axis
i 2 fx; y; zg

mathematical constant D 3:14159 : : :

a plane

w; h pixel width and height m

 standard deviation

 robot joint type,  D R for revolute and  D P for prismatic

† element of the Lie algebra of SE.3/, † D Œv 2 se.3/; v 2 R6

torque Nm

C Coulomb friction torque Nm

angle, second Euler angle, pitch angle rad

third Euler angle, roll angle, steered-wheel angle for mobile robot rad

! rotational rate rad s1

! angular velocity vector rad s1

$ rotational speed of a motor shaft, wheel or propeller rad s1

 element of the Lie algebra of SO.3/,  D Œv 2 so.3/; v 2 R3

v1 v2 dot, or inner, product, also v>

1 v 2 : R  R 7! R
n n

p
kk norm, or length, of vector, also v v: Rn 7! R0

v1  v 2 cross, or vector, product: R3  R3 7! R3

Q Rn 7! P n

. / P n 7! Rn

A1 inverse of A: Rnn 7! Rnn

AC pseudo-inverse of A: Rnm 7! Rmn

A adjugate of A, also det.A/A1 : Rnn 7! Rnn

A> transpose of A: Rnm 7! Rmn

A > transpose of inverse of A, .A> /1 .A1 /> : Rnn 7! Rnn

 xxiii
Nomenclature

Operator Description
˚ composition of abstract pose: xy ˚ y z 7! x z

composition of abstract pose with inverse: xy z

y x
y ˚ y z 7! x z
x y
inverse of abstract pose: y x

 transform a point (coordinate vector) by abstract relative pose: xy  yp 7! xp

./ maps incremental relative pose to differential motion: SE.3/ 7! R6

1
./ maps differential motion to incremental relative pose: R6 7! SE.3/

Πt translational component of pose: SE.n/ 7! Rn

ΠR rotational component of pose: SE.n/ 7! SO.n/

Πxy 2D pose to configuration: SE.2/ 7! R2  S1

Π skew-symmetric matrix: R 7! so.2/, R3 7! so.3/

Πaugmented skew-symmetric matrix: R3 7! se.2/, R6 7! se.3/

_ . / unpack skew-symmetric matrix: so.2/ 7! R, so.3/ 7! R3

_. / unpack augmented skew-symmetric matrix: se.2/ 7! R3 , se.3/ 7! R6

Ad. / adjoint representation: SE.3/ 7! R66

ad. / logarithm of adjoint representation: SE.3/ 7! R66

ı quaternion (Hamiltonian) multiplication: H  H 7! H

vM pure quaternion: R3 7! H

 equivalence of representation

' homogeneous coordinate equivalence

 smallest angular difference between two angles: S1  S1 7! Π; /

K. / forward kinematics: C 7! T

K 1 . / inverse kinematics: T 7! C

D1 . / manipulator inverse dynamics function: C ; RN ; RN 7! RN

P. / camera projection function: R3 7! R2

 image convolution

˝ image correlation

colormetric equivalence

˚ morphological dilation

morphological erosion

ı morphological opening

morphological closing

Œa; b interval a to b inclusive

.a; b/ interval a to b exclusive, not including a or b

Œa; b/ interval a to b, not including b

.a; b interval a to b, not including a

xxiv Nomenclature

Toolbox Conventions

4 A Cartesian point, a coordinate vector, is mathematically described as a column

vector in this book, but the MATLAB toolboxes generally expect it as row
vector.
4 A set of points is expressed as a matrix with rows representing the coordinate
vectors of individual points.
4 Time series data is generally expressed as a matrix with rows representing time
steps.
4 A rectangular region is frequently represented by a 4-vector: the coordinate of
its top-left corner, and its width and height Œx y w h.
4 A robot configuration, a set of joint angles, is expressed as a row vector.
4 A MATLAB matrix has subscripts .i; j / which represent row and column in-
dices respectively. Image coordinates are written .u; v/, so an image repre-
sented by a matrix I is indexed as I(v,u).
4 The indices of MATLAB matrices start from one.
4 Matrices with three or more dimensions are frequently used:
– A series of poses or orientations has 3 dimensions where the third index is
the sequence index, and A(:,:,k) is an SE.n/ or SO.n/ matrix.
– A color image has 3 dimensions: row, column, color plane.
– A grayscale image sequence has 3 dimensions: row, column, index.
– A color image sequence has 4 dimensions: row, column, color plane, index.

Common Abbreviations

1D – 1-dimensional
2D – 2-dimensional
3D – 3-dimensional
CoM – Center of mass
DoF – Degrees of freedom
n-tuple – A group of n numbers, it can represent a point or a vector
 1 1

Introduction
Contents

1.1 A Brief History of Robots – 2

1.2 Types of Robots – 4

1.3 Definition of a Robot – 8

1.4 Robotic Vision – 9

1.5 Ethical Considerations – 11

1.6 About the Book – 12

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023

P. Corke, W. Jachimczyk, R. Pillat, Robotics, Vision and Control, Springer Tracts in Advanced Robotics 147,
https://doi.org/10.1007/978-3-031-07262-8_1
 2 Chapter 1 Introduction

1.1 A Brief History of Robots

1
The word robot means different things to different people. Science fiction books
and movies have strongly influenced what many people expect a robot to be or
what it can do – sadly the practice of robotics is far behind this popular conception.
The word robot is also emotive and some people are genuinely fearful of a future
with robots, while others are concerned about what robots mean for jobs, privacy,
or warfare. One thing is certain though – robotics will be an important technology
in this century. Products such as vacuum cleaning robots have already been with
us for over two decades, and self-driving cars are now on the roads with us. These
are the vanguard of a wave of smart machines that will appear in our homes and
workplaces in the near to medium future.
In the eighteenth century, Europeans were fascinated by automata such as Vau-
canson’s duck shown in . Fig. 1.1a. These machines, complex by the standards
of the day, demonstrated what then seemed life-like behavior. The duck used a
cam mechanism to sequence its movements and Vaucanson went on to explore
the mechanization of silk weaving. Jacquard extended these ideas and developed a
loom, shown in . Fig. 1.1b, that was essentially a programmable weaving ma-
chine. The pattern to be woven was encoded as a series of holes on punched
This in turn influenced Sir Charles cards. 9 This machine has many hallmarks of a modern robot: it performed a phys-
Babbage in his quest to mechanize ical task and was reprogrammable.
computation, which in turn influenced The term robot first appeared in a 1920 Czech science fiction play “Rossum’s
Countess Ada Lovelace to formalize Universal Robots” by Karel Čapek (pronounced chapek). The word roboti was sug-
computation and create the first gested by his brother Josef, and in the Czech language has the same linguistic roots
algorithm.

a b

. Fig. 1.1 Early programmable machines. a Vaucanson’s duck (1739) was an automaton that could
flap its wings, eat grain, and defecate. It was driven by a clockwork mechanism and executed a single
program; b the Jacquard loom (1801) was a reprogrammable machine and the program was held on
punched cards (Image by George P. Landow from 7 https://www.victorianweb.org)

a b

. Fig. 1.2 Universal automation. a A plan view of the machine from Devol’s patent; b the first Unima-
tion robot – the Unimate – working at a General Motors factory (Image courtesy of George C. Devol)
1.1 A Brief History of Robots
3 1

a b

. Fig. 1.3 Manufacturing robots, technological descendants of the Unimate shown in . Fig. 1.2.
a A modern six-axis robot designed for high accuracy and throughput (Image courtesy ABB robotics);
b Universal Robots collaborative robot can work safely with a human co-worker (© 2022 Universal
Robots A/S. All Rights Reserved. This image has been used with permission from Universal Robots
A/S)

Excurse 1.1: George C. Devol

Devol (1912–2011) was a prolific American inventor, born in
Louisville, Kentucky. In 1932, he founded United Cinephone
Corp. which manufactured phonograph arms and amplifiers,
registration controls for printing presses and packaging ma-
chines. In 1954, he applied for US patent 2,988,237 for
Programmed Article Transfer which introduced the concept
of Universal Automation or “Unimation”. Specifically, it de-
scribed a track-mounted polar-coordinate arm mechanism
with a gripper and a programmable controller – the precursor
of all modern robots.
In 2011 he was inducted into the National Inventors Hall
of Fame. (Image courtesy of George C. Devol)

as servitude and slavery. The robots in the play were artificial people or androids
and, as in so many robot stories that follow this one, the robots rebel and it ends
badly for humanity. Isaac Asimov’s robot series, comprising many books and short
stories written between 1950 and 1985, explored issues of human and robot in-
teraction, and morality. The robots in these stories are equipped with “positronic

Excurse 1.2: Unimation Inc.

Devol sought financing to develop his unimation technology and at a cocktail party
in 1954 he met Joseph Engelberger who was then an engineer with Manning,
Maxwell and Moore. In 1956 they jointly established Unimation (1956–1982), the
first robotics company, in Danbury, Connecticut. The company was acquired by
Consolidated Diesel Corp. (Condec) and became Unimate Inc. a division of Con-
dec. Their first robot went to work in 1961 at a General Motors die-casting plant in
New Jersey. In 1968 they licensed technology to Kawasaki Heavy Industries which
produced the first Japanese industrial robot. Engelberger served as chief executive
until it was acquired by Westinghouse in 1982. People and technologies from this
company have gone on to be very influential across the whole field of robotics.
 4 Chapter 1 Introduction

1 Excurse 1.3: Joseph F. Engelberger

Engelberger (1925–2015) was an American engineer and en-
trepreneur who is often referred to as the “Father of Robotics”.
He received his B.S. and M.S. degrees in physics from
Columbia University, in 1946 and 1949, respectively. En-
gelberger was a tireless promoter of robotics. In 1966, he
appeared on The Tonight Show Starring Johnny Carson with a
Unimate robot which poured a beer, putted a golf ball, and di-
rected the band. He promoted robotics heavily in Japan, which
led to strong investment and development of robotic technol-
ogy in that country.
Engelberger served as chief executive of Unimation Inc.
until 1982, and in 1984 founded Transitions Research Cor-
poration which became HelpMate Robotics Inc., an early
entrant in the hospital service robot sector. He was elected to
the National Academy of Engineering, received the Beckman
Award and the Japan Prize, and wrote two books: Robotics
in Practice (1980) and Robotics in Service (1989). Each year
the Association for Advancing Automation (7 https://www.
automate.org) presents an award in his honor to “persons who
have contributed outstandingly to the furtherance of the sci-
ence and practice of robotics.”

brains” in which the “Three Laws of Robotics” are encoded. These stories have
influenced subsequent books and movies, which in turn have shaped the public
perception of what robots are. The mid-twentieth century also saw the advent of
the field of cybernetics – an uncommon term today but then an exciting science at
the frontiers of understanding life and creating intelligent machines.
The first patent for what we would now consider a robot manipulator was filed
in 1954 by George C. Devol and issued in 1961. The device comprised a mechan-
ical arm with a gripper that was mounted on a track and the sequence of motions
was encoded as magnetic patterns stored on a rotating drum. The first robotics
company, Unimation, was founded by Devol and Joseph Engelberger in 1956 and
their first industrial robot, shown in . Fig. 1.2b, was installed in 1961. The original
vision of Devol and Engelberger for robotic automation has subsequently become
a reality. Many millions of robot manipulators, such as shown in . Fig. 1.3, have
been built and put to work at tasks such as welding, painting, machine loading and
unloading, assembly, sorting, packaging, and palletizing. The use of robots has led
to increased productivity and improved product quality. Today, many products that
we buy have been assembled or handled by a robot.

1.2 Types of Robots

The first generation of robots were fixed in place and could not move about the
factory. By contrast, mobile robots shown in . Figs. 1.4 and 1.5 can move through
the world using various forms of mobility. They can locomote over the ground
using wheels or legs, fly through the air using fixed wings or multiple rotors, move
through the water, or sail over it.
An alternative taxonomy is based on the function that the robot performs. Man-
ufacturing robots operate in factories and are the technological descendants of the
first generation robots created by Unimation Inc. Service robots provide services
to people such as cleaning, personal care, medical rehabilitation or fetching and
1.2 Types of Robots
5 1

a b

. Fig. 1.4 Non-land-based mobile robots. a Small autonomous underwater vehicle (Todd Walsh © 2013 MBARI); b Global Hawk uncrewed aerial
vehicle (UAV) (Image courtesy of NASA)

a b

c d

. Fig. 1.5 Mobile robots. a Perseverance rover on Mars, self portrait. The mast contains many cameras including stereo camera pairs from which
the robot can compute the 3-dimensional structure of its environment (Image courtesy of NASA/JPL-Caltech/MSSS); b Savioke Relay delivery robot
(Image courtesy Savioke); c self-driving car (Image courtesy Dept. Information Engineering, Oxford Univ.); d Boston Dynamics Spot® legged robot
(Image courtesy Dorian Tsai)

carrying as shown in . Fig. 1.5b. Field robots, such as those shown in . Fig. 1.4,
work outdoors on tasks such as environmental monitoring, agriculture, mining,
construction, and forestry. Humanoid robots such as shown in . Fig. 1.6 have the
physical form of a human being – they are both mobile robots and service robots.
 6 Chapter 1 Introduction

1 Excurse 1.4: Rossum’s Universal Robots (RUR)

This 1920 play by Karel Čapek introduced the word “robot” The full play can be found at 7 https://www.gutenberg.org/
to the world. In the introductory scene Helena Glory is visit- ebooks/59112. One hundred years later, it is prescient and still
ing Harry Domin, the director general of Rossum’s Universal thought provoking. (Image courtesy of Library of Congress,
Robots, and his robotic secretary Sulla. item 96524672)
Domin – Sulla, let Miss Glory have a look at you.
Helena – (stands and offers her hand) Pleased to meet you.
It must be very hard for you out here, cut off from the rest of
the world [the factory is on an island]
Sulla – I do not know the rest of the world Miss Glory.
Please sit down.
Helena – (sits) Where are you from?
Sulla – From here, the factory
Helena – Oh, you were born here.
Sulla – Yes I was made here.
Helena – (startled) What?
Domin – (laughing) Sulla isn’t a person, Miss Glory, she’s a
robot.
Helena – Oh, please forgive me . . .

a b

. Fig. 1.6 Humanoid robots. a Honda’s Asimo humanoid robot (Image courtesy Honda Motor Co.
Japan); b Hubo robot that won the DARPA Robotics Challenge in 2015 (Image courtesy KAIST, Korea)

A manufacturing robot is typically an arm-type manipulator on a fixed base,

such as shown in . Fig. 1.3a, that performs repetitive tasks within a local work
cell. Parts are presented to the robot in an orderly fashion which maximizes the
advantage of the robot’s high speed and precision. High-speed robots are hazardous
and safety is achieved by excluding people from robotic work places – typically
the robot is placed inside a cage. In contrast, collaborative robots such as shown
in . Fig. 1.3b, are human safe – they operate at low speed and stop moving when
they encounter an obstruction.
1.2 Types of Robots
7 1

Excurse 1.5: Telerobots

The Manhattan Project in World War 2 developed the first
nuclear weapons and this required handling of radioactive
material. Remotely controlled arms were developed by Ray
Goertz at Argonne National Laboratory to exploit the manual
dexterity of human operators, while keeping them away from
the hazards of the material they were handling. The opera-
tors viewed the work space through thick lead-glass windows
or via a television link and manipulated the leader arm (on
the left). The follower arm (on the right) followed the mo-
tion, and forces felt by the follower arm were reflected back
to the leader arm, allowing the operator to feel weight and in-
terference force. Telerobotics is still important today for many
tasks where people cannot work but which are too complex for
a machine to perform by itself, for instance the underwater
robots that surveyed the wreck of the Titanic. (Image courtesy
of Argonne National Laboratory)

Field and service robots face specific and significant challenges. The first chal-
lenge is that the robot must operate and move in a complex, cluttered and changing
environment. For example, a delivery robot in a hospital must operate despite
crowds of people and a time-varying configuration of parked carts and trolleys. A
Mars rover, as shown in . Fig. 1.5a, must navigate rocks and small craters despite
not having an accurate local map in advance of its travel. Robotic, or self-driving
cars, such as shown in . Fig. 1.5c, must follow roads, avoid obstacles, and obey
traffic signals and the rules of the road. The second challenge for these types of
robots is that they must operate safely in the presence of people. The hospital de-
livery robot operates amongst people, the robotic car contains people, and a robotic
surgical device operates inside people.
Telerobots are robot-like machines that are remotely controlled by a human op-
erator. Perhaps the earliest example was a radio-controlled boat demonstrated by
Nikola Tesla in 1898 and which he called a teleautomaton. Such machines were
an important precursor to robots, and are still important today for tasks conducted
in environments where people cannot work, but which are too complex for a ma-
chine to perform by itself. For example, the “underwater robots” that surveyed the
wreck of the Titanic were technically remotely operated vehicles (ROVs). A mod-
ern surgical robot as shown in . Fig. 1.7 is also teleoperated – the motion of the
small tools inside the patient are remotely controlled by the surgeon. The patient
benefits because the procedure is carried out using much smaller incisions than the
old-fashioned approach where the surgeon works inside the body with their hands.
The various Mars rovers autonomously navigate the surface of Mars but human
operators provide the high-level goals. That is, the operators tell the robot where to
go and the robot itself determines the details of the route. Local decision making
on Mars is essential given that the communications delay is several minutes. Some
telerobots are hybrids, and the control task is shared or traded with a human oper-
ator. In traded control, the control function is passed back and forth between the
human operator and the computer. For example, an aircraft pilot can pass control
to an autopilot and take control back. In shared control, the control function is per-
formed by the human operator and the computer working together. For example,
an autonomous passenger car might have the computer keeping the car safely in
the lane while the human driver controls speed.
 8 Chapter 1 Introduction

1 Excurse 1.6: Cybernetics, Artificial Intelligence and Robotics

Cybernetics flourished as a research field from the 1930s until Machine (Wiener 1965). A characteristic of a cybernetic sys-
the 1960s and was fueled by a heady mix of new ideas and re- tem is the use of feedback which is common in engineering
sults from neurology, control theory, and information theory. and biological systems. The ideas were later applied to evolu-
Research in neurology had shown that the brain was an electri- tionary biology, psychology, and economics.
cal network of neurons. Harold Black, Henrik Bode and Harry In 1956, a watershed conference was hosted by John
Nyquist at Bell Labs were researching negative feedback and McCarthy at Dartmouth College and attended by Minsky,
the stability of electrical networks, Claude Shannon’s infor- Shannon, Herbert Simon, Allen Newell and others. This meet-
mation theory described digital signals, and Alan Turing was ing defined the term artificial intelligence (AI) as we know it
exploring the fundamentals of computation. Walter Pitts and today, with an emphasis on digital computers and symbolic
Warren McCulloch proposed an artificial neuron in 1943 and manipulation and led to new research in robotics, vision, natu-
showed how it might perform simple logical functions. In ral language, semantics and reasoning. McCarthy and Minsky
1951 Marvin Minsky built SNARC (from a B24 autopilot formed the AI group at MIT, and McCarthy left in 1962 to
and comprising 3000 vacuum tubes) which was perhaps the form the Stanford AI Laboratory. Minsky focused on artifi-
first neural-network-based learning machine as his graduate cially simple “blocks world”. Simon, and his student Newell,
project. William Grey Walter’s robotic tortoises showed life- were influential in AI research at Carnegie-Mellon Univer-
like behavior (see 7 Sect. 5.1). Maybe an electronic brain sity from which the Robotics Institute was spawned in 1979.
could be built! These AI groups were to be very influential in the develop-
An important early book was Norbert Wiener’s Cyber- ment of robotics and computer vision in the USA. Societies
netics or Control and Communication in the Animal and the and publications focusing on cybernetics are still active today.

. Fig. 1.7 The working end of a surgical robot, multiple tools work inside the patient but pass through
a single small incision (Image © 2015 Intuitive Surgical, Inc)

1.3 Definition of a Robot

» I can’t define a robot, but I know one when I see one

– Joseph Engelberger

So what is a robot? There are many definitions and not all of them are particularly
helpful. A definition that will serve us well in this book is
» a goal-oriented machine that can sense, plan, and act.

A robot senses its environment and uses that information, together with a goal, to
plan some action. The action might be to move the tool of a robot manipulator arm
to grasp an object, or it might be to drive a mobile robot to some place.
Sensing is critical to robots. Proprioceptive sensors measure the state of the
robot itself: the angle of the joints on a robot arm, the number of wheel revolutions
on a mobile robot, or the current drawn by an electric motor. Exteroceptive sensors
1.4 Robotic Vision
9 1
measure the state of the world with respect to the robot. The sensor might be a
simple bump sensor on a robot vacuum cleaner to detect collision. It might be
a GPS receiver that measures distances to an orbiting satellite constellation, or a
compass that measures the direction of the Earth’s magnetic field vector relative
to the robot. It might also be an active sensor that emits acoustic, optical or radio
pulses in order to measure the distance to points in the world based on the time
taken for a reflection to return to the sensor.

1.4 Robotic Vision

Another way to sense the world is to capture and interpret patterns of ambient
light reflected from the scene. This is what our eyes and brain do, giving us the
sense of vision. Our own experience is that vision is a very effective sensor for
most things that we do, including recognition, navigation, obstacle avoidance and
manipulation. We are not alone in this, and almost all animal species use eyes —
in fact evolution has invented the eye many times over. . Fig. 1.8 shows some of
the diversity of eyes found in nature.
It is interesting to note that even very simple animals, bees for example, with
brains comprising just 106 neurons (compared to our 1011 ) are able to perform
complex and life-critical tasks such as finding food and returning it to the hive
using only vision. For more complex animals such as ourselves, the benefits of
vision outweigh the very high biological cost of owning an eye: the complex eye
itself, muscles to move it, eyelids and tear ducts to protect it, and a large visual
cortex (one third of our brain) to process the data it produces.

a b

c d

. Fig. 1.8 The diversity of eyes. a Robber fly, Holocephala fusca; b jumping spider, Phidippus putnami (Images a and b courtesy Thomas Shahan,
7 https://thomasshahan.com); c scallop (Image courtesy of Sönke Johnsen), each of the small blue spheres is an eye; d human eye
 10 Chapter 1 Introduction

. Fig. 1.9 A cluster of cameras on an outdoor mobile robot: forward looking stereo pair, side looking
wide-angle camera, overhead panoramic camera mirror (Image courtesy of CSIRO)

The sense of vision has long been of interest to robotics researchers – cam-
eras can mimic the function of an eye and create images, and computer vision
algorithms can extract meaning from the images. Combined, they could create
powerful vision-based competencies for robots such as recognizing and manipu-
lating objects and navigating within the world. For example, a soccer playing robot
could determine the coordinate of a round red object in the scene, while a drone
could estimate its motion in the world based on how the world appears to move
relative to the drone. . Fig. 1.9 shows a robot with a number of different types of
cameras to enable outdoor navigation.
Technological developments have made it increasingly feasible for robots to
use cameras as eyes and computers as brains, especially with recent advancements
in deep learning. For much of the history of computer vision, dating back to the
1960s, electronic cameras were cumbersome and expensive, and computer power
was inadequate. Today we have CMOS cameras developed for cell phones that
cost just a few dollars each, and personal computers come standard with massive
parallel computing power. New algorithms, cheap sensors, and plentiful computing
power make vision a practical sensor today, and that is a strong focus of this book.
An important limitation of a camera, or an eye, is that the 3-dimensional struc-
ture of the scene is lost in the resulting 2-dimensional image. Despite this, humans
are particularly good at inferring the 3-dimensional nature of a scene using a num-
ber of visual cues. One approach, used by humans and robots, is stereo vision where
information from two eyes is combined to estimate the 3-dimensional structure of
the scene. The Mars rover shown in . Fig. 1.5a has a stereo camera on its mast,
and the robot in . Fig. 1.9 has a stereo camera on its turret.
If the robot’s environment is unchanging it could make do with an accurate map
and do away with sensing the state of the world, apart from determining its own
position. Imagine driving a car with the front window completely covered over and
1.5 Ethical Considerations
11 1

Excurse 1.7: Evolution of the Eye

It is believed that all animal eyes share a common ancestor in a proto-eye that
evolved 540 million years ago. However major evolutionary advances seem to have
occurred in just the last few million years. The very earliest eyes, called eyespots,
were simple patches of photoreceptor protein in single-celled animals. Multi-celled
animals evolved multicellular eyespots which could sense the brightness of light
but not its direction. Gradually, the eyespot evolved into a shallow cup shape which
gave a limited ability to discriminate directional brightness according to which cells
were illuminated. The pit deepened, the opening became smaller, and the number of
photoreceptor cells increased, forming a pin-hole camera that was capable of distin-
guishing shapes. Next came an overgrowth of transparent cells to protect the eyespot
which led to a filled eye chamber, and eventually the eye as we know it today. The
lensed eye has evolved independently seven different times across species. Nature
has evolved ten quite distinct eye designs including those shown in . Fig. 1.8.

just looking at the GPS navigation system. If you had the road to yourself, you
could probably drive successfully from A to B although it might be quite stressful.
However, if there were other cars, pedestrians, traffic signals or roadworks this
approach would not work. To deal with this realistic scenario you need to look
outwards – to sense the world and plan your actions accordingly. For humans this
is easy, done without conscious thought, but it is not yet easy to program a machine
to do the same – this is the challenge of robotic vision.

1.5 Ethical Considerations

A number of ethical issues arise from the advent of robotics. Perhaps the greatest
concern to the wider public is “robots taking jobs from people”. Already today,
artificial intelligence systems are encroaching on many information handling tasks
including image analysis, decision making, credit assessment, and most recently,
answering questions and writing text. People fear that robots will soon encroach on
physical tasks, but currently the skill of robots for everyday tasks is poor, reliability
and speed is low, and the cost is high. However, it is highly likely that, over time,
these challenges will be overcome – we cannot shy away from the fact that many
jobs now done by people could, in the future, be performed by robots.
The issue of robots and jobs, even today, is complex. Clearly there are jobs
which people should not do, for example working in unhealthy or hazardous en-
vironments. There are many low-skilled jobs where human labor is increasingly
hard to source, for instance in jobs like fruit picking. In many developed countries
people no longer aspire to hard physical outdoor work in remote locations. What
are the alternatives if people don’t want to do the work? Consider again the fruit
picking example, and in the absence of available human labor – do we stop eating
fruit? do we dramatically raise the wages of fruit pickers (and increase the cost of
fruit)? do we import fruit from other places (and increase the cost of fruit as well
as its environmental footprint)? or do we use robots to pick locally grown fruit?
In areas like manufacturing, particularly car manufacturing, the adoption of
robotic automation has been critical in raising productivity which has allowed that
industry to be economically viable in high-wage countries like Europe, Japan and
the USA. Without robots, these industries could not exist; they would not employ
any people, not pay any taxes, and not consume products and services from other
parts of the economy. Automated industry might employ fewer people, but it still
makes an important contribution to society. Rather than taking jobs, we could argue
 12 Chapter 1 Introduction

that robotics and automation has helped to keep manufacturing industries viable in
1 high-labor cost countries. How do we balance the good of the society with the good
of the individual?
There are other issues besides jobs. Consider self-driving cars. We are surpris-
ingly accepting of human-driven cars even though they kill more than one million
people every year, yet many are uncomfortable with the idea of self-driving cars
even though they will dramatically reduce this loss of life. We worry about who to
blame if a robotic car makes a mistake while the carnage caused by human drivers
continues. Similar concerns are raised when talking about robotic healthcare and
surgery – human surgeons are not perfect but robots are seemingly held to a much
higher account. There is a lot of talk about using robots to look after elderly people,
but does this detract from their quality of life by removing human contact, conver-
sation and companionship? Should we use robots to look after our children, and
even teach them? What do we think of armies of robots fighting and killing human
beings?
Robotic fruit picking, cars, health care, elder care, and child care might bring
economic benefits to our society but is it the right thing to do? Is it a direction that
we want our society to go? Once again, how do we balance the good of the society
with the good of the individual? These are deep ethical questions that cannot and
should not be decided by roboticists alone. But neither should roboticists ignore
them. We must not sleepwalk into a technological future just “because we can”. It
is our responsibility to help shape the future in a deliberate way and ensure that
it is good for all. This is an important discussion for all of society, and roboticists
have a responsibility to be active participants in this debate.

1.6 About the Book

This is a book about robotics and computer vision – separately, and together as
robotic vision. These are big topics and the combined coverage is necessarily
broad. The goals of the book are:
4 to provide a broad and solid base of understanding through theory and the use
of examples to make abstract concepts tangible;
4 to tackle more complex problems than other more specialized textbooks by
virtue of the powerful numerical tools and software that underpins it;
4 to provide instant gratification by solving complex problems with relatively
little code;
4 to complement the many excellent texts in robotics and computer vision;
4 to encourage intuition through hands-on numerical experimentation.

The approach used is to present background, theory, and examples in an integrated

fashion. Code and examples are first-class citizens in this book and are not rele-
gated to the end of the chapter or an associated web site. The examples are woven
into the discussion like this
>> R = rotm2d(0.3)
R =
0.9553 -0.2955
0.2955 0.9553

where the code illuminates the topic being discussed and generally results in a
crisp numerical result or a graph in a figure that is then discussed. The examples
illustrate how to use the software tools and that knowledge can then be applied to
other problems.
1.6 About the Book
13 1
1.6.1 MATLAB and the Book

» To do good work, one must first have good tools.

– Chinese proverb

The computational foundation of this book is MATLAB® , a software package

developed by MathWorks® . MATLAB is an interactive mathematical software en-
vironment that makes linear algebra, data analysis, and high-quality graphics a
breeze. It also supports a programming language which allows the creation of com-
plex algorithms. MATLAB is a popular package and one that is very likely to be
familiar to engineering students as well as researchers. It can run on the desktop
or in a web browser or be compiled into efficient C/C++/CUDA/HDL code for
embedded systems. MATLAB is proprietary software and must be licensed: many
universities provide licenses for students and academics; and many companies in
the technology sector, large and small, provide licenses for their staff.
A strength of MATLAB is that its functionality can be extended using toolboxes
which are collections of functions targeted at particular topics. While many MAT-
LAB toolboxes are open source, this book requires a number of licensed MATLAB
toolboxes, and the details are given in 7 App. A.
This book provides examples of how to use many toolbox functions in the con-
text of solving specific problems, but it is not a reference manual. Comprehensive
documentation of all toolbox functions is available through the MATLAB built-in
documentation browser or online at 7 https://mathworks.com/help/matlab.

1.6.2 Notation, Conventions, and Organization

The mathematical notation used in the book is summarized in the 7 Nomenclature

section. Since the coverage of the book is broad there are just not enough good
symbols to go around, and unavoidably some symbols have different meanings in
different parts of the book.

Excurse 1.8: MATLAB

The MATLAB software we use today has a long history. It spread to other universities and found a strong audience within
starts with the LINPACK and EISPACK projects, run by the the applied mathematics and engineering community. In 1984
Argonne National Laboratory in the 1970s, to produce high- Cleve Moler and Jack Little founded The MathWorks Inc.
quality, tested and portable mathematical software for super- which exploited the newly released IBM PC – the first widely
computers. LINPACK is a collection of routines for linear available desktop computer.
algebra and EISPACK is a library of numerical algorithms for Cleve Moler received his bachelor’s degree from Caltech
computing eigenvalues and eigenvectors of matrices. These in 1961, and a Ph.D. from Stanford University. He was a pro-
packages were written in Fortran which was then the lan- fessor of mathematics and computer science at universities
guage of choice for large-scale numerical problems. LAPACK including University of Michigan, Stanford University, and
is the intellectual descendant of those early packages and is the University of New Mexico. He has served as president of
the foundation of most linear algebra code today. the Society for Industrial and Applied Mathematics (SIAM)
Cleve Moler, then at the University of New Mexico, was and was elected to the National Academy of Engineering in
part of the LINPACK and EISPACK projects. He wrote the 1997.
first version of MATLAB in the late 1970s to allow interactive See also 7 https://sn.pub/9FRP5R which includes a video
use of LINPACK and EISPACK for problem solving without of Cleve Moler and also 7 http://history.siam.org/pdfs2/
having to write and compile Fortran code. MATLAB quickly Moler_final.pdf.
 14 Chapter 1 Introduction

There is a lot of MATLAB code in the book and this is indicated in fixed-width
1 font such as
>> 6 * 7
ans =
42

The MATLAB command prompt is >> and what follows is the command issued
to MATLAB by the user. Subsequent lines, without the prompt, are MATLAB’s
Code blocks with the prompt response. Long commands require continuation lines which also begin with >>. 9
characters can be copy and pasted into All functions, classes, and methods mentioned in the text or in code segments are
the MATLAB console and will be cross-referenced and have their own index at the end of the book, allowing you to
interpreted correctly. find different ways that particular functions can be used.
The code examples conform to the conventions for MATLAB example cod-
With the exception that we prefer the ing 9 and rely on recent MATLAB language extensions: strings which are
object-oriented dot syntax to functional delimited by double quotation marks (introduced in R 2016b); and name=value
syntax for object methods. syntax for passing arguments to functions (introduced in R 2021a), for example,
plot(x,y,LineWidth=2) instead of the old-style plot(x,y,"LineWidth",2).
Various boxes are used to organize and differentiate parts of the content.

These highlight boxes indicate content that is particularly important.

! These warning boxes highlight points that are often traps for those starting out.

As authors, we have to balance the tension between completeness, clarity, and

They are placed as marginal notes near conciseness. For this reason a lot of detail has been pushed into side notes 9 and
the corresponding marker. information boxes, and on a first reading, these can be skipped. Some chapters have
an Advanced Topics section at the end that can also be skipped on a first reading.
For clarity, references are cited sparingly in the text but are included at the end of
each chapter. However if you are trying to understand a particular algorithm and
apply it to your own problem, then understanding the details and nuances can be
important and the sidenotes, references and Advanced Topics will be helpful.
Each chapter ends with a Wrapping Up section that summarizes the important
lessons from the chapter, discusses some suggested further reading, and provides
some exercises. The Further Reading subsection discusses prior work and provides
extensive references to prior work and more complete description of the algo-
rithms. Resources provides links to relevant online code and datasets. Exercises
extend the concepts discussed within the chapter and are generally related to spe-
cific code examples discussed in the chapter. The exercises vary in difficulty from
straightforward extension of the code examples to more challenging problems.
Many of the code examples include a plot command which generates a figure.
The figure that is included in the book is generally embellished with axis labels,
grid, legends and adjustments to line thickness. The embellishment requires many
extra lines of code which would clutter the examples, so the book’s GitHub repos-
itory, see 7 App. A, provides the detailed MATLAB scripts that generate all of the
published figures.
chapter2.mlx Finally, the book contains links to online resources which can be simply
accessed through a web browser. Some of these have short URLs of the form

Excurse 1.9: Information

These information boxes provide technical, historical, or biographical information
7 sn.pub/KI1xtA that augment the main text.
1.6 About the Book
15 1
sn.pub/XXXXXX (clickable in the e-book version) and QR codes, which are placed
in the margin. Each chapter has a link to a MATLAB live script running on
MATLAB® Online™, allowing you to run that chapter’s code in a browser win-
dow (even on a tablet) with zero installation on your own computer. Some figures
depict three-dimensional scenes and the included two-dimensional image do not
do them justice – the links connect to web-based interactive 3D viewers of those
scenes.

1.6.3 Audience and Prerequisites

The book is intended primarily for third or fourth year engineering undergraduate
students, Masters students and first year Ph.D. students. For undergraduates the
book will serve as a companion text for a robotics, mechatronics, or computer
vision course or support a major project in robotics or vision. Students should
study Part I and the appendices for foundational concepts, and then the relevant
part of the book: mobile robotics, arm robots, computer vision or robotic vision.
The code in the book shows how to solve common problems, and the exercises at
the end of each chapter provide additional problems beyond the worked examples
in the book.
For students commencing graduate study in robotics, and who have previously
studied engineering or computer science, the book will help fill the gaps between
what you learned as an undergraduate, and what will be required to underpin your
deeper study of robotics and computer vision. The book’s working code base can
help bootstrap your research, enable you to get started quickly, and work produc-
tively on your own problems and ideas.
For those who are no longer students, the researcher or industry practitioner,
the book will serve as a useful companion for your own reference to a wide range
of topics in robotics and computer vision, as well as a handbook and guide for the
Toolboxes.
The book assumes undergraduate-level knowledge of linear algebra (matrices,
vectors, eigenvalues), basic set theory, basic graph theory, probability, calculus, dy-
namics (forces, torques, inertia) and control theory. The appendices provide a con-
cise reiteration of key concepts. Computer science students may be unfamiliar with
concepts in 7 Chaps. 4 and 9 such as the Laplace transform, transfer functions,
linear control (proportional control, proportional-derivative control, proportional-
integral control) and block diagram notation. That material could be skimmed over
on a first reading, and Albertos and Mareels (2010) may be a useful introduction
to some of these topics. The book also assumes the reader is familiar with using,
and programming in, MATLAB and also familiar with MATLAB object-oriented
programming techniques (which is conceptually similar to C++, Java or Python).
Familiarity with Simulink® , the graphical block-diagram modeling tool integrated
with MATLAB will be helpful but not essential.

1.6.4 Learning with the Book

The best way to learn is by doing. Although the book shows the MATLAB com-
mands and the response, there is something special about doing it for yourself.
Consider the book as an invitation to tinker. By running the code examples your-
self you can look at the results in ways that you prefer, plot the results in a different
way, or try the algorithm on different data or with different parameters. The paper
edition of the book is especially designed to stay open which enables you to type
in commands as you read. You can also look at the online documentation for the
 Another Random Document on
Scribd Without Any Related Topics
love any one?”
She turned very red, her heart was in her mouth, and she felt all
falsehood impossible before this emotion which was betraying her,
this repugnance for a lie which made the truth appear in her face in
spite of all.
“Yes,” she at last confessed, feebly. “But I beg you to let me go
away, sir, you are torturing me.”
She was now suffering in her turn. Was it not enough to have to
defend herself against him? Was she to be obliged to fight against
herself, against the breath of tenderness which sometimes took
away all her courage? When he spoke to her thus, when she saw
him so full of emotion, so overcome, she hardly knew why she still
refused; and it was only afterwards that she found, in the depths of
her healthy, girlish nature, the pride and the prudence which
maintained her intact in her virtuous resolution. It was by a sort of
instinct of happiness that she still remained so obstinate, to satisfy
her need of a quiet life, and not from any idea of virtue. She would
have fallen into this man's arms, her heart seduced, her flesh
overpowered if she had not experienced a sort of revolt, almost a
feeling of repulsion before the definite bestowal of her being,
ignorant of her future fate. The lover made her afraid, inspiring her
with that fear that all women feel at the approach of the male.
Mouret gave way to a gesture of gloomy discouragement. He
could not understand her. He turned towards his desk, took up some
papers and then laid them down again, saying: “I will retain you no
longer, mademoiselle; I cannot keep you against your will.”
“But I don't wish to go away,” replied she, smiling. “If you believe
me to be innocent, I will remain. One ought always to believe a
woman to be virtuous, sir. There are numbers who are so, I assure
you.”
Denise's eyes had involuntarily wandered towards Madame
Hédouin's portrait: that lady so wise and so beautiful, whose blood,
they said, had brought good fortune to the house. Mouret followed
the young girl's look with a start, for he thought he heard his dead
wife pronounce this phrase, one of her own sayings which he at
once recognised. And it was like a resurrection, he discovered in
Denise the good sense, the just equilibrium of her he had lost, even
down to the gentle voice, sparing of useless words. He was struck
by this resemblance, which rendered him sadder still.
“You know I am yours,” murmured he in conclusion. “Do what you
like with me.”
Then she resumed gaily: “That is right, sir. The advice of a
woman, however humble she may be, is always worth listening to
when she has a little intelligence. If you put yourself in my hands, be
sure I'll make nothing but a good man of you!”
She smiled, with that simple unassuming air which had such a
charm. He also smiled in a feeble way, and escorted her as far as the
door, as he would a lady.
The next day Denise was appointed first-hand. The dress and
costume department was divided, the management creating
especially for her one for children's costumes, which was installed
close to the ready-made one. Since her son's dismissal, Madame
Aurélie had been trembling, for she found the directors getting cool
towards her, and saw the young girl's power increasing daily. Would
they not shortly sacrifice her in favour of this latter, by taking
advantage of the first pretext? Her emperor's mask, puffed up with
fat, seemed to have got thinner from the shame which now stained
the whole Lhomme dynasty; and she made a show of going away
every evening on her husband's arm, for they were brought nearer
together by misfortune, and felt vaguely that the evil came from the
disorder of their home; whilst the poor old man, more affected than
her, in a sickly fear of being himself suspected of robbery, counted
over the receipts, again and again, noisily, performing miracles with
his amputated arm. So that, when she saw Denise appointed first-
hand in the children's costume department, she experienced such
joy that she paraded the most affectionate feeling towards the
young girl, really grateful to her for not having taken her place away.
And she overwhelmed her with attentions, treating her as an equal,
often going to talk to her in the neighbouring department, with a
stately air, like a queen-mother paying a visit to a young queen.
In fact, Denise was now at the summit. Her appointment as first-
hand had destroyed the last resistance. If some still babbled, from
that itching of the tongue which ravages every assemblage of men
and women, they bowed very low before her face. Marguerite, now
second-hand, was full of praise for her. Clara, herself, inspired with a
secret respect before this good fortune, which she felt herself
incapable of achieving, had bowed her head. But Denise's victory
was more complete still over the gentlemen; over Jouve, who now
bent almost double whenever he addressed her; over Hutin, seized
with anxiety on feeling his position giving way under him; and over
Bourdoncle, reduced at last to powerlessness. When the latter saw
her coming out of the director's office, smiling, with her quiet air,
and that the next day Mouret had insisted on the board creating this
new department, he had yielded, vanquished by a sacred terror of
woman. He had always given in thus before Mouret, recognising him
to be his master, notwithstanding his escapades and his idiotic love
affairs. This time the woman had proved the stronger, and he was
expecting to be swept away by the disaster.
However, Denise bore her triumph in a peaceable, charming
manner, happy at these marks of consideration, even affecting to
see in them a sympathy for the miseries of her debut and the final
success of her patient courage. Thus she received with a laughing
joy the slightest marks of friendship, and this caused her to be really
loved by some, she was so kind, sympathetic, and full of affection.
The only person for whom she still showed an invincible repugnance
was Clara, having learned that this girl had amused herself by taking
Colomban home with her one night as she had said she would do for
a joke; and he, carried away by his passion, was becoming more
dissipated every day, whilst poor Geneviève was slowly dying. The
adventure was talked of at The Ladies' Paradise, and thought very
droll.
 But this trouble, the only one she had outside, did not in any way
change Denise's equable temper. It was especially in her department
that she was seen at her best, in the midst of her little world of
babies of all ages. She was passionately fond of children, and she
could not have been placed in a better position. Sometimes there
were fully fifty girls and as many boys there, quite a turbulent
school, let loose in their growing coquettish desires. The mothers
completely lost their heads. She, conciliating, smiling, had the little
ones placed in a line, on chairs; and when there happened to be
amongst the number a rosy-cheeked little angel, whose pretty face
tempted her, she would insist on serving her herself, bringing the
dress and trying it on the child's dimpled shoulders, with the tender
precaution of an elder sister. There were fits of laughter, cries of joy,
amidst the scolding voices of the mothers. Sometimes a little girl,
already a grand lady, nine or ten years old, having a cloth jacket to
try on, would stand studying it before a glass, turning round, with an
absorbed air, her eyes sparkling with a desire to please. The
counters were encumbered with the things unpacked, dresses in
pink and blue Asian linen for children of from one to five years, blue
sailor costumes, with plaited skirt and blouse, trimmed with fine
cambric muslin, Louis XV. costumes, mantles, jackets, a pell-mell of
narrow garments, stiffened in their infantine grace, something like
the cloak-room of a regiment of big dolls, taken out of the
wardrobes and given up to pillage. Denise had always a few sweets
in her pockets, to appease the tears of some youngster in despair at
not being able to carry off a pair of red trousers; and she lived there
amongst these little ones as in her own family, feeling quite young
again herself from the contact of all this innocence and freshness
incessantly renewed around her skirts.
She now had frequent friendly conversations with Mouret. When
she went to the office to take orders and furnish information, he
kept her talking, enjoying the sound of her voice. It was what she
laughingly called “making a good man of him.” In her prudent,
cautious Norman head there sprang up all sorts of projects, ideas
about the new business which she had already ventured to hint at
when at Robineau's, and some of which she had expressed on the
evening of their walk in the Tuileries gardens. She could not be
occupied in any matter, see any work going on, without being moved
with a desire to introduce some improvement in the mechanism.
Then, since her entry into The Ladies' Paradise, she was especially
pained by the precarious position of the employees; the sudden
dismissals shocked her, she thought them iniquitous and stupid,
hurtful to all, to the house as much as to the staff. Her former
sufferings were still fresh in her mind, and her heart was seized with
pity every time she saw a new comer, her feet bruised, her eyes dim
with tears, dragging herself along in her misery in her silk dress,
amidst the spiteful persecution of the old hands. This dog's life made
the best of them bad; and the sad work of destruction commenced:
all eaten up by the trade before the age of forty, disappearing,
falling into unknown places, a great many dying in harness, some of
consumption and exhaustion, others of fatigue and bad air, a few
thrown on the street, the happiest married, buried in some little
provincial shop. Was it humane, was it just, this frightful
consumption of human life that the big shops carried on every year?
And she pleaded the cause of the wheel-work of the colossal
machine, not from any sentimental reasons, but by arguments
appealing to the very interests of the employers. To make a machine
solid and strong, it is necessary to use good iron; if the iron breaks
or is broken, there is a stoppage of work, repeated expenses of
starting, quite a loss of power.
Sometimes she would become quite animated, she would picture
an immense ideal bazaar, the phalansterium of modern commerce,
in which each one should have his exact share of the profits,
according to his merits, with the certainty of the future, assured to
him by a contract Mouret would feel amused at this, notwithstanding
his fever. He accused her of socialism, embarrassed her by pointing
out the difficulties of carrying out these schemes; for she spoke in
the simplicity of her soul, bravely trusting in the future, when she
perceived a dangerous hole underlying her tender-hearted plans. He
was, however, shaken, captivated by this young voice, still trembling
from the evils endured, so convinced and earnest in pointing out the
reforms which would tend to consolidate the house; yet he listened
while joking with her; the salesmen's position gradually improved,
the wholesale dismissals were replaced by a system of holidays
granted during the dead seasons, and there was also about to be
created a sort of benefit club which would protect the employees
against bad times and ensure them a pension. It was the embryo of
the vast trades' unions of the twentieth century.
Denise did not confine her attention solely to healing the wounds
from which she had herself bled; she conceived various delicate
feminine ideas, which, communicated to Mouret, delighted the
customers. She also caused Lhomme's happiness by supporting a
scheme he had long nourished, that of creating a band of music, in
which all the executants should be chosen from amongst the staff.
Three months later Lhomme had a hundred and twenty musicians
under his direction, the dream of his whole life was realised. And a
grand fête was given on the premises, a concert and a ball, to
introduce the band of The Ladies' Paradise to the customers and the
whole world. The newspapers took the matter up, Bourdoncle
himself, frightened by these innovations, was obliged to bow before
this immense advertisement. Afterwards, a recreation room for the
men was established, with two billiard tables and backgammon and
chess boards. Then classes were held in the house of an evening;
there were lessons in English and German, in grammar, arithmetic,
and geography; they even had lessons in riding and fencing. A
library was formed, ten thousand volumes were placed at the
disposal of the employees. And a resident doctor giving consultations
gratis was also added, together with baths, and hair-dressing and
refreshment saloons. Every want in life was provided for, everything
was to be obtained without going outside—board, lodging, and
clothing. The Ladies' Paradise sufficed entirely for all its own wants
and pleasures, in the very heart of Paris, taken up by all this clatter,
by this working city which was springing up so vigorously out of the
ruins of the old streets, at last opened to the rays of the sun.
 Then a fresh movement of opinion took place in Denise's favour.
As Bourdoncle, vanquished, repeated with despair to his friends that
he would give a great deal to put Denise into Mouret's arms himself,
it was concluded that she had not yielded, that her all-powerfulness
resulted from her refusal. From that moment she became immensely
popular. They knew for what indulgences they were indebted to her,
and they admired her for the force of her will. There was one, at
least, who could master the governor, who avenged all the others,
and knew how to get something else besides promises out of him!
So she had come at last, she who was to make him treat the poor
devils with a little respect! When she went through the shop, with
her delicate, self-willed head, her tender, invincible air, the salesmen
smiled at her, were proud of her, and would willingly have exhibited
her to the crowd. Denise, in her happiness, allowed herself to be
carried along by this increasing sympathy. Was it all possible? She
saw herself arrive in a poor dress, frightened, lost amidst the
mechanism of the terrible machine; for a long time she had had the
sensation of being nothing, hardly a grain of seed beneath these
millstones which were crushing a whole world; and now to-day she
was the very soul of this world, she alone was of consequence, able
at a word to increase or slacken the pace of the colossus lying at her
feet. And yet she had not wished for these things, she had simply
presented herself, without calculation, with the sole charm of her
sweetness. Her sovereignty sometimes caused her an uneasy
surprise; why did they all obey her? she was not pretty, she did
nothing wrong. Then she smiled, her heart at rest, feeling within
herself nothing but goodness and prudence, a love of truth and logic
which constituted all her strength.
One of Denise's greatest joys was to be able to assist Pauline. The
latter, being about to become a mother, was trembling, aware that
two other saleswomen in the same condition had been sent away.
The principals did not tolerate these accidents, maternity being
suppressed as cumbersome and indecent; they occasionally allowed
marriage, but would admit of no children. Pauline had, it was true,
her husband in the house; but still she felt anxious, it being almost
impossible for her to appear at the counter; and in order to
postpone a probable dismissal, she laced herself very tightly,
resolved to conceal her state as long as she could. One of the two
saleswomen who had been dismissed, had just been delivered of a
still-born child, through having laced herself up in this way; and it
was not certain that she herself would recover. Meanwhile,
Bourdoncle had observed that Pauline's complexion was getting very
livid, and that she had a painfully stiff way of walking. One morning
he was standing near her, in the under-linen department, when a
messenger, taking away a bundle, ran up against her with such force
that she cried out with pain. Bourdoncle immediately took her on
one side, made her confess, and submitted the question of her
dismissal to the board, under the pretext that she stood in need of
country air: the story of this accident would spread, and would have
a disastrous effect on the public if she should have a miscarriage, as
had already taken place in the baby linen department the year
before. Mouret, who was not at the meeting, could only give his
opinion in the evening. But Denise having had time to interfere, he
closed Bourdoncle's mouth, in the interest of the house itself. Did
they wish to frighten the heads of families and the young mothers
amongst their customers? And it was decided, with great pomp, that
every married saleswoman should, when in the family way, be sent
to a special midwife's as soon as her presence at the counter
became offensive to the customers.
The next day when Denise went up into the infirmary to see
Pauline, who had been obliged to take to her bed on account of the
blow she had received, the latter kissed her violently on both
cheeks. “How kind you are! Had it not been for you I should have
been turned away. Pray don't be anxious about me, the doctor says
it's nothing.”
Baugé, who had slipped away from his department, was also
there, on the other side of the bed. He likewise stammered his
thanks, troubled before Denise, whom he now treated as an
important person, of a superior class. Ah! if he heard any more nasty
remarks about her, he would soon close the mouths of the jealous
ones! But Pauline sent him away with a good-natured shrug of the
shoulders.
“My poor darling, you're always saying something stupid. Leave us
to talk together.”
The infirmary was a long, light room, containing twelve beds, with
their white curtains. Those who did not wish to go home to their
families were nursed here. But on the day in question, Pauline was
the only occupant, in a bed near one of the large windows which
looked on to the Rue Neuve-Saint-Augustin. And they immediately
commenced to exchange whispered words, tender confidences, in
the calm air, perfumed with a vague odour of lavender.
“So he does just what you wish him to? How cruel you are, to
make him suffer so! Come, just explain it to me, now I've ventured
to approach the subject. Do you detest him?” Pauline had retained
hold of Denise's hand, as the latter sat near the bed, with her elbow
on the bolster; and overcome by a sudden emotion, her cheeks
invaded with colour, she had a moment of weakness at this direct
and unexpected question. Her secret escaped her, she buried her
head in the pillow, murmuring:
“I love him!”
Pauline was astonished. “What! you love him? But it's very simple:
say yes.”
Denise, her face still concealed, replied “No!” by an energetic
shake of the head. And she did so, simply because she loved him,
without being able to explain the matter. No doubt it was ridiculous;
but she felt like that, she could not change her nature. Her friend's
surprise increased, and she at length asked: “So it's all to make him
marry you?”
At this the young girl sprung up, quite confused: “Marry me! Oh!
no! Oh! I assure you that I have never wished for anything of the
kind! No, never has such an idea entered my head; and you know
what a horror I have of all falsehood!”
 “Well, dear,” resumed Pauline, kindly, “you couldn't have acted
otherwise, if such had been your intention. All this must come to an
end, and it is very certain that it can only finish by a marriage, as
you won't let it be otherwise. I must tell you that every one has the
same idea; yes, they feel persuaded that you are riding the high
horse, in order to make him take you to church. Dear me! what a
funny girl you are!”
And she had to console Denise, who had again dropped her head
on to the bolster, sobbing, declaring that she would certainly go
away, since they attributed all sorts of things to her that had never
crossed her mind. No doubt, when a man loved a woman he ought
to marry her. But she asked for nothing, she had made no
calculations, she simply begged to be allowed to live quietly, with her
joys and her sorrows, like other people. She would go away.
At the same moment Mouret was going through the premises
below. He had wanted to forget his thoughts by visiting the works
once more. Several months had elapsed, the façade now reared its
monumental lines behind the vast hoardings which concealed it from
the public. Quite an army of decorators were at work: marble-
cutters, mosaic-workers, and others. The central group above the
door was being gilded; whilst on the acroteria were being fixed the
pedestals destined to receive the statues of the manufacturing cities
of France. From morning to night, in the Rue du Dix-Décembre,
lately opened to the public, a crowd of idlers stood gaping about,
their noses in the air, seeing nothing, but pre-occupied by the
marvels that were related of this façade, the inauguration of which
was going to revolutionise Paris. And it was on this feverish working-
ground, amidst the artists putting the finishing touches to the
realisation of his dream commenced by the masons, that Mouret felt
more bitterly than ever the vanity of his fortune. The thought of
Denise had suddenly arrested him, this thought which incessantly
pierced him with a flame, like the shooting of an incurable pain. He
had run away, unable to find a word of satisfaction, fearful lest he
should show his tears, leaving behind him the disgust of his triumph.
This façade, which was at last erected, seemed little in his eyes, very
much like one of those walls of sand that children build, and it might
have been extended from one end of the city to the other, elevated
to the starry sky, yet it would not have filled the emptiness of his
heart, that the “yes” of a mere child could alone fill.
When Mouret entered his office he was almost choking with sobs.
What did she want? He dared not offer her money now; and the
confused idea of a marriage presented itself amidst his young
widower's revolts. And, in the debility of his powerlessness, his tears
began to flow. He was very miserable.
 CHAPTER XIII.

O
ne morning in November, Denise was giving her first orders in
the department when the Baudus' servant came to tell her
that Mademoiselle Geneviève had passed a very bad night,
and wished to see her cousin immediately. For some time the young
girl had been getting weaker and weaker, and she had been obliged
to take to her bed two days before.
“Say I am coming at once,” replied Denise, very anxious.
The blow which was finishing Geneviève was Colomban's sudden
disappearance. At first, chaffed by Clara, he had stopped out several
nights; then, yielding to the mad desires of a quiet, chaste fellow, he
had become her obedient slave, and had not returned one Monday,
but had simply sent a farewell letter to Baudu, written in the studied
terms of a man about to commit suicide. Perhaps, at the bottom of
this passion, there was also the crafty calculation of a fellow
delighted at escaping a disastrous marriage. The draper's business
was in as bad a way as his betrothed; the moment was propitious to
break with them through any stupidity. And every one cited him as
an unfortunate victim of love.
When Denise arrived at The Old Elbeuf, Madame Baudu was there
alone, sitting motionless behind the pay-desk, with her small white
face, eaten up by anæmia, silent and quiet in the cold, deserted
shop. There were no assistants now. The servant dusted the shelves,
and it was even a question of replacing her by a charwoman. A
dreary cold fell from the ceiling, hours passed away without a
customer coming to disturb this silence, and the goods, no longer
touched, became mustier and mustier every day.
“What's the matter?” asked Denise, anxiously. “Is Geneviève in
danger?”
 Madame Baudu did not reply at first. Her eyes filled with tears.
Then she stammered: “I don't know; they don't tell me anything. Ah,
it's all over, it's all over.”
And she cast a sombre glance around the dark old shop, as if she
felt her daughter and the shop disappearing together. The seventy
thousand francs, produce of the sale of their Rambouillet property,
had melted away in less than two years in this gulf of competition.
In order to struggle against The Ladies' Paradise, which now kept
men's cloths and materials for hunting and livery suits, the draper
had made considerable sacrifices. At last he had been definitely
crushed by the swanskin cloth and flannels sold by his rival, an
assortment that had not its equal in the market. Little by little his
debts had increased, and, as a last resource, he had resolved to
mortgage the old building in the Rue de la Michodière, where Finet,
their ancestor, had founded the business; and it was now only a
question of days, the crumbling away had commenced, the very
ceilings seemed to be falling down and turning into dust, like an old
worm-eaten structure carried away by the wind.
“Your uncle is upstairs,” resumed Madame Baudu in her broken
voice. “We stay with her two hours each. Some one must look out
here; oh! but only as a precaution, for to tell the truths——”
Her gesture finished the phrase. They would have put the shutters
up had it not been for their old commercial pride, which still propped
them up in the presence of the neighbourhood.
“Well, I'll go up, aunt,” said Denise, whose heart was bleeding,
amidst this resigned despair that even the pieces of cloth themselves
exhaled.
“Yes, go upstairs quick, my girl. She's waiting for you. She's been
asking for you all night. She has something to tell you.”
But just at that moment Baudu came down. The rising bile gave
his yellow face a greenish tinge, and his eyes were bloodshot. He
was still walking with the muffled step with which he had quitted the
Sick room, and murmur-ed, as if he might be heard upstairs, “She's
asleep.”
And, thoroughly worn out, he sat down on a chair, wiping his
forehead with a mechanical gesture, puffing like a man who has just
finished some hard work. A silence ensued, but at last he said to
Denise: “You'll see her presently. When she is sleeping, she seems to
me to be all right again.”
There was again a silence. Face to face, the father and mother
stood looking at each other. Then, in a half whisper, he went over his
grief again, naming no one, addressing no one directly: “My head on
the block, I wouldn't have believed it! He was the last one. I had
brought him up as a son. If any one had come and said to me,
'They'll take him away from you as well; he'll fall as well,' I would
have replied 'Impossible, it could not be.' And he has fallen all the
same! Ah! the scoundrel, he who was so well up in real business,
who had all my ideas! And all for a young monkey, one of those
dummies that parade at the windows of bad houses! No! really, it's
enough to drive one mad!”
He shook his head, his eyes fell on the damp floor worn away by
generations of customers. Then he continued in a lower voice,
“There are moments when I feel myself the most culpable of all in
our misfortune. Yes, it's my fault if our poor girl is upstairs devoured
by fever. Ought not I to have married them at once, without yielding
to my stupid pride, my obstinacy in refusing to leave them the house
less prosperous than before? Had I done that she would now have
the man she loved, and perhaps their united youthful strength would
have accomplished the miracle that I have failed to work. But I am
an old fool, and saw through nothing; I didn't know that people fell
ill over such things. Really he was an extraordinary fellow: with such
a gift for business, and such probity, such simplicity of conduct, so
orderly in every way—in short, my pupil.”
He raised his head, still defending his ideas, in the person of the
shopman who had betrayed him. Denise could not bear to hear him
accuse himself, and she told him so, carried away by her emotion,
on seeing him so humble, with his eyes full of tears, he who used
formerly to reign as absolute master.
“Uncle, pray don't apologise for him. He never loved Geneviève,
he would have run away sooner if you had tried to hasten the
marriage. I have spoken to him myself about it; he was perfectly
well aware that my cousin was suffering on his account, and you see
that did not prevent him leaving. Ask aunt.”
Without opening her lips, Madame Baudu confirmed these words
by a nod. The draper turned paler still, blinded by his tears. He
stammered out: “It must be in the blood, his father died last year
through having led a dissolute life.”
And he once more looked round the obscure shop, his eyes
wandering from the empty counters to the full shelves, then resting
on Madame Baudu, who was still at the pay-desk, waiting in vain for
the customers who did not come.
“Come,” said he, “it's all over. They've ruined our business, and
now one of their hussies is killing our daughter.”
No one spoke. The rolling of the vehicles, which occasionally
shook the floor, passed like a funereal beating of drums in the still
air, stifled under the low ceiling. Suddenly, amidst this gloomy
sadness of the old dying shop, could be heard several heavy knocks,
struck somewhere in the house. It was Geneviève, who had just
awoke, and was knocking with a stick they had left near her bed.
“Let's go up at once,” said Baudu, rising with a start. “Try and be
cheerful, she mustn't know.”
He himself rubbed his eyes to efface the trace of his tears. As
soon as he had opened the door, on the first storey, they heard a
frightened, feeble voice crying: “Oh, I don't like to be left alone.
Don't leave me; I'm afraid to be left alone.” Then, when she
perceived Denise, Geneviève became calmer, and smiled joyfully.
“You've come, then! How I've been longing to see you since
yesterday. I thought you also had abandoned me!”
 It was a piteous sight. The young girl's room looked out on to the
yard, a little room lighted by a livid light At first her parents had put
her in their own room, in the front; but the sight of The Ladies'
Paradise opposite affected her so much, that they had been obliged
to bring her back to her own again. And there she lay, so very thin,
under the bed-clothes, that one hardly suspected the form and
existence of a human body. Her skinny arms, consumed by a
burning fever, were in a perpetual movement of anxious,
unconscious searching; whilst her black hair seemed thicker still, and
to be eating up her poor face with its voracious vitality, that face in
which was agonising the final degenerateness of a family sprung up
in the shade, in this cellar of old commercial Paris. Denise, her heart
bursting with pity, stood looking at her. She did not at first speak, for
fear of giving way to tears. At last she murmured:
“I came at once. Can I be of any use to you? You asked for me.
Would you like me to stay?”
“No, thanks. I don't want anything. I only wanted to embrace
you.”
Tears filled her eyes. Denise quickly leant over, and kissed her on
both cheeks, trembling to feel on her lips the flame of those hollow
cheeks. But Geneviève, stretching out her arms, seized and kept her
in a desperate embrace. Then she looked towards her father.
“Would you like me to stay?” repeated Denise. “Perhaps there is
something I can do for you.”
Geneviève's glance was still obstinately fixed on her father, who
remained standing, with a stolid air, almost choking. He at last
understood, and went away, without saying a word; and they heard
his heavy footstep on the stairs.
“Tell me, is he with that woman?” asked the sick girl immediately,
seizing her cousin's hand, and making her sit on the side of the bed.
“I want to know, and you are the only one can tell me. They're living
together, aren't they?” Denise, surprised by these questions,
stammered, and was obliged to confess the truth, the rumours that
were current in the shop. Clara, tired of this fellow, who was getting
a nuisance to her, had already broken with him, and Colomban,
desolated, was pursuing her everywhere, trying to obtain a meeting
from time to time, with a sort of canine humility. They said that he
was going to take a situation at the Grands Magasins du Louvre.
“If you still love him, he may return,” said Denise, to cheer the
dying girl with this last hope. “Get well quick, he will acknowledge
his errors, and marry you.”
Geneviève interrupted her. She had listened with all her soul, with
an intense passion that raised her in the bed. But she fell back
almost immediately. “No, I know it's all over! I don't say anything,
because I see papa crying, and I don't wish to make mamma worse
than she is. But I am going, Denise, and if I called for you last night
it was for fear of going off before the morning. And to think that he
is not happy after all!”
And Denise having remonstrated, assuring her that she was not so
bad as all that, she cut her short again, suddenly throwing off the
bed-clothes with the chaste gesture of a virgin who has nothing to
conceal in death. Naked to the waist, she murmured: “Look at me!
Is it possible?”
Trembling, Denise quitted the side of the bed, as if she feared to
destroy this fearful nudity with a breath. It was the last of the flesh,
a bride's body used up by waiting, returned to the first infantile
slimness of her young days. Geneviève slowly covered herself up
again, saying: “You see I am no longer a woman. It would be wrong
to wish for him still!” There was a silence. Both continued to look at
each other, unable to find a word to say. It was Geneviève who
resumed: “Come, don't stay any longer, you have your own affairs to
look after. And thanks, I was tormented by the wish to know, and
am now satisfied. If you see him, tell him I forgive him. Adieu, dear
Denise. Kiss me once more, for it's the last time.” The young girl
kissed her, protesting: “No, no, don't despair, all you want is loving
care, nothing more.” But the sick girl, shaking her head in an
obstinate way, smiled, quite sure of what she said. And as her cousin
was making for the door, she exclaimed: “Wait a minute, knock with
this stick, so that papa may come up. I'm afraid to stay alone.”
Then, when Baudu arrived in that small, gloomy room, where he
spent hours seated on a chair, she assumed an air of gaiety, saying
to Denise—“Don't come to-morrow, I would rather not. But on
Sunday I shall expect you; you can spend the afternoon with me.”
The next morning, at six o'clock, Geneviève expired after four
hours' fearful agony. The funeral took place on a Saturday, a
fearfully black, gloomy day, under a sooty sky which hung over the
shivering city. The Old Elbeuf, hung with white linen, lighted up the
street with a bright spot, and the candles burning in the fading day
seemed so many stars drowned in the twilight The coffin was
covered with wreaths and bouquets of white roses; it was a narrow
child's coffin, placed in the obscure passage of the house on a level
with the pavement, so near the gutter that the passing carriages had
already splashed the coverings. The whole neighbourhood exhaled a
dampness, a cellar-like mouldy odour, with its continual rush of
pedestrians on the muddy pavement.
At nine o'clock Denise came over to stay with her aunt. But as the
funeral was starting, the latter—who had ceased weeping, her eyes
burnt with tears—begged her to follow the body and look after her
uncle, whose mute affliction and almost idiotic grief filled the family
with anxiety. Below, the young girl found the street full of people, for
the small traders in the neighbourhood were anxious to show the
Baudus a mark of sympathy, and in this eagerness there was also a
sort of manifestation against The Ladies' Paradise, whom they
accused of causing Geneviève's slow agony. All the victims of the
monster were there—Bédoré and sister from the hosier's shop in the
Rue Gaillon, the furriers, Vanpouille Brothers, and Deslignières the
toyman, and Piot and Rivoire the furniture dealers; even
Mademoiselle Tatin from the underclothing shop, and the glover
Quinette, long since cleared off by bankruptcy, had made it a duty to
come, the one from Batignolle, the other from the Bastille, where
they had been obliged to take situations. Whilst waiting for the
hearse, which was late, these people, tramping about in the mud,
cast glances of hatred towards The Ladies' Paradise, the bright
windows and gay displays of which seemed an insult in face of The
Old Elbeuf, which, with its funeral trappings and glimmering candles,
cast a gloom over the other side of the street A few curious faces
appeared at the plate-glass windows; but the colossus maintained
the indifference of a machine going at full speed, unconscious of the
deaths it may cause on the road.
Denise looked round for her brother Jean, whom she at last
perceived standing before Bourras's shop, and she went and asked
him to walk with his uncle, to assist him if he could not get along.
For the last few weeks Jean had been very grave, as if tormented by
some worry. To-day, buttoned up in his black frock-coat, a full grown
man, earning his twenty francs a day, he seemed so dignified and so
sad that his sister was surprised, for she had no idea he loved his
cousin so much as that. Desirous of sparing Pépé this needless grief,
she had left him with Madame Gras, intending to go and fetch him in
the afternoon to see his uncle and aunt.
The hearse had still not arrived, and Denise, greatly affected, was
watching the candles burn, when she was startled by a well-known
voice behind her. It was Bourras. He had called the chestnut-seller
opposite, in his little box, against the public-house, and said to him:
“I say, Vigouroux, just keep a look-out for me a bit, will you? You
see I've closed the door. If any one comes tell them to call again.
But don't let that disturb you, no one will come.”
Then he took his stand on the pavement, waiting like the others.
Denise, feeling rather awkward, glanced at his shop. He entirely
abandoned it now; there was nothing left but a disorderly array of
umbrellas eaten up by the damp air, and canes blackened by the
gas. The embellishments that he had made, the delicate green paint
work, the glasses, the gilded sign, were all cracking, already getting
dirty, presenting that rapid and lamentable decrepitude of false
luxury laid over ruins. But though the old crevices were re-
appearing, though the spots of damp had sprung up over the
gildings, the house still held its ground obstinately, hanging on to the
flanks of The Ladies' Paradise like a dishonouring wart, which,
although cracked and rotten, refused to fall off.
“Ah! the scoundrels,” growled Bourras, “they won't even let her be
carried away.”
The hearse, which had at last arrived, had just got into collision
with one of The Ladies' Paradise vans, which was spinning along,
shedding in the mist its starry radiance, with the rapid trot of two
superb horses. And the old man cast on Denise an oblique glance,
lighted up under his bushy eyebrows. Slowly, the funeral started off,
splashing through the muddy pools, amid the silence of the
omnibuses and carriages suddenly pulled up. When the coffin,
draped with white, crossed the Place Gaillon, the sombre looks of
the cortege were once more plunged into the windows of the big
shop, where two saleswomen alone had run up to look on, pleased
at this distraction. Baudu followed the hearse with a heavy
mechanical step, refusing by a sign the arm offered by Jean, who
was walking with him. Then, after a long-string of people, came
three mourning coaches. As they passed the Rue Neuve-des-Petits-
Champs, Robineau ran up to join the cortege, very pale, and looking
much older.
At Saint-Roch, a great many women were waiting, the small
traders of the neighbourhood, who had been afraid of the crowd at
the house. The manifestation was developing into quite a riot; and
when, after the service, the procession started off back, all the men
followed, although it was a long walk from the Rue Saint-Honoré to
the Montmartre Cemetery. They had to go up the Rue Saint-Roch,
and once more pass The Ladies' Paradise. It was a sort of obsession;
this poor young girl's body was paraded round the big shop like the
first victim fallen in time of revolution. At the door some red flannels
were flapping like so many flags, and a display of carpets blazed
forth in a florescence of enormous roses and full-blown pæonies.
Denise had got into one of the coaches, being agitated by some
smarting doubts, her heart oppressed by such a feeling of grief that
she had not the strength to walk At that moment there was a stop,
in the Rue du Dix-Décembre, before the scaffolding of the new
façade which still obstructed the thoroughfare. 'And the young girl
observed old Bourras, left behind, dragging along with difficulty,
close to the wheels of the coach in which she was riding alone. He
would never get as far as the cemetery, she thought. He raised his
head, looked at her, and all at once got into the coach.
“It's my confounded knees,” exclaimed he. “Don't draw back! Is it
you that we detest?”
She felt him to be friendly and furious as in former days. He
grumbled, declared that Baudu must be fearfully strong to be able to
keep up after such blows as he had received. The procession had
resumed its slow pace; and on leaning out, Denise saw her uncle
walking with his heavy step, which seemed to regulate the rumbling
and painful march of the cortege. She then threw herself back into
the corner, listening to the endless complaints of the old umbrella
maker, rocked by the melancholy movement of the coach.
“The police ought to clear the public thoroughfare, my word!
They've been blocking up our street for the last eighteen months
with the scaffolding of their façade, where a man was killed the
other day. Never mind! When they want to enlarge further they'll
have to throw bridges over the street. They say there are now two
thousand seven hundred employees, and that the business will
amount to a hundred millions this year. A hundred millions! Just
fancy, a hundred millions!”
Denise had nothing to say in reply. The procession had just turned
into the Rue de la Chaussée d'Antin, where it was stopped by a block
of vehicles. Bourras went on, with a vague expression in his eyes, as
if he were dreaming aloud. He still failed to understand the triumph
achieved by The Ladies' Paradise, but he acknowledged the defeat of
the old-fashioned traders.
“Poor Robineau's done for, he's got the face of a drowning man.
And the Bédorés and the Vanpouilles, they can't keep going; they're
like me, played out Deslignières will die of apoplexy. Piot and Rivoire
have the yellow jaundice. Ah! we're a fine lot; a pretty cortege of
skeletons to follow the poor child. It must be comical for those
looking on to see this string of bankrupts pass. Besides, it appears
that the clean sweep is to continue. The scoundrels are creating
departments for flowers, bonnets, perfumery, shoemaking, all sorts
of things. Grognet, the perfumer in the Rue de Grammont, can clear
out, and I wouldn't give ten francs for Naud's shoe-shop in the Rue
d'Antin. The cholera has spread as far as the Rue Sainte-Anne,
where Lacassagne, at the feather and flower shop, and Madame
Chadeuil, whose bonnets are so well-known, will be swept away
before long. And after those, others; it will still go on! All the
businesses in the neighbourhood will suffer. When counter-jumpers
commence to sell soap and goloshes, they are quite capable of
dealing in fried potatoes. My word, the world is turning upside
down!”
The hearse was just then crossing the Place de la Trinité to ascend
the steep Rue Blanche, and from the corner of the gloomy coach
Denise, who, broken-hearted, was listening to the endless
complaints of the old man, could see the coffin as they issued from
the Rue de la Chaussée d'Antin. Behind her uncle, marching along
with the blind, mute face of an ox about to be poleaxed, she seemed
to hear the tramping of a flock of sheep led to the slaughter-house,
the discomfiture of the shops of a whole district, the small traders
dragging along their ruin, with the thud of damp shoes, through the
muddy streets of Paris. Bourras still went on, in a deeper voice, as if
slackened by the difficult ascent of the Rue Blanche.
“As for me, I am settled. But I still hold on all the same, and won't
let go. He's just lost his appeal case. Ah! that's cost me something,
what with nearly two years' pleading, and the solicitors and the
barristers! Never mind, he won't pass under my shop, the judges
have decided that such a work could not be considered as a
legitimate case of repairing. Fancy, he talked of creating underneath
a light saloon to judge the colours of the stuffs by gas-light, a
subterranean room which would have united the hosiery to the
drapery department! And he can't get over it; he can't swallow the
fact that an old humbug like me should stop his progress when
everybody are on their knees before his money. Never! I won't!
that's understood. Very likely I may be worsted. Since I have had to
go to the money-lenders, I know the villain is looking after my paper,
in the hope to play me some villanous trick, no doubt. But that
doesn't matter. He says 'yes,' and I say 'no,' and shall still say 'no,'
even when I get between two boards like this poor little girl who has
just been nailed up.”
When they reached the Boulevard de Clichy, the coach went at a
quicker pace; one could hear the heavy breathing of the mourners,
the unconscious haste of the cortege, anxious to get the sad
ceremony over. What Bourras did not openly mention, was the
frightful misery into which he had fallen, bewildered amidst the
confusion of the small trader who is on the road to ruin and yet
remains obstinate, under a shower of protested bills. Denise, well
acquainted with his situation, at last interrupted the silence by
saying, in a voice of entreaty:
“Monsieur Bourras, pray don't stand out any longer. Let me
arrange matters for you.”
But he interrupted her with a violent gesture. “You be quiet. That's
nobody's business. You're a good little girl, and I know you lead him
a hard life, this man who thought you were for sale like my house.
But what would you answer if I advised you to say 'yes?' You'd send
me about my business. Therefore, when I say 'no,' don't you
interfere in the matter.”
And the coach having stopped at the cemetery gate, he got out
with the young girl. The Baudus' vault was situated in the first alley
on the left. In a few minutes the ceremony was terminated. Jean
had drawn away his uncle, who was looking into the grave with a
gaping air. The mourners wandered about amongst the neighbouring
tombs, and the faces of all these shopkeepers, their blood
impoverished by living in their unhealthy shops assumed an ugly
suffering look under the leaden sky. When the coffin slipped gently
down, their blotched and pimpled cheeks paled, and their bleared
eyes, blinded with figures, turned away.
“We ought all to jump into this hole,” said Bourras to Denise, who
had kept close to him. “In burying this poor girl they are burying the
whole district. Oh! I know what I am saying, the old-fashioned
business may go and join the white roses they are throwing on to
her coffin.”
Denise brought back her uncle and brother in a mourning coach.
The day was for her exceedingly dull and melancholy. In the first
place, she began to get anxious at Jean's paleness, and when she
understood that it was on account of another woman, she tried to
quiet him by opening her purse, but he shook his head and refused,
saying it was serious this time, the niece of a very rich pastry-cook,
who would not accept even a bunch of violets. Afterwards, in the
afternoon, when Denise went to fetch Pépé from Madame Gras's,
the latter declared that he was getting too big for her to keep any
longer; another annoyance, for she would be obliged to find him a
school, perhaps send him away. And to crown all she was thoroughly
heart-broken, on bringing Pépé back to kiss his aunt and uncle, to
see the gloomy sadness of The Old Elbeuf. The shop was closed, and
the old couple were at the further end of the little room, where they
had forgotten to light the gas, notwithstanding the complete
obscurity of this winter's day. They were now quite alone, face to
face, in the house, slowly emptied by ruin; and the death of their
daughter deepened the shady corners, and was like the supreme
cracking which was soon to break up the old rafters, eaten away by
the damp. Beneath this destruction, her uncle, unable to stop
himself, still kept walking round the table, with his funeral-like step,
blind and silent; whilst her aunt said nothing, she had fallen into a
chair, with the white face of a wounded person, whose blood was
running away drop by drop. They did not even weep when Pépé
covered their cold cheeks with kisses. Denise was choked with tears.
That same evening Mouret sent for the young girl to speak of a
child's garment he wished to launch forth, a mixture of the Scotch
and Zouave costumes. And still trembling with pity, shocked at so
much suffering, she could not contain herself; she first ventured to
speak of Bourras, of that poor old man whom they were about to
ruin. But, on hearing the umbrella maker's name, Mouret flew into a
rage at once. The old madman, as he called him, was the plague of
his life, and spoilt his triumph by his idiotic obstinacy in not giving up
his house, that ignoble hovel which was a disgrace to The Ladies'
Paradise, the only little corner of the vast block that escaped his
conquest. The matter was becoming a regular nightmare; any one
else but Denise speaking in favour of Bourras would have run the
risk of being dismissed immediately, so violently was Mouret tortured
by the sickly desire to kick the house down. In short, what did they
wish him to do? Could he leave this heap of ruins sticking to The
Ladies' Paradise? It would be got rid of, the shop was to pass
through it. So touch the worse for the old fool! And he spoke of his
repeated proposals; he had offered him as much as a hundred
thousand francs. Wasn't that fair? He never higgled, he gave the
money required; but in return he expected people to be reasonable,
and allow him to finish his work! Did any one ever try to stop the
locomotives on a railway? She listened to him, with drooping eyes,
unable to find any but purely sentimental reasons. The old man was
so old, they might have waited till his death; a failure would kill him.
Then he added that he was no longer able to prevent things going
their course. Bourdoncle had taken the matter up, for the board had
resolved to put an end to it. She had nothing more to add,
notwithstanding the grievous pity she felt for her old friend.
After a painful silence, Mouret himself commenced to speak of the
Baudus, by expressing his sorrow at the death of their daughter.
They were very worthy people, very honest, but had been pursued
by the worst of luck. Then he resumed his arguments; at bottom,
they had really caused their own misfortune by obstinately sticking
to the old ways in their worm-eaten place; it was not astonishing
that the place should be falling about their heads. He had predicted
it scores of times; she must remember that he had charged her to
warn her uncle of a fatal disaster, if the latter still clung to his old-
fashioned stupid ways. And the catastrophe had arrived; no one in
the world could now prevent it They could not reasonably expect
him to ruin himself to save the neighbourhood. Besides, if he had
been foolish enough to close The Ladies' Paradise, another big shop
would have sprung up of itself next door, for the idea was now
starting from the four corners of the globe; the triumph of these
manufacturing and industrial cities was sown by the spirit of the
times, which was sweeping away the tumbling edifice of former
ages. Little by little Mouret warmed up, and found an eloquent
emotion with which to defend himself against the hatred of his
involuntary victims, the clamour of the small dying shops that was
heard around him. They could not keep their dead, he continued,
they must bury them; and with a gesture he sent down into the
grave, swept away and threw into the common hole the corpse of
old-fashioned business, the greenish, poisonous remains of which
were becoming a disgrace to the bright, sun-lighted streets of new
Paris. No, no, he felt no remorse, he was simply doing the work of
his age, and she knew it; she, who loved life, who had a passion for
big affairs, concluded in the full glare of publicity. Reduced to
silence, she listened to him for some time, and then went away, her
soul full of trouble.
That night Denise slept but little. A sleeplessness, traversed by
nightmare, kept her turning over and over in her bed. It seemed to
her that she was quite little, and she burst into tears, in their garden
at Valognes, on seeing the blackcaps eat up the spiders, which
themselves devoured the flies. Was it then really true, this necessity
for the world to fatten on death, this struggle for existence which
drove people into the charnel-house of eternal destruction?
Afterwards she saw herself before the vault into which they had
lowered Geneviève, then she perceived her uncle and aunt in their
obscure dining-room. In the profound silence, a heavy voice, as of
something tumbling down, traversed the dead air; it was Bourras's
house giving way, as if undermined by a high tide. The silence
recommenced, more sinister than ever, and a fresh rumbling was
heard, then another, then another; the Robineaus, the Bédorés, the
Vanpouilles, cracked and fell down in their turn, the small shops of
the neighbourhood were disappearing beneath an invisible pick, with
a brusque, thundering noise, as of a tumbril being emptied. Then an
immense pity awoke her with a start. Heavens! what tortures! There
were families weeping, old men thrown out into the street, all the
poignant dramas that ruin conjures up. And she could save nobody;
and she felt that it was right, that all this misery was necessary for
the health of the Paris of the future. When day broke she became
calmer, a feeling of resigned melancholy kept her awake, turned
towards the windows through which the light was making its way.
Yes, it was the need of blood that every revolution exacted from its
martyrs, every step forward was made over the bodies of the dead.
Her fear of being a wicked girl, of having assisted in the ruin of her
fellow-creatures, now melted into a heartfelt pity, in face of these
evils without remedy, which are the painful accompaniment of each
generation's birth. She finished by seeking some possible comfort in
her goodness, she dreamed of the means to be employed in order to
save her relations at least from the final crash.
Mouret now appeared before her with his passionate face and
caressing eyes. He would certainly refuse her nothing; she felt sure
he would accord her all reasonable compensation. And her thoughts
went astray in trying to judge him. She knew his life, was aware of
the calculating nature of his former affections, his continual
exploitation of woman, mistresses taken up to further his own ends,
and his intimacy with Madame Desforges solely to get hold of Baron
Hartmann, and all the others, such as Clara and the rest, pleasure
bought, paid for, and thrown out on the pavement. But these
beginnings of a love adventurer, which were the talk of the shop,
were gradually effaced by the strokes of genius of this man, his
victorious grace. He was seduction itself. What she could never have
forgiven was his former deception, his lover's coldness under the
gallant comedy of his attentions. But she felt herself to be entirely
without rancour, now that he was suffering through her. This
suffering had elevated him. When she saw him tortured by her
refusal, atoning so fully for his former disdain for woman, he seemed
to have made amends for all his faults.
That morning Denise obtained from Mouret the compensation she
might judge legitimate the day the Baudus and old Bourras should
succumb. Weeks passed away, during which she went to see her
uncle nearly every afternoon, escaping from her counter for a few
minutes, bringing her smiling face and brave courage to enliven the
sombre shop. She was especially anxious about her aunt, who had
fallen into a dull stupor since Geneviève's death; it seemed that her
life was quitting her hourly; and when people spoke to her she
would reply with an astonished air that she was not suffering, but
that she simply felt as if overcome by sleep. The neighbours shook
their heads, saying she would not live long to regret her daughter.
One day Denise was coming out of the Baudus', when, on turning
the corner of the Place Gaillon, she heard a loud cry. The crowd
rushed forward, a panic arose, that breath of fear and pity which so
suddenly seizes a crowd. It was a brown omnibus, belonging to the
Bastille-Batignolles line, which had run over a man, coming out of
the Rue Neuve-Saint-Augustin, opposite the fountain. Upright on his
seat, with furious gestures, the driver was pulling in his two kicking
horses, and crying out, in a great passion:
“Confound you! Why don't you look out, you idiot!”
The omnibus had now stopped, and the crowd had surrounded
the wounded man, and, strange to say, a policeman was soon on the
spot. Still standing up, invoking the testimony of the people on the
knife-board, who had also got up, to look over and see the wounded
man, the coachman was explaining the matter, with exasperated
gestures, choked by his increasing anger.
“It's something fearful. This fellow was walking in the middle of
the road, quite at home. I called out, and he at once threw himself
under the wheels!”
A house-painter, who had run up, brush in hand, from a
neighbouring house, then said, in a sharp voice, amidst the clamour:
“Don't excite yourself. I saw him, he threw himself under. He jumped
in, head first. Another unfortunate tired of life, no doubt.”
Others spoke up, and all agreed upon it being a case of suicide,
whilst the policeman pulled out his book and made his entry. Several
ladies, very pale, got out quickly, and ran away without looking back,
filled with horror by the soft shaking which had stirred them up
when the omnibus passed over the body. Denise approached,
attracted by a practical pity, which prompted her to interest herself
in all sorts of street accidents, wounded dogs, horses down, and
tilers falling off roofs. And she immediately recognised the
unfortunate fellow who had fainted away, his clothes covered with
mud.
“It's Monsieur Robineau,” cried she, in her grievous astonishment.
The policeman at once questioned the young girl, and she gave
his name, profession, and address. Thanks to the driver's energy,
the omnibus had twisted round, and thus only Robineau's legs had
gone under the wheels, but it was to be feared that they were both
broken. Four men carried the wounded draper to a chemist's shop in
the Rue Gaillon, whilst the omnibus slowly resumed its journey.
“My stars!” said the driver, whipping up his horses, “I've done a
famous day's work.”
Denise followed Robineau into the chemist's. The latter, waiting for
a doctor who could not be found, declared there was no immediate
danger, and that the wounded man had better be taken home, as he
lived in the neighbourhood. A lad started off to the police-station to
order a stretcher, and Denise had the happy thought of going on in
front and preparing Madame Robineau for this frightful blow. But she
had the greatest trouble in the world to get into the street through
the crowd, which was struggling before the door. This crowd,
attracted by death, was increasing every minute; men, women, and
children stood on tip-toe, and held their own amidst a brutal
pushing, and each new comer had his version of the accident, so