Developmental robotics

Developmental robotics (DevRob), sometimes called epigenetic robotics, is a scientific field which aims at studying the developmental mechanisms, architectures and constraints that allow lifelong and open-ended learning of new skills and new knowledge in embodied machines. As in human children, learning is expected to be cumulative and of progressively increasing complexity, and to result from self-exploration of the world in combination with social interaction. The typical methodological approach consists in starting from theories of human and animal development elaborated in fields such as developmental psychology, neuroscience, developmental and evolutionary biology, and linguistics, then to formalize and implement them in robots, sometimes exploring extensions or variants of them. The experimentation of those models in robots allows researchers to confront them with reality, and as a consequence, developmental robotics also provides feedback and novel hypotheses on theories of human and animal development.

Developmental robotics is related to but differs from evolutionary robotics (ER). ER uses populations of robots that evolve over time, whereas DevRob is interested in how the organization of a single robot's control system develops through experience, over time.

DevRob is also related to work done in the domains of robotics and artificial life.

Background

Can a robot learn like a child? Can it learn a variety of new skills and new knowledge unspecified at design time and in a partially unknown and changing environment? How can it discover its body and its relationships with the physical and social environment? How can its cognitive capacities continuously develop without the intervention of an engineer once it is "out of the factory"? What can it learn through natural social interactions with humans? These are the questions at the center of developmental robotics. Alan Turing, as well as a number of other pioneers of cybernetics, already formulated those questions and the general approach in 1950, but it is only since the end of the 20th century that they began to be investigated systematically.

Because the concept of adaptive intelligent machines is central to developmental robotics, it has relationships with fields such as artificial intelligence, machine learning, cognitive robotics or computational neuroscience. Yet, while it may reuse some of the techniques elaborated in these fields, it differs from them from many perspectives. It differs from classical artificial intelligence because it does not assume the capability of advanced symbolic reasoning and focuses on embodied and situated sensorimotor and social skills rather than on abstract symbolic problems. It differs from cognitive robotics because it focuses on the processes that allow the formation of cognitive capabilities rather than these capabilities themselves. It differs from computational neuroscience because it focuses on functional modeling of integrated architectures of development and learning. More generally, developmental robotics is uniquely characterized by the following three features:

1. It targets task-independent architectures and learning mechanisms, i.e. the machine/robot has to be able to learn new tasks that are unknown by the engineer;
2. It emphasizes open-ended development and lifelong learning, i.e. the capacity of an organism to acquire continuously novel skills. This should not be understood as a capacity for learning "anything" or even “everything”, but just that the set of skills that is acquired can be infinitely extended at least in some (not all) directions;
3. The complexity of acquired knowledge and skills shall increase (and the increase be controlled) progressively.

Developmental robotics emerged at the crossroads of several research communities including embodied artificial intelligence, enactive and dynamical systems cognitive science, connectionism. Starting from the essential idea that learning and development happen as the self-organized result of the dynamical interactions among brains, bodies and their physical and social environment, and trying to understand how this self-organization can be harnessed to provide task-independent lifelong learning of skills of increasing complexity, developmental robotics strongly interacts with fields such as developmental psychology, developmental and cognitive neuroscience, developmental biology (embryology), evolutionary biology, and cognitive linguistics. As many of the theories coming from these sciences are verbal and/or descriptive, this implies a crucial formalization and computational modeling activity in developmental robotics. These computational models are then not only used as ways to explore how to build more versatile and adaptive machines but also as a way to evaluate their coherence and possibly explore alternative explanations for understanding biological development.

Research directions

Skill domains

Due to the general approach and methodology, developmental robotics projects typically focus on having robots develop the same types of skills as human infants. A first category that is important being investigated is the acquisition of sensorimotor skills. These include the discovery of one's own body, including its structure and dynamics such as hand-eye coordination, locomotion, and interaction with objects as well as tool use, with a particular focus on the discovery and learning of affordances. A second category of skills targeted by developmental robots are social and linguistic skills: the acquisition of simple social behavioural games such as turn-taking, coordinated interaction, lexicons, syntax and grammar, and the grounding of these linguistic skills into sensorimotor skills (sometimes referred as symbol grounding). In parallel, the acquisition of associated cognitive skills are being investigated such as the emergence of the self/non-self distinction, the development of attentional capabilities, of categorization systems and higher-level representations of affordances or social constructs, of the emergence of values, empathy, or theories of mind.

Mechanisms and constraints

The sensorimotor and social spaces in which humans and robot live are so large and complex that only a small part of potentially learnable skills can actually be explored and learnt within a life-time. Thus, mechanisms and constraints are necessary to guide developmental organisms in their development and control of the growth of complexity. There are several important families of these guiding mechanisms and constraints which are studied in developmental robotics, all inspired by human development:

1. Motivational systems, generating internal reward signals that drive exploration and learning, which can be of two main types:
   • extrinsic motivations push robots/organisms to maintain basic specific internal properties such as food and water level, physical integrity, or light (e.g. in phototropic systems);
   • intrinsic motivations push robot to search for novelty, challenge, compression or learning progress per se, thus generating what is sometimes called curiosity-driven learning and exploration, or alternatively active learning and exploration;
2. Social guidance: as humans learn a lot by interacting with their peers, developmental robotics investigates mechanisms that can allow robots to participate to human-like social interaction. By perceiving and interpreting social cues, this may allow robots both to learn from humans (through diverse means such as imitation, emulation, stimulus enhancement, demonstration, etc. ...) and to trigger natural human pedagogy. Thus, social acceptance of developmental robots is also investigated;
3. Statistical inference biases and cumulative knowledge/skill reuse: biases characterizing both representations/encodings and inference mechanisms can typically allow considerable improvement of the efficiency of learning and are thus studied. Related to this, mechanisms allowing to infer new knowledge and acquire new skills by reusing previously learnt structures is also an essential field of study;
4. The properties of embodiment, including geometry, materials, or innate motor primitives/synergies often encoded as dynamical systems, can considerably simplify the acquisition of sensorimotor or social skills, and is sometimes referred as morphological computation. The interaction of these constraints with other constraints is an important axis of investigation;
5. Maturational constraints: In human infants, both the body and the neural system grow progressively, rather than being full-fledged already at birth. This implies, for example, that new degrees of freedom, as well as increases of the volume and resolution of available sensorimotor signals, may appear as learning and development unfold. Transposing these mechanisms in developmental robots, and understanding how it may hinder or on the contrary ease the acquisition of novel complex skills is a central question in developmental robotics.

From bio-mimetic development to functional inspiration.

While most developmental robotics projects interact closely with theories of animal and human development, the degrees of similarities and inspiration between identified biological mechanisms and their counterpart in robots, as well as the abstraction levels of modeling, may vary a lot. While some projects aim at modeling precisely both the function and biological implementation (neural or morphological models), such as in Neurorobotics, some other projects only focus on functional modeling of the mechanisms and constraints described above, and might for example reuse in their architectures techniques coming from applied mathematics or engineering fields.

Open questions

As developmental robotics is a relatively new research field and at the same time very ambitious, many fundamental open challenges remain to be solved.

First of all, existing techniques are far from allowing real-world high-dimensional robots to learn an open-ended repertoire of increasingly complex skills over a life-time period. High-dimensional continuous sensorimotor spaces constitute a significant obstacle to be solved. Lifelong cumulative learning is another one. Actually, no experiments lasting more than a few days have been set up so far, which contrasts severely with the time needed by human infants to learn basic sensorimotor skills while equipped with brains and morphologies which are tremendously more powerful than existing computational mechanisms.

Among the strategies to explore to progress towards this target, the interaction between the mechanisms and constraints described in the previous section shall be investigated more systematically. Indeed, they have so far mainly been studied in isolation. For example, the interaction of intrinsically motivated learning and socially guided learning, possibly constrained by maturation, is an essential issue to be investigated.

Another important challenge is to allow robots to perceive, interpret and leverage the diversity of multimodal social cues provided by non-engineer humans during human-robot interaction. These capacities are so far, mostly too limited to allow efficient general-purpose teaching from humans.

A fundamental scientific issue to be understood and resolved, which applied equally to human development, is how compositionality, functional hierarchies, primitives, and modularity, at all levels of sensorimotor and social structures, can be formed and leveraged during development. This is deeply linked with the problem of the emergence of symbols, sometimes referred to as the "symbol grounding problem" when it comes to language acquisition. Actually, the very existence and need for symbols in the brain are actively questioned, and alternative concepts, still allowing for compositionality and functional hierarchies are being investigated.

During biological epigenesis, morphology is not fixed but rather develops in constant interaction with the development of sensorimotor and social skills. The development of morphology poses obvious practical problems with robots, but it may be a crucial mechanism that should be further explored, at least in simulation, such as in morphogenetic robotics.

Another open problem is the understanding of the relation between the key phenomena investigated by developmental robotics (e.g., hierarchical and modular sensorimotor systems, intrinsic/extrinsic/social motivations, and open-ended learning) and the underlying brain mechanisms.

Similarly, in biology, developmental mechanisms (operating at the ontogenetic time scale) interact closely with evolutionary mechanisms (operating at the phylogenetic time scale) as shown in the flourishing "evo-devo" scientific literature. However, the interaction of those mechanisms in artificial organisms, developmental robots, in particular, is still vastly understudied. The interaction of evolutionary mechanisms, unfolding morphologies and developing sensorimotor and social skills will thus be a highly stimulating topic for the future of developmental robotics.

Main journals

• IEEE Transactions on Cognitive and Developmental Systems (previously known as IEEE Transactions on Autonomous Mental Development): https://cis.ieee.org/publications/t-cognitive-and-developmental-systems

Main conferences

• International Conference on Development and Learning: http://www.cogsci.ucsd.edu/~triesch/icdl/
• Epigenetic Robotics: https://www.lucs.lu.se/epirob/
• ICDL-EpiRob: http://www.icdl-epirob.org/ (the two above joined since 2011)
• Developmental Robotics: http://cs.brynmawr.edu/DevRob05/

The NSF/DARPA funded Workshop on Development and Learning was held April 5–7, 2000 at Michigan State University. It was the first international meeting devoted to computational understanding of mental development by robots and animals. The term "by" was used since the agents are active during development.

See also

• Evolutionary developmental robotics
• Robot learning

External links

Technical committees

• IEEE Technical Committee on Cognitive and Developmental Systems (CDSTC), previously known as IEEE Technical Committee on Autonomous Mental Development, [1]
• IEEE Technical Committee on Cognitive Robotics, https://www.ieee-ras.org/cognitive-robotics
• IEEE Technical Committee on Robot Learning, https://www.ieee-ras.org/robot-learning/

Academic institutions and researchers in the field

• Lund University Cognitive Science - Robotics Group
• Cognitive Development Lab, University of Indiana, US
• Michigan State University – Embodied Intelligence Lab
• Inria and Ensta ParisTech FLOWERS team, France: Exploration, interaction and learning in developmental robotics
• University of Tokyo—Intelligent Systems and Informatics Lab
• Cognitive Robotics Lab of Juergen Schmidhuber at IDSIA and Technical University of Munich
• LIRA-Lab, University of Genova, Italy
• CITEC at University of Bielefeld, Germany
• Vision Lab, Psychology Department, Southern Illinois University Carbondale
• FIAS (J. Triesch lab.)
• LPP, CNRS (K. Oregan lab.)
• AI Lab, SoftBank Robotics Europe, France
• Departement of Computer Science, University of Aberdeen
• Asada Laboratory, Department of Adaptive Machine Systems, Graduate School of Engineering, Osaka University, Japan
• The University of Texas at Austin, UTCS Intelligent Robotics Lab
• Bryn Mawr College's Developmental Robotics Project: research projects by faculty and students at Swarthmore and Bryn Mawr Colleges, Philadelphia, PA, USA
• Jean Project: Information Sciences Institute of the University of Southern California
• Cognitive Robotics (including Hide and Seek) at the Naval Research Laboratory Archived August 8, 2010, at the Wayback Machine
• The Laboratory for Perceptual Robotics, University of Massachusetts Amherst Amherst, USA
• Centre for Robotics and Neural Systems, Plymouth University Plymouth, United Kingdom
• Laboratory of Computational Embodied Neuroscience, Institute of Cognitive Science and Technologies National Research Council, Rome, Italy
• Neurocybernetic team, ETIS Lab., ENSEA – University of Cergy-Pontoise – CNRS, France
• Machine Perception and Cognitive Robotics Lab, Florida Atlantic University, Boca Raton, Florida
• Adaptive Systems Group, Department of Computer Science, Humboldt University of Berlin, Germany
• Cognitive Developmental Robotics Lab (Nagai Lab), The University of Tokyo, Japan

Related large-scale projects

• RobotDoC Project (funded by European Commission)
• Italk Project (funded by European Commission)
• IM-CLeVeR Project (funded by European Commission)
• ERC Grant EXPLORERS Project (funded by European Research Council)
• RobotCub Project (funded by European Commission)
• Feelix Growing Project (funded by European Commission)

Courses

The first undergraduate courses in DevRob were offered at Bryn Mawr College and Swarthmore College in the Spring of 2003 by Douglas Blank and Lisa Meeden, respectively. The first graduate course in DevRob was offered at Iowa State University by Alexander Stoytchev in the Fall of 2005.