ENGT5256 Robotics: Coursework

The coursework of ENGT5256 Robotics module includes a design study report and a
presentation. The assessment of your work will be calculated as a weighted sum of your
mark from the report and the presentation, as presented in the table below.

Type of Duration or Assessment weighting Deadline or

assessment volume % Date

Design Study 3000-3500 words 75% 21st of May

Report

Presentation 15min 25% 27th of May

You will select one of the problems listed below and implement a robotic solution. You will
work on the robotic simulation platform called CoppeliaSim (V-Rep). You will write Matlab
codes that will control a robot in the simulation. For each problem, you will be given an
environment model. You should not use CoppeliaSim code to control the robots. You may
not get any marks if you use CoppeliaSim to control the robots. You are free to use
CoppeliaSim code if you like to control other elements (such as conveyor belt, doors).
Your coursework will be evaluated based the following: i) robot design (choice of robot and
sensors), ii) block diagram of your robotic solution, iii) performance of your solution. You
should evaluate the performance of your solution based on two criteria: i) accuracy, and ii)
speed.
You can work on a problem as an individual or as a group of 2-3. The work of each individual
in the group should be explicitly described both in the report and in the presentation.

Please send me the following information via email by 25th of March

1. The problem you selected
2. The name of the group members, if you are working in a group
 List of Problems

1. Domestic assistive robot that is integrated with IoT home sensors

A domestic assistive robot is a type of service robots that is designed to help people with
their daily life activities at home [1, 2, 3, 4]. To increase its capabilities, an assistive robot
should be integrated with smart home sensors (e.g., window contact sensor) [1, 4].

In this problem, you are going to design and implement a service robot that can quickly
respond to an anomaly detected by smart home sensors. In particular, you will consider the
following anomalies: i) bedroom light is on during daytime for more than 10mins, ii) motion
detected in the living room during the night. To simulate the smart home sensors, you are
going to create a Matlab web app where you can change the status of virtual sensors.

TIPS:

• First start by writing a Matlab code that makes the robot in simulation to navigate to
a specific position on the map
• Check the details given in last section of this document about using CoppeliaSim
• Choose the sensors you think is required for mapping and create a map of the
environment by using SLAM
• Write the code for a Matlab web app that changes the status of (virtual) sensors in
particular rooms
• Check this tutorial on how to write Matlab web apps
(https://uk.mathworks.com/videos/how-to-build-a-web-site-powered-by-matlab-
106529.html)
• Write the code that checks the status of sensors and make the robot to navigate to
the room of the sensor with abnormal state

[1] Coşar, S., Fernandez-Carmona, M., Agrigoroaie, R. et al. ENRICHME: Perception and
Interaction of an Assistive Robot for the Elderly at Home. Int J of Soc Robotics (2020).
https://doi.org/10.1007/s12369-019-00614-y

[2] David Fischinger, Peter Einramhof, Konstantinos Papoutsakis, Walter Wohlkinger, Peter
Mayer, Paul Panek, Stefan Hofmann, Tobias Koertner, Astrid Weiss, Antonis Argyros, Markus
Vincze, Hobbit, a care robot supporting independent living at home: First prototype and
lessons learned, Robotics and Autonomous Systems, Volume 75, Part A, 2016, Pages 60-78,
ISSN 0921-8890, https://doi.org/10.1016/j.robot.2014.09.029.

[3] CARL Assistive Robot, https://www.youtube.com/watch?v=gKcZsNQadGU

[4] ENRICHME Assistive Robot, https://www.youtube.com/watch?v=geMQf9oSoTU

 2. Unmanned search & rescue robot that finds people in a disaster site

In the event of large crises, such as earthquakes, typhoons, the primary task of the fire and
rescue services is to search for human survivors on the incident site. This is a complex and
dangerous task, which may lead to loss of lives among the human crisis managers
themselves. The introduction of unmanned search and rescue (SAR) devices offers a
valuable tool to save human lives and to speed up the search and rescue process [1, 2].

In this problem, you are going to design and implement an unmanned ground SAR robot
that looks for victims in the disaster site. The robot should explore the environment and
look for possible victims in the disaster site.

TIPS:

• First start by writing a Matlab code that makes the robot in simulation to navigate to
a specific position on the map
• Check the details given in last section of this document about using CoppeliaSim
• Create a map of the environment by using SLAM
• Choose sensors that you think are required to detect people
• Write the code to detect people using the sensor data
• Write the code that will make robot to simultaneously explore the environment and
detect people if they are present in the scene

[1] Geert De Cubber, Daniela Doroftei, Konrad Rudin, Karsten Berns, Anibal Matos, Daniel
Serrano, Jose Sanchez, Shashank Govindaraj, Janusz Bedkowski, Rui Roda, Eduardo Silva and
Stephane Ourevitch (August 23rd 2017). Introduction to the Use of Robotic Tools for Search
and Rescue, Search and Rescue Robotics - From Theory to Practice, IntechOpen, DOI:
10.5772/intechopen.69489. Available from: https://www.intechopen.com/books/search-
and-rescue-robotics-from-theory-to-practice/introduction-to-the-use-of-robotic-tools-for-
search-and-rescue

[2] Search and Rescue Robots, https://www.youtube.com/watch?v=1H-QpCwne5Y

 3. A robotic manipulator that sorts out objects by colour and shape

Robotic arms have gained a wide range of use in the industrial applications. It could replace
human labour in trivial tasks that are repetitive or requiring a lot of lift force [1, 2, 3, 4].
Object manipulation is one of the common uses of robotic arms. It is essential for an object
manipulator to understand the type of object. To achieve this several sensors such as
cameras are used to detect the shape and colour of objects [1, 3, 4].
In this problem, you will design and implement a Pick and Place robot that can sort out
object by colour and shape. There will be different types of objects coming on top of a
conveyor belt. Your robot should select blue round and red cuboid objects, put them in the
first and second tables, respectively.
TIPS:

• First start by writing a Matlab code that makes the manipulator in simulation to
reach to a specific position (You should not use CoppeliaSim code to control the
manipulator)
• Check the details given in last section of this document about using CoppeliaSim
• Write the code for gripping and placing an object
• Choose sensors that you think are required to detect object by colour and shape
• Write the code to detect objects using the sensor data
• Write the code that will make robot to pick up the required objects and place them
on the correct tables
• You can check CoppeliaSim code to stop conveyor belt

[1] Martinez-Martin, E.; del Pobil, A.P. Vision for Robust Robot Manipulation. Sensors 2019,
19, 1648. https://www.mdpi.com/441142
[2] Md. Hazrat Ali, Aizat K., Yerkhan K., Zhandos T., Anuar O., Vision-based Robot
Manipulator for Industrial Applications, Procedia Computer Science, Volume 133, 2018,
Pages 205-212, ISSN 1877-0509, https://doi.org/10.1016/j.procs.2018.07.025.
[3] ABB Robotic Manipulator, https://www.youtube.com/watch?v=aPTd8XDZOEk
[4] Universal Robot, https://www.youtube.com/watch?v=3q1a3Rp3H7M
 4. A fleet of agri-robots that helps fruit pickers

The introduction of robots into agricultural domain is expected to yield productivity gains as
well as improving produce quality. As the agricultural domain is generally very labour
intensive, introducing automation in relatively small doses can have significant positive
impacts [1]. Soft fruit production involves many operations such as planting, plant care, fruit
picking and transportation of the crop to storage facilities. In a typical strawberry
production, strawberry plants are grown on table top trays in polytunnels [2]. It is a very
labour intensive and time-consuming process to pick strawberries in large farms.
Agricultural robots (agri-robots) provide an efficient solution that can have positive impact
on workers’ health and productivity [3].
In this problem, you will design and implement a fleet of agri-robots that performs in-field
logistics operations and autonomous transportation providing support for soft-fruit pickers.
You will coordinate the movement of 2 robots that will go into a polytunnel, which has 5
lanes, to help the pickers to transport strawberries to a food collection unit (FCU). In the
beginning, the robots will be in the FCU. The robots will go to lanes 1, 3, 4, 2 and 5 in this
order. Assuming a picker is waiting at the end of the lane, robot will go inside a lane and
reach to the end of it. The simulation will not include pickers. Thus, the robot will wait for 2
mins supposing a picker is loading the strawberries. Once the loading is done, the robot will
head back to FCU to unload the strawberries. The unloading process will take 4 mins.
You will coordinate which robot should go to which lane by considering the time required
for reaching the target lane, loading time and unloading time. You will ensure that the
robots collect the strawberries in all lanes in the order above as quick as possible.
TIPS:

• First start by writing a Matlab code that makes the robot in simulation to navigate to
a specific position on the map
• Check the details given in last section of this document about using CoppeliaSim
• Choose the sensors you think is required for mapping and create a map of the
environment by using SLAM
• Check the time a robot takes to go to each lane.
• Write the code that will coordinate 2 robots to reach all lanes in given order

[1] Das, Gautham, Cielniak, Grzegorz, From, Pal and Hanheide, Marc (2018) Discrete Event
Simulations for Scalability Analysis of Robotic In-Field Logistics in Agriculture – A Case Study.
In: IEEE International Conference on Robotics and Automation, Workshop on Robotic Vision
and Action in Agriculture, 21-25 May 2018, Brisbane.
[2] A Table Top Strawberry Production, https://www.youtube.com/watch?v=q3XUXH76YBE
[3] Thorvald II Agricultural Robot, https://www.youtube.com/watch?v=bUI-h7dhN9w
 Design Study Report
You will write a report that describes the robotic solution you implemented. Please use the
IEEE conference paper format1. Your report should roughly consist of five sections. It should
start with an “Introduction” section where you briefly explain the problem you target. Then,
in “Related Work” section, you should briefly talk about existing working solutions in
academia and industry (you can use the references given at the bottom of each problem).
The third section (“Proposed Approach”) should explain the details of your robotic solution.
Then, in “Experimental Results” section you should describe the results you obtained in your
simulation. In this section, you should evaluate your solution based on two criteria: i)
accuracy, and ii) speed. The last section (“Conclusion”) should briefly summarise your
approach and experimental results. You can also briefly talk about future directions; which
parts of your approach can be extended and how can it be improved.

Presentation
After you submitted your report, you are going to present the work you have done. You are
going to prepare a 15min. presentation. You are free to choose the structure of your
presentation. Similar to your report, your presentation can have five parts (Introduction,
Related Work, Proposed Approach, Experimental Results, and Conclusion). It is important
that your presentation should include a live demonstration of your robotic solution.

How you should approach to your problem

1. Read the description of all the problems listed. You can also check the references to
see some examples of existing work in this field. Then, choose the problems that you
would like to work on
2. Select a robot from CoppeliaSim that is appropriate for your task
3. Select sensors from CoppeliaSim that is required for your solution and mount them
on your robot or in the environment
4. Plan the robotic solution that will achieve the required task. Build a block diagram of
your robotic solution
5. Work on the coding of each block. It is a good practice to have one (Matlab) function
per block. Each function should get the input from the previous block and create an
output that will be fed to the next block. This will allow your robotic solution to have
a modular structure. In case of errors or bugs, it will help you to easily understand
which part of your code is wrong.
6. Write your main code that will bring together all the codes you have written before.

1
https://www.ieee.org/content/dam/ieee-org/ieee/web/org/conferences/conference-template-
a4.docx
 CoppeliaSim (V-Rep)
CoppeliaSim is an open-source free robotic simulation software. You are going to use the
CoppeliaSim Edu version. You can download and install the educational version from here:
https://coppeliarobotics.com/downloads
After you install CoppeliaSim, you need to establish its connection with Matlab. Follow the
instructions in these two videos:
https://www.youtube.com/watch?v=piI5wYEXUms
https://www.youtube.com/watch?v=mal48Vd-DQY&t=151s
Check the following tutorial video about the basic operations you can do in CoppeliaSim
simulation.
https://www.youtube.com/watch?v=PwGY8PxQOXY
In CoppeliaSim, you can create various types of robots by designing parts (e.g., wheels) and
configuring their mechanism. CoppeliaSim also includes the model of many well-known
robots used in research and industry. In this coursework, you are going to use existing robot
models. You will focus on designing the whole robotic solution for the problem you have
chosen. In CoppeliaSim, you will also find the model of real sensors that you have seen in
the lecture. According to your robotic solution, you will also use these existing sensor
models and mount them on your robot or in the environment.
In order to attach a sensor to you robot, first you need to move the sensor where you want
to attach on the robot. You will use Position Dialog to move the sensor. Check the following
help file to understand how you can move objects:
https://www.coppeliarobotics.com/helpFiles/en/positionDialog.htm
After you move the sensor you prefer, you will do one of the following:

1) Select your sensor in “Scene hierarchy” window, then

select your robot while pressing control on the keyboard.
Then, click [Menu bar --> Edit --> Make last selected object
parent]
2) In “Scene hierarchy” window, drag your sensor onto your
robot.
This will attach the sensor to the body of the robot. Now, when your robot moves, it will
move together with the sensor attached to it.
CoppeliaSim also allows you to code in its own language called LUA. It is a programming
language that is very similar to Python. The codes in LUA allows you to control basic
operations of robots and sensors. For instance, you can turn the wheels of the robot, or you
can get sensor readings from a laser scanner. To accomplish the task in the problems, you
are going to use Matlab codes to perform mapping and navigation of robots.
Assume you have selected Pioneer P3DX robot and a Velodyne 3D Lidar. You have attached
the 3D lidar on top of your robot (see Figure 1). In Figure 2, you will find an example of code
blocks that will operate the robot do perform mapping. CoppeliaSim and Matlab
components are shown as orange and blue boxes, respectively. You can see that the basic
components of the robot (wheels and 3D lidar) are controlled by CoppeliaSim. Higher-level
functionalities, such as mapping, are performed by Matlab.
You will follow a similar approach to complete the task required in the problem you have
selected. You will use CoppeliaSim to provide you sensor reading. Then, you will implement
advanced functionalities, such as navigation, mapping, person detection, using Matlab.
Finally, you will send motion commands from Matlab to CoppeliaSim to move your robot.
For further details about the CoppeliaSim, check the help files:
https://www.coppeliarobotics.com/helpFiles/index.html

Figure 1: A Pioneer P3DX robot with a Velodyne 3D Lidar in CoppeliaSim simulation

3D Lidar SLAM Map

Wheels Motion
Commands

Figure 2: Block diagram of mapping task performed using CoppeliaSim and Matlab. Orange and blue
boxes represent CoppeliaSim and Matlab components, respectively.