Searching and Tracking People with Cooperative Mobile Robots
Abstract
Social robots should be able to search and track people in order to help them. In this paper we present two different techniques of coordinated multi-robots for searching and tracking people. A probability map (belief) of a target person location is maintained, and to initialize and update it, two methods were implemented and tested: one based on a reinforcement learning algorithm and the other based on a particle filter. The person is tracked if visible, otherwise an exploration is done by making a balance, for each candidate location, between the belief, the distance, and whether close locations are already being explored by other robots of the team. The validation of the approach was accomplished throughout an extensive set of simulations using up to five agents and a large amount of dynamic obstacles; furthermore, over three hours of real-life experiments with two robots searching and tracking were recorded and analysed.
Experimental Videos
Videos from experiments with the Multi-agent HB-PF Explorer. Two mobile robots (Tibi and Dabo) cooperatively search for and track a person recognized using AR Markers. Environment and map files: maps page.
Video Legend
Each experiment video shows three sections:
- Left: Map and probability maps
- Right-top: Video focusing on Tibi
- Right-bottom: Video focusing on Dabo
Map Elements
- Dabo: Blue body, white head
- Tibi: Orange body, white head
- Obstacles: Black and dark gray
- Laser detections: Blue/orange line/dots
- Path: Blue/orange lines showing executed path
- Person detection:
- Leg detection: blue/orange dots
- Last used person location: red dot (combination of leg and AR Marker detection)
- Last detected person location: white dot
Probability Map Elements
- Robot self: Large blue circle
- Other robot: Small blue circle
- Detected person's location: Large red circle
- Other robot's detection: Small red circle
- Obstacles: Black squares
- Probability: Light blue (0) to white (low) to red (high)
Limitations
The robots detect the person in two phases: 1) by laser detection of legs, and 2) by AR tag. The AR tag alone results in some false positives, so tags are only accepted when a laser detection is close enough. When a false positive occurs, the probability temporarily increases for that location, but recovers as the false positive is typically detected only briefly.
Exploration
Exploration experiments without the person present, to evaluate cooperative area coverage.
Exploration 1
Both robots explored continuously. One stopped for a minute due to a hardware issue but recovered. Near the end both took the same route as their goals converged, but maintained sufficient spacing to cover it thoroughly.
Exploration 2
Dabo was unable to navigate the ramp autonomously — the narrow passage caused the laser to detect the inclined floor as an obstacle. It was teleoperated up the ramp.
Exploration 3
Both robots explored the area. Dabo again needed manual assistance up the ramp due to limited manoeuvring space.
Search & Track
The robots search for the person and follow once found. Observations are shared so both robots update their belief maps.
Search 1
Dabo found the person; Tibi updated her belief based on Dabo's observations and moved to the person's location.
Search 2
The robots took longer, reaching the person's location last. They crossed paths twice as shortest routes converged, and both avoided the ramp twice. When the person was found and then quickly lost, beliefs diverged. Tibi spotted the person again; communication was lost at the end so Dabo had no updated belief.
Search 3
Robots started at opposite ends. Dabo quickly found the person but then stopped due to a technical issue; Tibi continued towards the person.
Search 4
Multiple experiments with different start positions. Once the robot found the person it followed for a period of time.
Following with Static and Dynamic Obstacles
Follow
Both robots follow the person through the environment.
Search and Follow
The robots start by searching, then follow for an extended time. Only one robot's video feed is available at the start. The experiment ended due to a hardware issue with Dabo.
Follow 2
The robots start with the person visible and follow throughout. One robot stopped intermittently due to hardware issues; the other continued searching and tracking.
Follow in a Group
Three other people walk in front of the target as dynamic obstacles. The robots maintain tracking of the correct person throughout.
Experiment Statistics
Summary of over 3 hours of real-life experiments with two robots:
| Metric | Exploration | Search & Track | Tracking | Total |
|---|---|---|---|---|
| Distance per robot (km) | 1.2 | 1.2 | 0.7 | 3.2 |
| Total time (h) | 1.1 | 1.2 | 0.9 | 3.2 |
| Avg. visibility (%) | 0 | 16.3 | 36.4 | 15.3 |
| Avg. distance to person (m) | - | 8.4 ± 6.4 | 8.4 ± 5.6 | 8.3 ± 5.9 |
| Avg. time found (s) | - | 106.8 ± 138.7 | 23.5 ± 42.5 | 72.9 ± 117.6 |