Tracking using a camera

Here’s Kieferbot being tracked through a camera:

It took a few iterations but I settled on matching the colour of the floor to create a floor mask, then findCountours to find the contours as a tree, and then searching the children of the root contours. As the bot is black, the children were matched against a ‘black pixels’ mask, and the contour with the most black pixels picked.

The camera is an Android phone running IP Webcam and sharing the back camera as MJPEG over HTTP over Wifi at 10 FPS. It could be higher, but the laptop doing the image processing has trouble keeping up with 30 FPS. The latency looks good - visually less than 200 ms.

The tracking seems reliable. If the bot goes off the bottom of the frame then other black areas are picked, but this can be detected by checking for a jump in position.

Next step is to integrate the camera position, phone sensors, and wheel velocity to get a controllable position.