Human Tracking

Contract under Zenith Robotics

July - Aug 2018

The objective of this project was to track a specific target human using a ZED depth camera, running on a Jetson TX2. The output of the algorithm is the centroid and depth of the target, for use later in a following algorithm. The project was written in Python, using Keras with a Tensorflow backend, OpenCV for the computer vision, and ROS for data flow.

​

Though pedestrian tracking is a solved problem, this project differed slightly because it needed to track a specific target and handle occlusions, disappearing/reappearing, and general pose changes. 

​

The algorithm has two main components, the person detection, and the target classification. Without reinventing the wheel, I used YOLO for the person detection, which returned bounding boxes for all pedestrians in the frame. Then another Conv Net classifies each person in the frame as Target or Not. 

​

Precalculated weights for YOLO are used since pedestrian tracking is a common application. However the other Conv Net must be trained at runtime based on the desired target. Therefore an initialization step is used to capture training data of the Target, and Not-the-Target. 

​

Ultimately, this algorithm works rather reliably in distinguishing the target from other humans. But as always, there is room for improvement. Currently, the algorithm is not using the depth data in the classification. It is also memoryless, meaning it doesn't use any temporal data. 

​

One potential improvement could incorporate a Kalman Filter for target estimation. This would incorporate the temporal data and use the depth data. Another improvement could use online learning for the Conv Net so it can update and adapt to new environments. A widely different approach could possibly use an LSTM.

​

Link to Github repo: https://github.com/austinchun21/ZED-Tracking