Tracking is indeed one of the challenging problems since it is highly dependent on frame-rate. Are you planning on a real-time system?
Let me suggest you some ideas to look at.
Tensorflow is available for both Python 2.7 and Python 3.x. You can install via pip install tensorflow or pip install tensorflow-gpu. You can refer this link for more details.
OpenCV is also available for both Python 2.7 and Python 3.x. You can install it via pip install opencv-python.