Tensorflow Object Tracking

It seems Tensorflow evolves pretty fast. Today we are testing object tracking based on Tensorflow.

1. Environment

➜  ~ python
Python 3.6.6 (default, Sep 12 2018, 18:26:19) 
[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> tf.__version__
'1.12.0-rc0'
>>> import cv2
>>> cv2.__version__
'3.4.3'

2. Object Tracking

2.1 Concepts

There are several fundamental concepts to be re-emphasized (Here, we took one single concerned object as our example. There might be multiple concerned objects):

• detection: You don’t know whethere there is a concerned object in the field of view or not, which you will know after the detection. And, if there is such a concerned object in the view, the object location is to be given.
• tracking: You know where the concerned object was. Based on the prior knowledge, you are to determine where this object is going to be next?
• location: Both detection and tracking are looked on as locating the concerned object.
• recognition: Only after the concerned object has been located, more detailed information may be recognized afterwards.

2.2 Testing

Now, let’s test out Deep Object Tracking Implementation in Tensorflow.

➜  tf-adnet-tracking git:(master) ✗ python runner.py by_dataset  --vid-path=./data/freeman1/
VIDEOIO ERROR: V4L: can't open camera by index 0
Test:  0.0
Ratio:  0.0
Frame Rate:  0.0
Width:  0.0
Height:  0.0
Brightness:  0.0
Contrast:  0.0
Saturation:  0.0
Hue:  0.0
Gain:  0.0
Exposure:  0.0
2018-11-04 19:20:54.997017: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:993] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-11-04 19:20:54.997514: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found device 0 with properties: 
name: GeForce GTX 980M major: 5 minor: 2 memoryClockRate(GHz): 1.1265
pciBusID: 0000:01:00.0
totalMemory: 3.94GiB freeMemory: 3.14GiB
2018-11-04 19:20:54.997565: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2018-11-04 19:20:55.222053: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-11-04 19:20:55.222092: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988]      0 
2018-11-04 19:20:55.222100: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0:   N 
2018-11-04 19:20:55.222278: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2854 MB memory) -> physical GPU (device: 0, name: GeForce GTX 980M, pci bus id: 0000:01:00.0, compute capability: 5.2)
[2018-11-04 19:20:55,738] [networks] [INFO] all global variables initialized
[2018-11-04 19:20:55,865] [networks] [INFO] conv1/weights:0 : original weights assigned. [0]=[[[-0.08093996 -0.03
[2018-11-04 19:20:55,877] [networks] [INFO] conv1/biases:0 : original weights assigned. [0]=-1.5706328
[2018-11-04 19:20:55,922] [networks] [INFO] conv2/weights:0 : original weights assigned. [0]=[[[-0.00856966 -0.00
[2018-11-04 19:20:55,936] [networks] [INFO] conv2/biases:0 : original weights assigned. [0]=-0.06253582
[2018-11-04 19:20:56,016] [networks] [INFO] conv3/weights:0 : original weights assigned. [0]=[[[ 0.00339902  0.00
[2018-11-04 19:20:56,033] [networks] [INFO] conv3/biases:0 : original weights assigned. [0]=-0.071603894
[2018-11-04 19:20:56,236] [networks] [INFO] fc4/weights:0 : original weights assigned. [0]=[[[ 2.6850728e-03  2
[2018-11-04 19:20:56,254] [networks] [INFO] fc4/biases:0 : original weights assigned. [0]=0.10009623
[2018-11-04 19:20:56,286] [networks] [INFO] fc5/weights:0 : original weights assigned. [0]=[[[-0.01758034  0.01
[2018-11-04 19:20:56,310] [networks] [INFO] fc5/biases:0 : original weights assigned. [0]=0.13894661
[2018-11-04 19:20:56,332] [networks] [INFO] fc6_1/weights:0 : original weights assigned. [0]=[[[ 0.001359    0.01
[2018-11-04 19:20:56,358] [networks] [INFO] fc6_1/biases:0 : original weights assigned. [0]=0.01
[2018-11-04 19:20:56,379] [networks] [INFO] fc6_2/weights:0 : original weights assigned. [0]=[[[-1.23225665e-02  
[2018-11-04 19:20:56,400] [networks] [INFO] fc6_2/biases:0 : original weights assigned. [0]=0.0
[]
[2018-11-04 19:20:56,482] [ADNetRunner] [INFO] ---- start dataset l=326
[2018-11-04 19:20:58,451] [ADNetRunner] [INFO] ADNetRunner.initial_finetune t=1541388056.484
[2018-11-04 19:20:59,194] [ADNetRunner] [DEBUG] redetection success=True
[2018-11-04 19:20:59,487] [ADNetRunner] [DEBUG] finetuned
[2018-11-04 19:21:01,750] [ADNetRunner] [DEBUG] finetuned
[2018-11-04 19:21:03,691] [ADNetRunner] [DEBUG] redetection success=False
[2018-11-04 19:21:03,975] [ADNetRunner] [DEBUG] finetuned
[2018-11-04 19:21:04,878] [ADNetRunner] [DEBUG] finetuned
[2018-11-04 19:21:07,524] [ADNetRunner] [DEBUG] finetuned
[2018-11-04 19:21:08,925] [ADNetRunner] [DEBUG] redetection success=True
[2018-11-04 19:21:09,249] [ADNetRunner] [DEBUG] finetuned
[2018-11-04 19:21:10,825] [ADNetRunner] [DEBUG] finetuned
[2018-11-04 19:21:11,768] [ADNetRunner] [DEBUG] redetection success=True
[2018-11-04 19:21:12,096] [ADNetRunner] [DEBUG] finetuned
[2018-11-04 19:21:14,021] [ADNetRunner] [DEBUG] finetuned
[2018-11-04 19:21:16,867] [ADNetRunner] [DEBUG] finetuned
[2018-11-04 19:21:19,710] [ADNetRunner] [DEBUG] finetuned
[2018-11-04 19:21:22,677] [ADNetRunner] [DEBUG] finetuned
[2018-11-04 19:21:25,711] [ADNetRunner] [DEBUG] finetuned
[2018-11-04 19:21:26,202] [ADNetRunner] [DEBUG] redetection success=True
[2018-11-04 19:21:26,567] [ADNetRunner] [DEBUG] finetuned
[2018-11-04 19:21:26,837] [ADNetRunner] [DEBUG] redetection success=True
[2018-11-04 19:21:27,152] [ADNetRunner] [DEBUG] finetuned
[2018-11-04 19:21:29,387] [ADNetRunner] [DEBUG] finetuned
[2018-11-04 19:21:31,371] [ADNetRunner] [INFO] ----
[2018-11-04 19:21:31,371] [ADNetRunner] [INFO] total: 34.8886
initial_finetune: 1.9675
tracking.do_action: 5.4896
tracking.save_samples.roi: 0.5116
tracking.save_samples.feat: 17.9738
tracking.online_finetune: 6.2583
[2018-11-04 19:21:31,371] [ADNetRunner] [INFO] 9.344 FPS