Data-centric platform for Computer Vision

Data-centric platform for Computer Vision

Deep Video Analytics aims to revolutionize visual data analysis by providing a comprehensive platform for storage, analysis & sharing.

Deep Video Analytics is no longer being maintained / developed.

Process videos, livestreams or just set of images

  • Upload videos, image datasets. Import form S3, GS or list of urls.

  • Support for ingesting from Youtube using youtube-dl.

  • Support for ingesting from live streaming websites using Streamlink.

  • Support of importing standard Vision datasets such as MSCOCO along with annotations.

  • All imported videos, livestreams and image datasets are stored in standardized manner to enables seamless tracking of annotations, transformation and analyses.

  • Videos, images along with generated metadata, annotation and index vectors can be exported as a zip file.

Pre-trained detection, OCR, recognition and indexing models

  • All pre-trained models can be used out of box, with zero configuration.

  • Incetion V3 for indexing and visual search.

  • MTCNN Face detector.

  • Facenet face recognition/retrieval support.

  • MSCOCO trained object detector using TF object detection API.

  • OCR Pipline support via TF CTPN textbox detector and CRNN text recongizer.

  • Open Images multi label classification model for image tagging.

  • Support for additional models VGG, AVA, Open Images object detector, and any model compatible with TF object detection API though Visual Data Network.

Visual Search support using FAISS

  • Complete Visual Search stack, with retrievers that continuously refresh.

  • Support for a variety of pre-transforms & approximate nearest neighbors algorithms via FAISS and LOPQ.

  • Setup analysis/detection -> indexing piplelines.

  • Store FAISS indexes for frames, regions, clips, tubes etc.

  • Seamlessly compare performance of difference approximation

  • Enable creation of HyperVisualRelations between regions/tubes.

  • Export, combines and reuse indexes in any other application.

Map Reduce on video segments, sequences of frames or set of images

  • Deep Video Analytics allows segmentation (GOP segments) aware processing of videos.

  • Enables creation of Map Reduce style processing workflows.

  • Tasks in DAG are monitored for completition and automatically retried.

  • Setup DAG of operations to on multiple videos, datasets to support variety of objectives such as exporting results, creating training sets/training models, performing clustering of detected objects (e.g. faces) etc.

Robust language, data and execution model

  • DVAPQL Deep Video Analytics Processing and Query Language: All operations performed by DVA are represented as Directed Acyclic Graph of tasks specified in JSON.

  • Metadata is stoed in Postgres database, RabbitMQ is used with Celery for queues. To speed up processing Redis is used as in-memory cache and as data structre server.

  • The execution model provides guarantees about consistency and enables retries. All operations are immutable and associated with TEvent objects that are addressed using UUIDs. This allows partial import/exports and trakcing across different DVA instances.

REST API, Testing & Performance tracking

  • REST API to simplify development of new front-ends and applications

  • Continuous Integration testing via Travis CI. Automated performance tracking using Google Cloud Builder.

  • Entire project is delivered in two (non-GPU & GPU) Docker container images with all dependencies.

  • Development mode with Django Debug Toolbar for an easy development exprience.

Share DVAPQL scripts using Visual Data Network

Visual Data Network Github repo

  • Visual Data Network enables easy sharing of datasets, annotations, models and scripts.

  • Import models such as VGG, AVA and Open Images object detector.

  • Pull and process datasets from YFCC 100M and MSCOCO.

  • Apply DVAPQL templates to an existing video for doing framewise OCR.

  • Train FAISS approximate indexing model using LFW data for face recognition.

Single instance deployment with docker compose

Deep Video analytics is implemented using Docker and works with latest version of Docker & docker-compose installed.

cpu mode Docker will use stable branch and launch all workers on a single container.

gpu mode Docker will create seperate container per worker and use a shared volume for storing data. You can tweak worker-GPU placement and fraction of GPU memory allocated in config.json after running "./dvactl configure". This mode requires latest version of nvidia-docker2 configured as default docker runtime.

dev mode: Docker will map the local directory to container directory for interactive development, further Djagno is set to DEBUG model enabled and django_debug_toolbar is installed and automatically configured.

git clone
cd DeepVideoAnalytics
./dvactl configure
Please select mode { dev, cpu, gpu, kube } >>cpu
# If gpu option is selected, you can choose number of GPUs
./dvactl start
# Wait few minutes for container images to be downloaded
# You should be able to use both Web UI & jupyter
# To stop containers but retain volumes run
./dvactl stop
# To stop and delete containers and volumes run
./dvactl clean

Scalable deployment with Kubernetes

Deep Video analytics can be deployed on Kubernetes.

The kubernetes deployment enables seamless scaling up/down cluster to leverage pre-emptible and GPU instances.

  • Kubernetes deployment is tested on GKE.

  • All resources are launched in a seperate namespace to enable easy cleanup.

  • Three stateful sets are used to run a Postgres DB, RabbitMQ and a Redis instance, with all three exposed as an internal service via cluster IP.

  • Webserver are exposed to external traffic via a load balancer.

  • Model specific and global workers can be dynamically scaled or run on pre-emptible nodes.

git clone
cd DeepVideoAnalytics
./dvactl configure
Please select mode { dev, cpu, gpu, kube } >>kube
# Creates a Kuberentes cluster
./dvactl create
# Following command applies deployments and services usign kubectl
./dvactl start
# Wait few minutes for container images to be downloaded
# You should be able to use both Web UI & jupyter
# To stop containers but retain volumes run
./dvactl stop
# To stop and delete containers and volumes run
./dvactl clean


For a quick overview of design choices and vision behind this project we strongly recommend going through following presentation.

Issues, Questions & Contact

Please submit all software related bugs and questions using Github issues, for other questions you can contact me at

© 2017-2020 Akshay Bhat.
All rights reserved.