Data-centric platform for Computer Vision

Data-centric platform for Computer Vision

Deep Video Analytics aims to revolutionize visual data analysis by providing a comprehensive platform for storage, analysis & sharing.

Upload

Upload videos or set of images. Download Youtube urls automatically. Browse & annotate uploaded videos. Ability to import pre-indexed datasets.

Process

Perform frame extraction, object recognition on videos. Annotate frames, detections with bounding boxes, labels and metadata.

Search

Extracted objects, along with entire frames and crops, are indexed using deep features. Feature vectors are used for visual search retrieval.

Deploy

Deploy on variety of machines with/without GPUs, local & cloud. Docker + Kubernetes enables scalable deployment across clouds.

We take significant efforts to ensure that following models (code+weights included) work without having to write any code.

Features

  • Visual Search as a primary interface

  • Upload videos, image datasets.

  • Ingest from various sources such as AWS S3, Youtube.

  • Pre-trained recognition, detection & OCR models.

  • Train custom detector models

  • User Interface for visualization, annotation & monitoring

  • REST API to simplify development of new front-ends applications

  • Deep Video Analytics Processing and Query Language for specifying tasks

  • Videos, frames, indexes, etc. stored in media directory, served through nginx

  • Perform full-text search on text metadata and names

  • Configure by specifying environment variables

  • Manage GPU memory/utilization by dynamically managing workers

Models

  • Indexing using Google inception V3 trained on Imagenet

  • Multiple object detectors from TF object detection API

  • Face detection/alignment/recognition using MTCNN and Facenet

  • Open Images multi-label inception v3 for text tags

  • Deep OCR using CTPN & CRNN

Import external datasets using VDN

  • MSCOCO

  • Labeled Faces in the Wild

Train new models

  • Fine-tune YOLO v2 detector using custom set of regions

  • Start using trained detector instantly by launching workers that process queue assigned to the detector.

Visual Data Network

Visual Data Network enables easy sharing of datasets, annotations, models and scripts.

visit VDN Github repo

Single instance for dev, test and fixed-scale use

Deep Video analytics is implemented using Docker and works with latest version of Docker & docker-compose installed. Deployments with GPU require latest nvidia-docker.

cpu mode

Docker will use stable branch and launch all workers on a single container.

gpu mode

Provide number of GPU's. Docker will create seperate container per worker and use a shared volume for storing data. You might need to tweak worker-GPU placement and fraction of GPU memory allocated.

dev mode

Docker will map the local directory to container directory for interactive development.

test mode

Docker will create seperate container per worker and use S3 for storing data.

git clone https://github.com/AKSHAYUBHAT/DeepVideoAnalytics
cd DeepVideoAnalytics
./dvactl configure
Please select mode { dev, test, cpu, gpu, kube } >>cpu
# If gpu option is selected, you can choose number of GPUs
./dvactl start
# Wait few minutes for container images to be downloaded
# You should be able to use both Web UI & jupyter
# To stop containers but retain volumes run
./dvactl stop
# To stop and delete containers and volumes run
./dvactl clean

Scalable deployment with Kubernetes for production

Deep Video analytics can be deployed on Kuberentes.

The kubernetes deployment enables seamless scaling up/down cluster to leverage pre-emptible and GPU instances.

YAML files used in deployment
git clone https://github.com/AKSHAYUBHAT/DeepVideoAnalytics
cd DeepVideoAnalytics
./dvactl configure
Please select mode { dev, test, cpu, gpu, kube } >>kube
# Creates a Kuberentes cluster
./dvactl creates
# Following command applies deployments and services usign kubectl
./dvactl start
# Wait few minutes for container images to be downloaded
# You should be able to use both Web UI & jupyter
# To stop containers but retain volumes run
./dvactl stop
# To stop and delete containers and volumes run
./dvactl clean

Presentation

For a quick overview of design choices and vision behind this project we strongly recommend going through following presentation.

Deep Video Analytics uses DVAPQL for processing and querying visual data in a consistent manner. DVAPQL specification & examples can be found here

Paper & Citation

The anatomy of large-scale visual data processing engine

Issues, Questions & Contact

Please submit all software related bugs and questions using Github issues, for other questions you can contact me at akshayubhat@gmail.com.

© 2017 Akshay Bhat, Cornell University.
All rights reserved.