Upload videos or set of images. Download Youtube urls automatically. Browse & annotate uploaded videos. Ability to import pre-indexed datasets.
Perform scene detection, frame extraction on videos. Annotate frames, detections with bounding boxes, labels and metadata.
Extracted objects, along with entire frames and crops, are indexed using deep features. Feature vectors are used for visual search retrieval.
Deploy on variety of machines with/without GPUs, local & cloud. Docker compose enables automated setup of Postgres & RabbitMQ.
Visual Search as a primary interface
Upload videos, image datasets.
Ingest from various sources such as AWS S3, Youtube.
Pre-trained recognition, detection & OCR models.
Train custom detector models
User Interface for visualization, annotation & monitoring.
REST API to simplify development of new front-ends applications.
Deep Video Analytics Processing and Query Language for specifying tasks
Videos, frames, indexes, etc. stored in media directory, served through nginx.
Perform full-text search on text metadata and names.
Configure by specifying environment variables.
Manage GPU memory/utilization by dynamically launching & shutting down workers.
Indexing using Google inception V3 trained on Imagenet
Multiple object detectors from TF object detection API
Open Images multi-label inception v3 for text tags
Deep OCR using CTPN & CRNN
Labeled Faces in the Wild
Fine-tune YOLO v2 detector using custom of set of regions.
Start using trained detector instantly by launching workers that process queue assigned to the new custom detector.
For a quick overview of design choices and vision behind this project we strongly recommend going through following presentation. Also stored as readme.pdf inside the repo.
DVAPQL enables processing and querying of visual data in a consistent manner using Deep Video Analytics. All functionality of Deep Video Analytics can be expressed in form of DVAPQL scripts which are then launched on distributed workers. The data model underlying DVA makes it simple to reason about state of the system.
Deep Video analytics is implemented using Docker and works on Mac, Windows and Linux. Make sure you have latest version of Docker installed.
git clone https://github.com/AKSHAYUBHAT/DeepVideoAnalytics cd DeepVideoAnalytics/docker && docker-compose up # Above command will automatically pull container images from docker-hub
You need to have latest version of Docker and nvidia-docker installed. The GPU Dockerfile is slightly different from the CPU version dockerfile.
pip install --upgrade nvidia-docker-compose git clone https://github.com/AKSHAYUBHAT/DeepVideoAnalytics cd DeepVideoAnalytics/docker nvidia-docker-compose -f custom/docker-compose-gpu.yml up # Above command will automatically pull container images from docker-hub
Schroff, Florian, Dmitry Kalenichenko, and James Philbin. "Facenet: A unified embedding for face recognition and clustering." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015.
Szegedy, Christian, et al. "Going deeper with convolutions." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015.
Zhang, Kaipeng, et al. "Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks." IEEE Signal Processing Letters 23.10 (2016): 1499-1503.
Liu, Wei, et al. "SSD: Single shot multibox detector." European Conference on Computer Vision. Springer International Publishing, 2016.
Redmon, Joseph, et al. "You only look once: Unified, real-time object detection." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.
Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional neural networks." Advances in neural information processing systems. 2012.
Johnson, Jeff, Matthijs Douze, and Hervé Jégou. "Billion-scale similarity search with GPUs." arXiv preprint arXiv:1702.08734 (2017).