Optasia: A Relational Platform for Efficient Large-Scale Video Analytics
Yao Lu, Aakanksha Chowdhery, Srikanth Kandula
Microsoft
ACM SoCC 2016. PDF

The publicly avaliable version of Optasia is now coming via Azure Data Lake.
A tutorial can be found here.


Camera deployments are ubiquitous; however exiting methods to analyze video
feeds from many cameras do not scale and are error-prone. For example, below lists
lists code examples to extract SIFT keypoints from many video frames.

Spark script to extract SIFT keypoints
  1. import logging
  2. import io
  3. import sys
  4. import os
  5.  
  6. import cv2
  7. import numpy as np
  9. def extract_sift_features:
  10.  
  11.     def extract_sift_features_nested(imgfile_imgbytes):
  12.         try:
  13.             imgfilename, imgbytes = imgfile_imgbytes
  14.             nparr = np.fromstring(buffer(imgbytes), np.uint8)
  15.             img = cv2.imdecode(nparr, 0)
  16.             extractor = cv2.SIFT()
  17.             kp, descriptors = extractor.detectAndCompute(img, None)
  18.             return [(imgfilename, descriptors)]
  19.         except Exception, e:
  20.             logging.exception(e)
  21.             return []
  22.  
  23.     return extract_opencv_features_nested
  24.  
  25. if __name__ == "__main__":
  26.     sc = SparkContext(appName="sift_extractor")
  27.     sqlContext = SQLContext(sc)
  28.  
  29.     try:
  30.         image_seqfile_path = sys.argv[1]
  31.         feature_parquet_path = sys.argv[2]
  32.         partitions = int(sys.argv[3])
  33.     except:
  34.         print("Usage: spark-submit sift_extraction.py "
  35.         <image_input_path> <feature_output_path> <partitions>")
  36.  
  37.     images = sc.sequenceFile(image_seqfile_path, minSplits=partitions)
  38.  
  39.     features = images.flatMap(extract_sift_features)
  40.     features = features.filter(lambda x: x[1] != None)
  41.     features = features.map(lambda x: (Row(fileName=x[0], features=x[1].tolist())))
  42.     featuresSchema = sqlContext.createDataFrame(features)
  43.     featuresSchema.registerTempTable("images")
  44.     featuresSchema.write.parquet(feature_parquet_path)

To run this script, one must upload the images to Hadoop compatible storage
(Swift), specify degree of parallism (e.g., 100), and run:
       spark-submit --executor-memory 8g sift.py swift://spark.swift1/images.hseq ...
       swift://spark.swift1/images.parquet 100
The execution optimality cannot be garanteed.


We present a system, Optasia, that is friendly to end-users while efficiently executes
the scripe on the cluster. End-users do not need to worry about degree of parallism,
task scheduling, etc. The analystic units are highly modulized, decoupling the roles
of algorithm and application engineers.

Optasia script to extract SIFT keypoints
  1. USING Optasia;
  2.  
  3. images =
  4.     EXTRACT id : int,
  5.             frame : string
  6.     FROM SPARSE STREAMSET @"images/"
  7.     USING ImageExtractor();
  9. feat =
  10.     PROCESS images
  11.     USING SIFTProcessor()
  12.     PRODUCE id,
  13.             feature;
  14.  
  15. OUTPUT feat
  16. TO @"features.txt"
  17. USING DefaultTextOutputter();

A powerful query optimization technique is applied to generate optimal execution
plans so that resource useage is greatly reduced. More details described in the paper.



Click here for example dataflow and queries used in the paper.
Click here for example SCOPE wrappers in C#.