DeveloperWeek Global 2020 DeveloperWeek Global 2020

Tuesday, June 16, 2020

- PDT
Real-Time Analytics on Computer Vision Data
Join on Hopin
Dhruba Borthakur
Dhruba Borthakur
Rockset, CTO
Tushar Dadlani
Tushar Dadlani
Standard Cognition, Computer Vision Engineering Manager

Walk into a store, grab the items you want, and walk out without having to interact with a cashier or even use a self-checkout system. That’s the no-hassle shopping experience showcasing the AI-powered checkout pioneered by Standard Cognition. The company makes use of Computer Vision to remove the need for checkout lines of any sort in physical retail locations. Streaming Vision-Data is converted into a stream of metadata and the need of the hour is to be able to do continuous analytics on this metadata. The variety and velocity of this metadata stream is very high and requires special purpose analytics tools that are geared for real-time analytics. Application developers need to be able to prototype rapidly on this metadata so that they can try out different analytical models quickly.

This talk describes how Standard Cognition uses Rockset for rapid prototyping of application models on vision data. In specific, we first discuss multiple challenges associated with analysis of vision data, why a traditional database was insufficient for our needs and why we chose a realtime database to address the following challenges:

* Describe the three flavors of velocity of vision data and the policies of keeping high frequency (~ 500 Hz) data on the store premises, immediately processing low-frequency (~ 5Hz) data in the cloud and streaming medium frequency (~ 50 Hz) data for realtime analytics.
* Describe why and how the schema of the generated metadata changes from day to day, which means that analytical tools we use need to be able to handle very frequent schema changes. These changes are typically the addition of new columns, columns with mixed types, complex objects inside a column, etc.
* Describe how we created an application-developer platform REST-api by encapsulating complex analytical SQL queries within Query Lambdas. This allows our application developer to rapidly iterate and build data powered applications on production data sets.

We share with you the workflow we have created for analytical processing of vision data, the salient features of that workflow, and its uniqueness compared to traditional data processing systems.