Video annotation for machine learning with WebVMT at TPAC? #41

Video Annotation

Web Video Map Tracks (WebVMT) is an open format for synchronising location with video on the web. Recent developments have broadened its scope to enable any data to be synchronised with a video file in a web browser, which may be useful for machine learning use cases.

WebVMT provides a web-friendly way of annotating video footage to help train and test machine learning algorithms involving imagery. Data embedded in a video file can be exported to a VMT file to enable web access agnostically of encoding and to aggregate data from disparate sources. VMT files are text-based and include comments so they're easily readable by humans which facilitates multi-stage processing by either humans or machines. Text and binary data formats can be quickly displayed with video in a web browser and processed using reusable web apps.

Dash Cam Use Case

For example, dash cam footage analysed in OGC Testbed-21 identified pothole locations and vehicle collisions from accelerometer data with associated video. Machine learning could extend this example by quantifying pothole severity or identifying the object struck from image analysis of a video frame synchronised with that incident.

Please let me know if you’re interested in discussing ways to integrate machine learning data with web video or exploring overlapping interests further at TPAC in Dublin. Thanks.