Video Plug-ins

The ‘visual channel’ within a video signal contains a wealth of information that can be extracted to help form an index of the video. Examples of visual analysis include keyframing, face detection and recognition, on-screen text recognition, logo recognition, and more. Typically, a visual analysis is restricted to a narrow problem domain because image understanding is a very difficult problem to solve in the generic sense. As a result, Autonomy Virage employs a Media Analysis framework for incorporating modular algorithms for visual analysis, while still maintaining real-time performance criteria.

Keyframe

“Intelligent Keyframing” is a central Autonomy Virage capability that drives the entire concept and usefulness of cataloging video. The VideoLogger analysis engine creates storyboards that visually depict the most important content in the video, and are optimized for data storage.

The Autonomy Virage keyframing engine is built on patented technology developed for image search and “similarity matching.” For video, the keyframing engine measures “dissimilarity” between subsequent frames of the video stream. The keyframing engine is based on proprietary media analysis algorithms that operate in color space, and on shapes and edges. The idea is to compute these properties in an extremely efficient manner so that the computations can be applied 10-30 times per second, yet provide good visual discrimination. A resulting keyframe is a visual representation of a group of frames in the video.

The definition of “visually different” is a subjective quantity and can vary significantly by application domain. For example, consider the transition differences of music videos compared to news. To accommodate this variability, the difference threshold value is exposed in the VideoLogger user interface as a sensitivity slider representing more- or less-aggressive keyframing. The sensitivity setting can also be set through application development interfaces so that it may be remotely tuned as needed. The sensitivity control is augmented by a ‘bandpass filter’ that allows you to specify minimum and maximum intervals for grabbing keyframes.

Facial Identification

Autonomy Virage’s Facial Identification plug-in provides automatic real-time detection and identification of faces that appear within the video and can compare these to a database of known individuals. For example, news broadcasts use this to create an accurate index to locate footage where perhaps notorious individuals appear, so that the required footage can be easily accessed, cross-referenced, and re-used.

Optical Character Recognition

Autonomy Virage’s Optical Character Recognition (OCR) plug-in provides automatic real-time detection and identification of characters and numbers that appear in the video. The plug-in configuration settings allow you to optimize its operation in a variety of ways, including definition of regions of interest, contrast settings (light-on-dark, dark-on-light), and lexicon options. For example, news broadcasts tend to use the lower third of the screen to display location and reporter names; by defining this as a region of interest, you can significantly optimize the plug-in performance and reduce computation needs.

Scene Analysis

Autonomy Virage is able to extract content by "watching" video and identifying key objects within the image. This technology also recognizes changes in the scene, enabling it to extract more information from the video file than can be indexed manually. This includes recognizing key objects such as a house, a person, or a sequence of rapidly changing events. Combined with Intelligent Keyframing, Scene Analysis can create storyboards automatically, giving users an unprecedented level of control over their video assets.