Amazon Video Rekognition

Videos submitted to Amazon Rekognition can be scanned for the same types of content as images, in addition to detecting Technical Cues. Videos are analyzed at a granularity of 500ms intervals, and clip markers are created showing the different ranges in which certain items appear. For example you might see a marker from 0s - 5s which says “Forest, Bear” and then a marker from 5s - 8s which just says “Forest”, meaning that the Bear (thankfully) left the frame at the 5s mark.

For job retrieval, the settings available in the Worker Plugin are:

Detection Types (either chosen manually or taken from a CatDV field of your choosing)

Minimum Confidence (0 to 99)

Minimum Marker Interval (in seconds)

Allow Repeat Matches (show just the first instance of a detected item, or all instances)

Show Confidence in Marker Description (e.g. ‘Confidence 83.55’)

Draw Bounding Boxes

Bounding Box Colour (Any valid CSS colour like “lightblue” or “#3698CF”)

Technical Cues

…will return frame-accurate marker ranges identifying these technical cues: Black Frames, Colour Bars, End Credits, and a marker range for each distinct shot or segment within the clip. Black frames with audio are considered to be part of the content and won’t be tagged. Note that for this detection type, the minimum marker interval and ‘allow repeat matches’ parameters are ignored to ensure the complete results are delivered.

Bounding Boxes

…for Labels, Faces and Celebrities are supported as part of the new marker annotation feature in the Web Client. If enabled, CatDV will create a unique marker for each instance of a recognized element, with a bounding box drawn around the element. This is handy for identifying and tracking movement of elements within a video.

When designing workflows around Bounding Boxes, you can dramatically change the results by changing the minimum marker interval or disabling ‘allow repeat matches’. This could let you show just the first instance of an element, or to show super detailed bounding box information you could set the minimum interval to 0 and enable ‘allow repeat matches’.

Note: Bounding boxes are not yet supported in the CatDV Desktop Client. Editing clip markers in the Desktop Client (13.0.12 as of this writing) will remove any bounding box data.

Video Rekognition: Best Practices

When submitting video analysis actions, CatDV will automatically upload the original media file if it is H.264 and less than 250MB in size. Media of other encoding types or above 250MB in size will be transcoded at 540p, 30fps for analysis to prevent huge uploads / storage costs. See Advanced Properties for more options around this.

Video analysis in particular can become expensive at scale. For this reason it is recommended that you do not automatically run video analysis on all of your content, but rather selectively feed or curate your video analysis jobs. Amazon has a limit of 20 concurrent jobs per account - currently the plugin does not have a queueing mechanism for running more than 20 jobs at once, but that functionality may be added in a future update.