New Sequence Player

In CatDV 12.1 we also put a lot of effort into extending and optimising the new sequence player in the native helper. Up until that the only real way of playing a sequence that assembles media from different files (both proper sequences and metaclips that might include video spanned over 2 or more files, plus P2 and Avid op atom clips) was to rely on QuickTime. (The FFmpeg NH player supports audio in a different file from the video, but that’s about it.)

The NH new sequence player allows different video and audio files to be assembled into a sequence that can be played as a whole, without requiring QuickTime. It’s built on top of “tracks” and “bitstream decoders” that provide low level decoding using different technologies, including making use of hardware acceleration if available. It’s also multi threaded, to provide much better seeking and playback performance all round, even with 4K files. The new sequence player will therefore become the new default player for most file types even if it’s just playing a single file and not a sequence as such.

There are different types of track, for example TinMan video track, FFMpeg audio track, RED video track, and so on. In the case of FFmpeg video tracks we use FFmpeg to decode the container format and then use special bit stream decoders to decode the video, depending on the format and what hardware is available:

· NVIDIA hardware acceleration on Windows and Linux to decode mpeg2 and h264, and also h265 with modern graphics cards

· CUDA hardware acceleration on Windows and Linux to decode ARRI and RED frames

· Metal hardware acceleration on macOS to decode RED frames

· VideoToolBox acceleration on Mac to decode h264, and h265 on High Sierra and with the latest hardware

· ProRes decoder using Apple’s official libraries to decode ProRes on Mac, Windows, and Linux

· ProRes encoder using Apple's official libraries to encode ProRes on macOS, Windows and Linux

· Using FFmpeg and avlib to decode the bitstream is the fallback for all other codecs

Note above that ProRes hardware acceleration will tend to only be available on newer Mac machines.

Initially, the sequence player used a movie file per item, something that worked well for a sequence. For example, a P2 file might comprise one video track and four audio tracks over five files. So, when a sequence was created with the aforementioned P2 movie, there'd be one sequence track containing the video track and four sequence tracks containing the audio tracks.

We quickly realised though that most people would be using the sequence player to play a single file, like the regular FFmpeg or AVFoundation players. Because of this, in CatDV 14 we made changes so that a single movie file could be shared between sequence tracks if we knew the same single file was the source of all video and audio tracks required, as opposed to opening the same file multiple times as is expected for a sequence. This improved performance quite a bit (for example, seeking over one file as opposed to multiple files).

For situations where a single movie is used (meaning all sequence tracks can share the movie file as mentioned above) we also added support for growing files. The sequence will recognise when a movie file has changed via an initial grow or a subsequent grow and handle how the movie is extended to all sequence tracks that might be sharing it.

Many enhancements have been made for "sequence mode" also. When a sequence is opened, the most common use case is that it will comprise of low-level FFmpeg tracks. In the FFmpeg importer, a component that the FFmpeg audio and video track both use, various checks are made to analyse the movie and determine how to reliably seek over a file. Although this used to be performed on a per file basis, it's now cached to improve opening time by quite a bit, especially for very complex sequences.

We also added asynchronous commands to low level FFmpeg audio/video tracks so that commands can be batched in parallel, massively speeding up the time to seek on a sequence track, close a sequence track etc, something that was a real problem with very complex sequences.