Masking and Overlaying

[Ichthyostega]

Work out how video content can be masked and combined / overlayed. This includes developing a sane (!) concept how to deal with »premultiplied alpha«

Generally speaking, combining media calculation chains may include some kind of adaptation, transcoding or even rendering. The question if an overlaying and adaptation is even possible and by which means shall be delegated to the 1330 »Stream Type System« and rules-based configuration.

Notable examples:

• two adjacent video clips are combined with a soft fade
• a video clip is placed on top of another one, using a combination mode
  (⟶see Gimp for a comprehensive definition of combination modes,
  which is used in many other OpenSource media applications as well)
• in addition, clips may have an alpha channel, and the alpha is premultiplied or not premultiplied, possibly requiring to insert a premultiplying or de-premultiplying node. Premultiplying is a destructive operation and thus the placement of a premultiplying node is a concern of optimal Render-Path buidling
• there can be a possibly animated mask, defined by splines, which must be rendered and then applied (which again raises the question of how to deal with premultiplication); alternatively, masking can be derived from a side-chain of the render-path-graph (e.g. from a colour matte, green-screen or AI-generated)
• the clips may have different colour models, colour resolution or more general data representation, necessitating conversions; it depends on rules-based configuration if such a conversion is acceptable for a given project; otherwise the connection must be flagged as error, which can then be overruled on a case-by-case base. Capturing and persisting this information is explicitly part of this concept and has to be considered carefully
• a 2D clip can be included into a 3D video; likewise a monaural sound clip can be incorporated into a stereo, surround or ambisonics mix. These cases require to render the inserted source into an extended presentation space.

[Undercover Agent]

Migration MasterTickets ⟼ Subtickets-plugin