define Activity-Term and invoke render Activities

[Ichthyostega]

As a Lumiera developer,
I need the ability to define and invoke simple render Activities,
to be able to post a render job to the Scheduler and get it executed there.

Scheduler implementation details

Originally, the Scheduler was planned just to invoke »Jobs«. On closer investigation, this turned out to be a oversimplification, since there are several flavours of jobs, which are bound to observe dependency constraints and can be triggered by external events (asynchronous IO). The original plan was elaborated thusly, so that the Scheduler now processes Activity records, which in turn are used to represent, manage and activate the render job functions. The #1311 BlockFlow provides an allocation scheme to support this intended usage. What is required now to complete the Scheduler in this modified form is

• a binding and setup to define terms of activities
• a lightweight execution context, backed by the Scheduler infrastructure

Acceptance Criteria

Scenario-1: build Activity-Term

Given

an Activity Language setup with a BlockFlow and execution stubs (λ)

When

I invoke the builder

Then

• an ActivityTerm is provided as copyable setup object in local stack frame
• the Activity records comprising this term are allocated into the BlockFlow
• the Activities are properly wired and connected (intrusive single linked list)
• the pointer to the Activity entrance point is ready for handover to the Scheduler

Scenario-2: invoke an Activity

Given

• a properly defined and wired Activity record of type INVOKE, GATE or NOTIFY
• an execution context bound to some stubs or action functions (λ)

When

I call the "invocation" function with the pointer to the Activity

Then

• the corresponding action λ is invoked in the execution context
• arguments are passed in accordance with the Activity's type selector field

[Ichthyostega]

Conceptually, this topic has been addressed and the low-level »mechanics« for activating render activities has been implemented comprehensively, fulfilling all currently foreseeable needs of time-bound and event triggered job activation. Effectively, way more ground has been covered than was initially intended; this became possible by abstracting the concrete scheduler functionality as a set of λ-functions and using a mock implementation to record and verify function invocations. This approach allows to reason about the right way to activate the Activity terms on a mostly abstract-logical level, and then to build and verify just the low-level wiring and dispatch logic. The »Activity Language« could thus be completed largely as a cohesive low-level execution framework, which can subsequently be used to express more intricate dependency relations between render jobs, while the scheduler is only required to process the Activity activations as wired and parametrised.

wiring templates

The analysis indicates that presumably all performance situations can be reduced to the following three execution templates

• CALC_JOB : a synchronous, CPU bound media computation task. It can optionally be gated to enforce a deadline, and to block until a predetermined number of prerequisite tasks has been fulfilled. Prerequisites are registered by sending a notification message upon completion
• LOAD_JOB : this setup uses the same basic scheme, but the execution chain is severed after triggering the JobFunctor. This corresponds to a model of asynchronous IO operations, where the extended IO task is dispatched only, expecting a callback upon completion. This callback will then activate the rest of the Activity chain, including the NOTIFY-Activity to unblock follow-up jobs depending on this IO operation.
• META_JOB : a reduced scheme for internal jobs used to carry on the Job-Planning itself, and for performing internal maintenance work

These three execution schemes have been covered in SchedulerActivity_test, thereby also subsuming the Acceptance Criteria of this ticket detailed above.

[Ichthyostega <prg@…>]

In 2e28f5d/Lumiera:

Activity-Lang: abstracted execution framework complete and tested (closes: #1319)

[Undercover Agent]

Migration MasterTickets ⟼ Subtickets-plugin