Orchestration

Main logic for orchestrating the agents in the system is responsible for managing the flow of information between the different steps and agent(s) responsible for them. The orchestrator is responsible for creating the task queue, executing the tasks in the task queue, and composing or verifying the results. The orchestrator is also responsible for managing the agents (i.e. initializing and running all of their operations) in the system and their interactions with each other.

The interface of the Orchestrator is defined with initialization

class Orchestrator(ABC):

    def __init__(self) -> None:

        self._taskqueue = TaskQueue()
        self._config = ConfigModel()

and its main function run as:

@abstractmethod
def run(self, 
        instruction: str, 
        input_session: str
    ) -> str

The run method of the orchestrator handles a specific instruction. The orchestrator generates a plan using the planner (if specified), creates a task queue with the plan, completes the tasks in the task queue using the agent(s) in the Executing step, and follows with the composition or verification (or both). If only the Execution step is present, the orchestration is done without the use of task queue and the process is similar to the Agent class.

Default orchestration

The DynamicAgentOrchestrator is the default orchestration mechanism in Elemental. Its initialization signature is

class DynamicAgentOrchestrator(Orchestrator):
    def __init__(
            self,
            planner: Agent | AgentTeam | None,
            plan_verifier: Agent | AgentTeam | None,
            replanner: Agent | AgentTeam | None,
            executor: Agent | AgentTeam | None,
            verifier: Agent | AgentTeam | None,
            composer: Agent | AgentTeam | None,
        ) -> None

The default orchestration pattern in Elemental is a sequence of steps that are executed in a specific order. The steps are:

Planning (Optional)- The planner generates a plan based on the instruction and input session. The plan is a sequence of tasks that need to be executed.
Plan Verification (Optional) - The plan is verified to ensure that it is valid and can be executed. This step is optional and can be skipped if the planner is not used.
Execution (Mandatory) - The tasks in the task queue are executed using the agent(s) in the system. The execution step is mandatory and must be present in the orchestration.
RePlanning (Optional) - The plan is re-generated based on the results of the execution step. This step is invoked after every task from the Task Queue is completed. The remainder of the plan is subject to be modified by the agent(s) in this step.
Composition (Optional) - The results of the execution step are composed to generate the final result.
Verification (Optional) - The final result is verified to check if the result could be viable solution to the original task.

The figure below displays the flow of the information in the DynamicAgentOrchestrator which is the default orchestration mechanism in Elemental.

Default Agent Orchestration Pattern in Elemental — Default agent orchestration pattern in Elemental. Boxes represent logical steps in the orchestration process. The arrows represent the flow of information between the steps. Gray box is mandatory and white boxes are optional. Each box represents either single agent or team of agent. Green boxes **Task** and **Result** represent the user visible input and output. If the **Execution** step is the only step present, the orchestration is done without the use of task queue.

For clarity the above figure does not include the Task Queue and operations that simply access it or are able to modify it. The Task Queue and operations in the information flow that access and modify it are introduced in the figure below. All of the planning steps that include Planner, Plan Verification, and RePlanning are able to modify the Task Queue. The Planning step does create the initial version of the subtasks and their dependencies in the Task Queue. Plan Verification is a double check on the structure of dependencies and the completenes of the subtask's descriptions. The RePlanning step is invoked after every task from the Task Queue is completed. The remainder of the plan is subject to be modified by the agent(s) in this step.

In the figure below, the Task Queue is displayed as a purple box. The operations between a given step of the workflow and Task Queue are indicated with the purple arrows.

The exact passing of information between different steps may occur directly or through the use of Task Queue. Both Planner and Plan Verification steps are creating the initial set of tasks and populating the Task Queue. After this in Execution, tasks are fetched from the queue one-by-one and executed, if the RePlanning step is used, it alternates with the executing. RePlanner reviews the remaining subtasks in the queue based on the results of those already completed.

Composition step has access to all results from the Task Queue. Verification step receives the message directly from the Composition if enabled.

Orchestration Without Task Queue

Execution is the only mandatory step in the orchestration process. The orchestration can be done without the use of Task Queue. In this case, the orchestration is done in a similar way to the Agent class. The Execution step is responsible for executing the task directly taken from the user input and its result is then used as a final response.

Default Agent Orchestration With Execution Step Only — Orchestration with no **Planning** step and **Task Queue**.

Agent Selection

Driver