genroc

    docs / Guides / Process definition

    Tasks and expressions

    Learn how tasks pass data to each other.

    Tasks can produce data and pass it on to the tasks that follow. That is what the task output and expressions are for.

    Task output

    nameUnique process identifier.: output-exampleUnique process identifier.
    tasksOrdered list of execution tasks. Control advances linearly unless a switch case redirects.:
      - idTask identifier, unique within the definition.: task1Task identifier, unique within the definition.
        outputtasks.task1.output — integer: 1tasks.task1.output — integer
        switchRequired. Routing: a shorthand ("next", "end", "$task-id") or an ordered list of cases.: nextRequired. Routing: a shorthand ("next", "end", "$task-id") or an ordered list of cases.
    
      - idTask identifier, unique within the definition.: task2Task identifier, unique within the definition.
        outputtasks.task2.output — integer: 2tasks.task2.output — integer
        switchRequired. Routing: a shorthand ("next", "end", "$task-id") or an ordered list of cases.: nextRequired. Routing: a shorthand ("next", "end", "$task-id") or an ordered list of cases.
    
      - idTask identifier, unique within the definition.: task_sumTask identifier, unique within the definition.
        outputoutputs.task1 + outputs.task2 → integer: "$: outputsoutputs → object{task1, task2}.task1outputs.task1 → integer + outputsoutputs → object{task1, task2}.task2outputs.task2 → integer"
        switchRequired. Routing: a shorthand ("next", "end", "$task-id") or an ordered list of cases.: endRequired. Routing: a shorthand ("next", "end", "$task-id") or an ordered list of cases.

    Task output defines what will be passed to following tasks. If the output is defined, the following tasks can read it through the outputs.<task_id> expression.

    The $: prefix marks an expression, which we explain below.

    Expressions

    Genroc supports expressions, which let you compute values dynamically and pass them from one task to another. In an expression you can reference the following variables:

    • input - the process input defined by input_schema
    • outputs.<task_id> - output of a task that runs before this one on the way here (if it defines one; one that may have been skipped can be null)
    • self - data about the current task, not available outside it (see looping)
    • config - configuration values resolved from environment variables (we will get to this later)

    Error handling adds error and last_error, which we get to in error handling.

    You can also use basic operators like ==, !=, <, <=, >, >=, &&, ||, ??.

    Expressions can also compose objects or arrays:

    {name: "John"}        # returns object
    ["John", "Marie"]     # returns array

    map function

    For working with arrays, genroc also supports map function. It takes an array as a first argument and callback as second. In callback you can transform the items into different shapes.

    "$: map(names, (name) => { who: name })"
    # Array<names> -> Array<{who: string}>

    String template

    This is a simple use case: it lets you compose a string value dynamically.

    "Hello ${input.user_name}!"

    The value passed into a template must be a primitive value (string, number or boolean), and it cannot be null.

    You can escape ${} by doubling the $, e.g.:

    "Hello $${input.user_name}!"

    Expression of any type

    By prefixing the string with $: you are not creating a template. The whole expression result is used and its type is preserved:

    "$: outputs.task1 + outputs.task2"  # integer

    This way you can also pass whole objects, e.g.:

    "$: input"                          # { user_name: ... }

    Raw expressions

    In switch/case fields a static value would make no sense, so there is no $: or ${} syntax, you write the expression directly:

    switch:
      - case: "input.user_name == 'Brenda'"
        goto: ...

    The same syntax is used also for on_error/case fields, which we will discuss later.

    Numbers

    Expressions support +, -, *, / and %. Numbers are arbitrary-precision decimals (up to 1000 significant digits; a result past that fails the process with engine.expression), following the General Decimal Arithmetic specification (the one Python's decimal module implements), computed with the apd library. Genroc keeps them exact, also when it only passes them through, so a long id like 9007199254740993 arrives unchanged, which many JSON tools don't guarantee.

    +, - and * are exact, decimals included:

    0.1 + 0.2              # 0.3
    0.1 + 0.2 == 0.3       # true
    input.id + 1           # 9007199254740994

    / is the only operator that rounds, to 34 significant digits (the precision of IEEE 754 decimal128), with ties rounded away from zero:

    1 / 3                  # 0.3333333333333333333333333333333333
    2 / 3 * 3              # 2.0000000000000000000000000000000001

    The way a number is written decides its type: 3 is an integer, 3.0 is a number. % works on integers only, so 7 % 2 is fine and 7 % 2.0 is refused when you apply the definition.

    A client reading results in JavaScript loses precision on large numbers in JSON.parse, so a long id can still break there, on the client's side.

    Looping

    You can also use switch to create loops. We however need a way to express a condition when to stop and keep track of current state.

    nameUnique process identifier.: looperUnique process identifier.
    # number of iterations as an input
    input_schemaJSON Schema used to validate the input payload when starting a new instance.:
      typeThe value's JSON type, or a list of types it may take.: integerThe value's JSON type, or a list of types it may take.
    tasksOrdered list of execution tasks. Control advances linearly unless a switch case redirects.:
      - idTask identifier, unique within the definition.: task1Task identifier, unique within the definition.
        # counting the iterations (previous output + 1)
        output(self.previous ?? 0) + 1 → integer: "$: (selfself → object{previous?}.previousself.previous → integer|null ?? 0) + 1(self.previous ?? 0) + 1 → integer"
        switchRequired. Routing: a shorthand ("next", "end", "$task-id") or an ordered list of cases.:
            # looping condition
          - caseself.output < input → boolean: "selfself → object{output, previous?}.outputself.output → integer < inputinput → integer"
            goto"end" to terminate, "next" to advance, or "$task-id" to jump to a task.: $task1"end" to terminate, "next" to advance, or "$task-id" to jump to a task.
          - goto"end" to terminate, "next" to advance, or "$task-id" to jump to a task.: end"end" to terminate, "next" to advance, or "$task-id" to jump to a task.

    This process takes input of single integer (number of iterations) and then loops on task1 while counting the number of iterations in the output field. In the switch statement we compare the counter (task output) with the required number of iterations (process input).

    The counter lives in self:

    • self.previous - the output of the last run of this task, null on the first run (hence the ??)
    • self.output - the output this run just produced, readable only in switch

    Actions add self.result, which we get to in the next chapter. Which slot can read which is in the expressions reference.

    Shapes

    Shapes are a convenient way to transform data. The best example is the task output field.

    Fully static

    A shape can be fully static in YAML, with no expressions at all.

    output:
      ok: true

    Combined

    Define the object shape statically and pass the values dynamically.

    output:
      name: "$: input.user_name"

    Expression only

    Because expressions can define objects, you can build the whole structure in one expression, or pass a variable that already has the expected shape.

    output: "$: { name: input.user_name }"

    Slots

    You can use shapes and expressions anywhere a dynamic value is expected. In the examples above that is:

    SlotWhereSyntax
    caseswitchraw expression
    outputtaskshape
    outputprocessshape

    Every action and clause introduced later brings its own slots, e.g. a fetch body is a shape and its url a string template, but the syntax is always one of these.

    Process output

    A process can optionally define an output. It looks like this:

    tasks:
      - id: task_sum
        output: "$: outputs.task1 + outputs.task2"
        switch: end
    
    output:
      sum: "$: outputs.task_sum"

    It has the same shape and syntax as a task output. It is evaluated once the process reaches switch: end. It cannot access self, as it is not part of any task.

    The output is the contract for the outside world: it is what callers read, and what a parent checks a child's result against. Task outputs are internal — visible in genctl detail, but nothing outside is written against them.