Validation and types

Learn about validation, inference and type checking.

Genroc requires you to validate the data entering a process, so it always knows what shape that data has.

Why?

Durable processes are tricky, because they can survive for a long time. A common scenario is that you are using multiple endpoints on your API. For example, one to fetch some information and another to mutate something.

The issue is that your API can change in the meantime. If you fetch some data, then pause the process for a month and resume it, it will be talking to the new API with assumptions from the old one.

Strict validation can’t prevent these issues, but makes them easy to detect.

Static checking and inference

Because the inputs are typed, we can statically check every expression: that it references only properties that exist, and that its operations use the right types.

We can also infer expression types, so if you use an expression to build a new object, genroc infers its type automatically.

Inferred context

Through inference, Genroc creates a typed context for every task. Based on what tasks were (or could have been) executed before it, we can check exactly which outputs can be accessed in the expressions.

This creates a continuous flow of types through the process, so you always know what outputs are accessible and what their types are.

Nullability

Genroc tracks whether a field can be null. It doesn’t allow nullable fields in templates and operations that don’t make sense on a null type (e.g. <). Comparing to null is always allowed, so x == null is how you test for one.

There are two ways to get from a nullable value to one you can use. Give it a fallback with the ?? operator:

url: "${input.server_url ?? 'http://localhost:1234'}/user"

Or check it first. A guard on the left of && narrows the right side, because the right side only runs when the left one held:

case: "input.retries != null && input.retries > 3"

|| works the same way in reverse — its right side runs only when the left one failed — and ! swaps the two. Several guards chain, so a != null && b != null && a > b narrows both.

Note that the check has to come first. input.retries > 3 && input.retries != null is still refused, because the comparison is reached before anything has been proven.

You can also use default to fill in a missing value.

input_schema:
  type: object
  properties:
    server_url:
      type: string
      default: http://localhost:1234

Even though the value is not required here, genroc knows it can never be null inside the process, because if you don’t provide it, the default value will be used. Combining required and default is refused, because then the default can never be applied.

This changes, however, if it’s specified as type: [string, "null"]. You can then pass null explicitly and the default value is not used.

Genroc has only one NIL value, null, so in the expressions there is no difference between nullable and non-required values (they both contain null).

Demonstration on a real example

nameUnique process identifier.: nudge-inactive-userUnique process identifier.
input_schemaJSON Schema used to validate the input payload when starting a new instance.:
  typeThe value's JSON type, or a list of types it may take.: objectThe value's JSON type, or a list of types it may take.
  propertiesThe named members of an object, each a schema.:
    user_id: { typeThe value's JSON type, or a list of types it may take.: stringThe value's JSON type, or a list of types it may take. }
  requiredWhich properties must be present. A property not listed here may be absent, and reading it yields null.: [user_id]
tasksOrdered list of execution tasks. Control advances linearly unless a switch case redirects.:
  - idTask identifier, unique within the definition.: load_userTask identifier, unique within the definition.
    actionDescribes the action to perform. Omit for switch-only (routing) tasks.:     # here the action can access `input`
      typeHTTP call. URL, method, headers and body are all expressions.: fetchHTTP call. URL, method, headers and body are all expressions.
      methodHTTP method, lowercase (e.g. get, post) or a template such as ${ input.method }. Required — the verb is never guessed.: getHTTP method, lowercase (e.g. get, post) or a template such as ${ input.method }. Required — the verb is never guessed.
      urlRequest URL. May contain ${ } interpolations, e.g. ${ config.server_url }/path.: "https://api.example.com/users/${inputinput → object{user_id}.user_idinput.user_id → string}"
      responsesStatus pattern -> JSON Schema for that body. A 2xx key types self.result and accepts the status.:
        200:
          typeThe value's JSON type, or a list of types it may take.: objectThe value's JSON type, or a list of types it may take.
          propertiesThe named members of an object, each a schema.:
            email: { typeThe value's JSON type, or a list of types it may take.: stringThe value's JSON type, or a list of types it may take. }
            activated: { typeThe value's JSON type, or a list of types it may take.: booleanThe value's JSON type, or a list of types it may take. }
          requiredWhich properties must be present. A property not listed here may be absent, and reading it yields null.: [email, activated]
    outputtasks.load_user.output — object{email}:     # you can access `input` and `self{result, headers, status}` (coming from fetch)
      emailemail — string: "$: selfself → object{headers, result, status}.resultself.result → object{activated, email}.emailself.result.email → string"
    switchRequired. Routing: a shorthand ("next", "end", "$task-id") or an ordered list of cases.:     # here you can access both `self.result` and `self.output` if you want (+ everything from before)
      - caseself.result.activated → boolean: "selfself → object{headers, output, result, status}.resultself.result → object{activated, email}.activatedself.result.activated → boolean"
        goto"end" to terminate, "next" to advance, or "$task-id" to jump to a task.: end"end" to terminate, "next" to advance, or "$task-id" to jump to a task.
      - goto"end" to terminate, "next" to advance, or "$task-id" to jump to a task.: $nudge"end" to terminate, "next" to advance, or "$task-id" to jump to a task.

  - idTask identifier, unique within the definition.: nudgeTask identifier, unique within the definition.
    actionDescribes the action to perform. Omit for switch-only (routing) tasks.:     # now `outputs.load_user` is available (output of the previous task)
      typeHTTP call. URL, method, headers and body are all expressions.: fetchHTTP call. URL, method, headers and body are all expressions.
      methodHTTP method, lowercase (e.g. get, post) or a template such as ${ input.method }. Required — the verb is never guessed.: postHTTP method, lowercase (e.g. get, post) or a template such as ${ input.method }. Required — the verb is never guessed.
      urlRequest URL. May contain ${ } interpolations, e.g. ${ config.server_url }/path.: "https://api.example.com/emails"
      bodytasks.nudge.action.body — object{to}:
        toto — string: "$: outputsoutputs → object{load_user}.load_useroutputs.load_user → object{email}.emailoutputs.load_user.email → string"
    outputtasks.nudge.output — object{nudged}:
      nudgednudged — boolean: true
    switchRequired. Routing: a shorthand ("next", "end", "$task-id") or an ordered list of cases.: endRequired. Routing: a shorthand ("next", "end", "$task-id") or an ordered list of cases.

outputoutput — object{email, nudged}:
  emailemail — string: "$: outputsoutputs → object{load_user, nudge}.load_useroutputs.load_user → object{email}.emailoutputs.load_user.email → string"
  # nudge task might not run, so the output can be null
  nudgednudged — boolean: "$: outputsoutputs → object{load_user, nudge}.nudgeoutputs.nudge → object{nudged}|null.nudgedoutputs.nudge.nudged → boolean|null ?? falsefalse → boolean"

TIP: Install the genroc vscode extension and copy the process above to a new definition. You can then hover any expression part and see what its type is. It will also show you errors live.

Declaring what enters the process

In the example, there are two places where outside data enter the process - input_schema and responses.

Genroc automatically validates that data and then you can safely access it in expressions.

Inferred types

Every task output is inferred through the output shape.

load_user output is typed as an object with an email: string field
nudge output is an object with a nudged: boolean field

The process output is also inferred; the inferred context is based on where all the goto: end are in the process.

Static checking

We check that the expressions access existing properties. If you make a typo (e.g. "$: outputs.load_user.emial"), genroc will throw an error.

The context of a slot

Each slot can have a slightly different context depending on previous tasks and on its position in the task. For example, you can’t use self.result in the action configuration, because the result is available only after the action has executed.

Other checks

Genroc also validates switch/goto targets (that the task exists) and doesn’t allow unreachable tasks.

Config

Config lets a process read environment variables. You can define config_schema. It must be a flat object, e.g.:

name: process-with-config
config_schema:
  type: object
  properties:
    server_url:
      type: string
  required: [server_url]

You can then provide config properties through environment variables like this:

# scoped to a single process
GENROC_PROCESS_WITH_CONFIG_SERVER_URL=http://example.project
# or globally for all processes
GENROC_GLOBAL_SERVER_URL=http://example.project

You can then access config anywhere in the process expressions.

    url: "${config.server_url}/users/${input.user_id}"

Genroc loads and validates config values every time it interacts with the process. You can declare any primitive type; genroc validates the value and converts it. If the validation doesn’t pass, the process is not allowed to register. So if the value is required, you need to set the env before applying the definition (on the server).

Secrets

In the config you can mark a property as secret.

name: process-with-config
config_schema:
  type: object
  properties:
    server_token:
      type: string
      secret: true
  required: [server_token]

This will redact the secrets that would otherwise leak into stdout (server logs). Secrets can still leak into db-stored logs or process state - so they will be visible through genctl logs and so on.

The secret property can only be used in the config.

The benefits

Genroc’s type system ensures validity at two different moments, which complement each other.

At registration - everything is statically checked - all the expressions, the types they produce, goto targets, reachability. A typo in an expression never reaches production. genctl apply --check-only runs exactly these checks, so you can run them in CI; they happen on the server, so you then know that apply will pass.

At runtime - the outside data entering the process is always validated. This ensures that every statically checked expression recieves the data it expects, and breaking API changes are detected.

Scenarios

What is sent by the fetch is judged by the server; what we receive is judged against the responses you declared.

what changed	how you find out
the request is no longer accepted	the server answers `4xx`, caught as `http.400`, `http.404`, …
the response body no longer passes validation	`result.invalid`
the response gained extra fields	nothing - they are dropped (non-breaking change)

Process state upgradability

Because genroc knows what the state schema should look like at every stage of the process, it can decide whether an old version of the process is upgradable to a newer version. We will look at this feature later.