Workflow concepts

Pipelogiq coordinates runtime work across application handlers. Your code defines what a handler does; Pipelogiq stores the run, dispatches eligible stages, accepts results, and exposes progress for operators.

Applications and pipeline runs

An application owns its pipelines, API keys, schedules, and worker registrations. Dashboard users are assigned to applications. A pipeline is one execution with a numeric ID, name, stages, timestamps, and current status. Creating another run with the same name normally creates another pipeline; the name is not an idempotency key.

A schedule stores a reusable pipeline definition and trigger. Each firing creates an ordinary pipeline with its own history. An API-created pipeline is already a run, not a reusable template.

Use names that a person can recognize, such as dispatch-invoice. Attach keywords and context to identify the business operation rather than putting every identifier into the name.

Stages and handlers

A stage name identifies the step within the pipeline. A handler name identifies the application code that can execute it. Several stages may use the same handler with different inputs. Workers must register the same handler name that the pipeline specifies.

Stages run in declaration order by default. A stage with an explicit dependsOn list waits for those named stages within its pipeline. Treat stage names as stable identifiers: use unique names without commas, tabs, or line breaks. Use the exact names and check that every dependency exists; do not use database stage IDs as dependencies.

{
  "name": "dispatch-invoice",
  "stages": [
    { "stageName": "load", "stageHandlerName": "LoadInvoiceHandler" },
    { "stageName": "email", "stageHandlerName": "EmailInvoiceHandler", "options": { "dependsOn": ["load"] } },
    { "stageName": "archive", "stageHandlerName": "ArchiveInvoiceHandler", "options": { "dependsOn": ["load"] } }
  ]
}

After load completes, email and archive can become eligible independently. Actual concurrency depends on workers and delivery capacity. There is no drag-and-drop workflow authoring surface; define workflows in the SDK or API.

The schema includes runInParallelWith, failIfOutputEmpty, notifyOnFailure, and runAsUser, but these are not general execution guarantees in the current server. Use named dependencies for ordering and implement output validation and business authorization in your handlers.

Two kinds of worker

Starting the Docker stack does not provide your payment, email, or document handlers. The installation test supplies a temporary HTTP application worker so you can verify the complete path before adding your own.

An execution attempt has an executionId and attempt. These are distinct from the pipeline ID and stage ID. Workers echo them in results so a delayed response from an older attempt cannot overwrite a newer execution.

Input, output, context, and keywords

Stage input and result output are strings. JSON input must therefore be serialized as a string in raw HTTP requests:

{
  "stageName": "email",
  "stageHandlerName": "EmailInvoiceHandler",
  "input": "{\"invoiceId\":\"invoice-42\"}"
}

Context is the pipeline's shared set of key/value items. Values are strings with optional type metadata. Workers can add/update context and send deletion markers. Keywords are key/value labels useful for grouping and finding runs. They are not a secret store.

{
  "pipelineKeywords": [{ "key": "workflow", "value": "invoicing" }],
  "pipelineContextItems": [
    { "key": "invoiceId", "value": "invoice-42" },
    { "key": "accessToken", "value": "provided-at-runtime", "isSensitive": true }
  ]
}

The token value above illustrates the shape; provide a real runtime value through your application. Sensitive context is redacted in public run views, while execution workers receive the actual value. Mark values sensitive before they enter the workflow. Redaction is not automatic identification of all personal or confidential data.

Events, approvals, and dynamically added stages

isEvent marks an event stage; it still follows durable stage scheduling and result processing. It is not a subscription that waits for an arbitrary external event to arrive.

A handler can return isWaitingForApproval:true. The stage then stays at WaitingForApproval until an application calls the approval-resume API. Approval decisions are implemented through the SDK/API, not a dashboard approve/reject button. See reliability.

An application can append stages to a nonterminal run, or return appended stages in a worker result. This supports work discovered during execution. Appending after the pipeline is terminal is rejected; it is not a way to reopen a finished run.

There is no general stage timer option. Use Once schedules for a future pipeline start and retry options for retry delays.