How-to guide

Query and write data with Kiwi Fabric

Use the Kiwi Fabric activities to run federated SQL and distributed transactions from a workflow, with no database coupling.

Flow Current 2 min read Updated

Prerequisites

  • A Kiwi Fabric instance with registered data sources, and an API key
  • The Kiwi Fabric connector entitlement on your licence

Instead of connecting a workflow directly to each database, point it at Kiwi Fabric. One federated SQL statement can span several systems, and a saga can write to them atomically with automatic rollback. The workflow contains no connection strings for the underlying systems.

Activities#

Type Purpose
kiwi_fabric_connection Open a connection (baseUrl and apiKey) for the other Fabric activities to reuse
kiwi_fabric_query Run a federated SQL query
kiwi_fabric_saga Run a multi-step distributed transaction with automatic compensation
kiwi_fabric_datasource Register a data source and/or a queryable capability, for example to make an uploaded file queryable by name
kiwi_fabric_file Upload a file into Kiwi Fabric's managed storage, or list and delete uploaded files

Query#

json
{
  "id": "a_orders",
  "type": "kiwi_fabric_query",
  "inputs": [
    { "name": "connection", "from": "kiwi_fabric_connection_1.output.connection" },
    { "name": "sql", "value": "SELECT o.id, c.name FROM orders o JOIN crm_customers c ON o.customer_id = c.id WHERE o.total > @minTotal" },
    { "name": "parameters", "value": { "minTotal": 100 } }
  ],
  "outputs": [ { "name": "rows", "from": "output.rows" } ]
}

Outputs: rows, rowCount, originalSql, totalMs and sourceSummaries (per-source telemetry). Always use @name parameters rather than concatenating values into SQL.

Distributed transaction#

json
{
  "id": "a_place_order",
  "type": "kiwi_fabric_saga",
  "inputs": [
    { "name": "connection", "from": "kiwi_fabric_connection_1.output.connection" },
    { "name": "steps", "value": [
      { "id": "s1", "table": "orders", "operation": "insert", "payload": { "customerId": 42, "total": 199.99 } }
    ] }
  ]
}

steps uses the same format as a Kiwi Fabric saga (id, table, operation, dependsOn, payload, and so on). Outputs: sagaId, success, durationMs, errorMessage, steps and gaps. If the saga fails, Kiwi Fabric compensates the steps already done; check success and route accordingly with an if_else. See Run a distributed transaction in the Kiwi Fabric documentation.

Connections#

Each activity either reuses a kiwi_fabric_connection node or accepts baseUrl and apiKey directly. Prefer a saved Connection so the API key is stored encrypted and not written into the workflow. See Connections and credentials.

Expected result

  • Workflow steps read from, and transactionally write to, several systems through Kiwi Fabric