Use LLM activities

Kiwi Flow has four LLM activity types. Calls go through a provider-agnostic abstraction, so you can switch between OpenAI, Azure OpenAI, Anthropic and Ollama (self-hosted) without changing workflow logic.

Text operations#

llm_text builds its prompts for you when you pick a built-in operation. The prompt and systemPrompt fields apply only to custom. If operation is absent, the activity behaves as custom.

{
  "id": "a_summary",
  "type": "llm_text",
  "llm_text": {
    "provider": "anthropic",
    "model": "<model-id>",
    "operation": "summarize",
    "operationConfig": { "maxCharacters": 300, "format": "bullets", "language": "english" },
    "temperature": 0.3,
    "maxTokens": 1024
  },
  "inputs": [ { "name": "content", "from": "a_prev.output.body" } ],
  "outputs": [ { "name": "summary", "from": "output.text" } ]
}

Besides output.text, extract adds each parsed field as a named output (when outputFormat is json), classify adds category and categories, and sentiment adds sentiment and an optional reason.

AI-driven routing#

llm_decision asks the model for JSON and validates it against outputSchema before mapping it to outputs. Pair it with switch:

{
  "id": "a_llm",
  "type": "llm_decision",
  "llm_decision": {
    "provider": "anthropic",
    "model": "<model-id>",
    "outputSchema": { "decision": "string", "riskLevel": "string", "confidence": "number" },
    "temperature": 0.2,
    "maxTokens": 512
  },
  "inputs": [ { "name": "prompt", "value": "Given orderValue={{inputs.orderValue}}, classify the risk. Return JSON." } ],
  "outputs": [
    { "name": "decision",  "from": "output.decision" },
    { "name": "riskLevel", "from": "output.riskLevel" }
  ],
  "next": "a_risk_switch"
}

Then route on a_llm.output.decision with a switch. See Branch, loop and run in parallel.

Retrieval-augmented answers#

llm_rag searches a vector corpus (set with corpusId, topK, minScore in its config block, or supply documents at run time) and returns answer, the retrieved context, sources and chunkCount. When PostgreSQL is the platform database, embeddings are stored with pgvector; other database engines can use an alternative vector store.

Providers and keys#

Supported providers: openai, azure_openai, anthropic and ollama. You bring your own API key. Register provider configurations as AI assistants:

kiwi flow ai-assistant-set --name default --provider Anthropic --model <model-id> --api-key "<provider-key>"
kiwi flow ai-assistant-test --name default
kiwi flow ai-assistant-default --name default

Azure OpenAI and self-hosted models also need --endpoint (and --deployment for Azure).