Configuration reference
Capacity planning
Measured behaviour under load, per-language worker concurrency, and what to scale to hit a response-time target.
Note
Kiwi Distro is currently in private preview. Interfaces and behaviour described here can change before general availability. Contact Kiwify Labs for access.
These findings come from a real-worker load sweep on AWS (2026-09-14). The shape of the results is expected to generalise; the specific numbers belong to that topology. Re-run the sweep on your own deployment before relying on any figure.
The headline#
Below a concurrency roughly equal to the number of registered workers for a service, response time is flat and low, dominated by network and dispatch overhead. Above it, response time rises sharply, because requests queue behind a full worker pool. The lever for a response-time target is worker capacity matched to expected peak concurrent in-flight requests, not middleware tuning.
The measured curve#
Setup: 8 real worker processes (two each of .NET, Node.js, Python and Java samples) on one 2-vCPU host, three middleware instances behind a network load balancer, one MongoDB and one Redis (not an HA topology). A trivial hello method, so latency is overhead rather than handler work.
| Concurrency | Throughput | Mean | p50 | p95 | p99 |
|---|---|---|---|---|---|
| 4 | 234.9 req/s | 17.0 ms | 11.4 ms | 36.0 ms | 51.6 ms |
| 8 | 353.5 req/s | 22.6 ms | 13.8 ms | 51.7 ms | 70.1 ms |
| 16 | 96.0 req/s | 166.1 ms | 209.3 ms | 240.7 ms | 291.4 ms |
| 32 | 158.7 req/s | 200.9 ms | 211.3 ms | 278.3 ms | 426.6 ms |
| 64 | 209.1 req/s | 303.7 ms | 296.9 ms | 606.5 ms | 629.6 ms |
There were no failures at any level. The cliff sits between concurrency 8 and 16, where the worker count was exactly 8.
Note
This was a single run per level from one driver process with one trivial method. It establishes the pattern, not a repeatable benchmark. It also did not separate application-level queueing from CPU contention on the shared worker host.
Per-worker concurrency by SDK#
| SDK | Mechanism | Default | What more CPU buys you |
|---|---|---|---|
| .NET | runtime:request-process-parallelism |
Auto: CPU count × 2, clamped to 8–64 | More vCPUs raise the ceiling automatically |
| Java | Thread pool sized by maxConcurrency |
32 | Helps, but raise maxConcurrency explicitly to use a bigger host |
| Node.js | maxConcurrency |
32 | A process uses about one core. Run more processes on a bigger host |
| Python | max_concurrency (async semaphore) |
32 | Bound by the GIL. Run more processes |
Middleware knobs#
Three middleware settings exist, but were not the bottleneck in these tests: kiwi:min-threads (200), kiwi:task-queues:worker-wakeup-parallelism (100) and kiwi:task-queues:result-awaiter-parallelism (100). At about 455 requests per second with 500 synthetic workers, fleet-wide in-flight work was roughly 48 requests, around 16% of the 100-slot ceiling on either queue.
Choosing a worker language#
- Steady, provisioned-ahead load: .NET and Java scale cleanly with vCPU count.
- Bursty load with reactive scale-out: Node.js becomes request-ready fastest. Measured process-internal start-up (median of three, local machine): Node.js 249 ms, .NET 559 ms, Java 563 ms, Python 744 ms. Python was not meaningfully faster than .NET or Java.
That figure covers process start-up only, not container image pull or instance boot, which usually dominate a real scale-out event. If a burst also brings CPU-heavy handlers, Node.js and Python still need more processes to match what fewer .NET or Java processes can do.