Scaling

Moʻolelo's services scale horizontally on different signals depending on their role. This page covers the typical knobs and the sizing math.

How each service scales

Service	Scaling signal	Scaling mechanism
webhook-ingress	HTTP requests/sec	Standard Kubernetes HPA on CPU
connector	One pod per (tenant, source) pair	Replica count per tenant; small footprint
iceberg-sink	One pod per <region>.chunks.tokenized topic (single-writer per partition)	Replica count = 1 per topic
entity-extractor-dispatcher	Pulsar lag on the input topic	KEDA ScaledObject
tokenizer-driver	Pulsar lag on <region>.chunks.embedded	KEDA ScaledObject
vespa-projector	Pulsar lag on <region>.chunks.tokenized	KEDA ScaledObject
Parser (T1 only)	Request rate / saturation	Standard Kubernetes HPA on CPU
Embedder (T1 only)	Request rate / saturation	Standard Kubernetes HPA on CPU
Parser/Embedder (T2 KubeRay)	Per-deployment ongoing requests	Ray Serve autoscaler

KEDA configuration

The Pulsar-lag-scaled services use KEDA ScaledObject resources. Defaults from the Helm charts:

keda:
  enabled: true
  pollingInterval: 5             # seconds; KEDA checks lag every 5s
  cooldownPeriod: 30             # seconds; wait before scaling down after queue empty
  msgBacklog: 100                # threshold; one replica per 100 messages backlog
  minReplicas: 1
  maxReplicas: 10

Tune msgBacklog per workload. Larger values give bigger batches per replica (lower per-record overhead, higher latency); smaller values give more concurrency (lower latency, higher overhead).

For the entity-extractor-dispatcher and other parser-fronted consumers, the practical scaling limit is the parser/embedder throughput — adding more dispatcher replicas past that point just creates HTTP queue contention.

Sizing the substrate

The Apache Pulsar, Nessie, Postgres, and Vespa instances scale independently of Moʻolelo's own services. Rough sizing:

Pulsar

For a single-region single-tenant deployment:

Records/day	Brokers	BookKeepers	Storage
< 1M	1	1	100 GB
1M – 10M	2	3	500 GB
10M – 100M	3	5	2 TB

Pulsar's tiered storage to MinIO/S3 lets you keep less hot storage on the brokers; configure offload-after-rollover via Pulsar's policies.

Nessie

A single Nessie instance handles ~1000 commits per second. For typical Moʻolelo workloads, one is enough up to ~10M records ingested per day. Run two with shared Postgres for HA.

Postgres for Nessie: small (1 vCPU / 2 GB) is sufficient up to a million Iceberg snapshots.

Vespa

Sizing depends heavily on chunk count, embedding dimension, and query rate. Rough rules:

Chunks indexed	Content nodes	RAM per node	Disk per node
< 1M	1	16 GB	100 GB SSD
1M – 10M	2	32 GB	500 GB SSD
10M – 100M	4–6	64 GB	1 TB SSD
> 100M	8+, distributed	64–128 GB	2 TB+ SSD

The dense embedding dominates RAM — bge-m3 produces 1024 floats per chunk × 4 bytes = 4 KB per chunk for the HNSW index alone. Plan accordingly.

For query latency, separate the container nodes (which serve queries) from the content nodes (which hold the index) and scale them independently. Vespa's documentation has full sizing guidance.

Postgres (Moʻolelo's own)

For Moʻolelo's connector_configs and connector_cursors tables, a small Postgres (1 vCPU / 2 GB) is sufficient for hundreds of tenants. The data is tiny — one row per (tenant, source) for configs, one row per (tenant, source, instance, resource, kind) for cursors.

Throughput math

A useful exercise when sizing:

1. Records per day ÷ 86400 = average records per second.
2. Multiply by ~5 for peak hours (most workloads aren't uniform).
3. Multiply by ~3 for chunks per record (depends on document size — code repositories produce more, simple wiki pages produce fewer).
4. Now you have peak chunks per second through the entity-extractor-dispatcher and downstream stages.

Example: 10M records/day → ~115 records/sec average → ~580 records/sec peak → ~1740 chunks/sec peak through the embedding pipeline.

At ~100 chunks/sec/replica with vLLM on a single GPU, that's ~17 GPUs to keep up at peak. With CPU sentence-transformers at ~15 chunks/sec/replica, it's ~120 CPU replicas — possible but probably better-served by moving to a GPU topology.

Scaling a multi-region deployment

Each region is an independent Moʻolelo deployment with its own substrate. Cross-region writes don't happen (by design — see Data residency), so each region scales on its own load profile.

The shared services across regions:

• The license-key validation public key (embedded in every binary; same across regions).
• The Moʻolelo container images (mirrored to your registry, same images everywhere).
• Optionally, a single source-system OAuth app that issues tokens redistributed across regions.

Everything else is region-local.

When to add a region

Add a region when:

• A new jurisdiction's compliance requires data residency on its own soil.
• A new geography's user base would suffer too much latency querying a far-away region.
• An existing region is past your operational comfort zone for a single substrate.

Adding a region is an install operation — same Helm charts, new namespace + storage backend + Postgres + Vespa cluster. The connector configurations are per-tenant per-region, so existing tenants can opt into the new region by inserting connector config rows there.

What's not auto-scaled

• The Iceberg sink. By design — exclusive Pulsar subscription means single-writer. If a single iceberg-sink can't keep up, partition the input topic and run one sink per partition.
• The connector services. Each (tenant, source) pair runs in one connector instance per replica; growing replicas increases parallelism across tenants but doesn't speed up a single backfill.
• Postgres, Pulsar, Nessie, Vespa themselves. These scale via their own operator/chart mechanisms; Moʻolelo's Helm charts don't manage them.

Seat gate (MAU)

Every query coordinator delegates cap decisions to the master webserver's POST /api/internal/seat-claim endpoint — there is one cluster-wide active_subjects ledger, on the master. The coordinator holds a 1-hour in-process cache of admit decisions, so repeated hits from the same subject don't round-trip per Search.

Sizing implications:

• Master RPS. Worst-case master RPS = (active_subjects × coordinator_replicas) / cache_ttl. A 100k-MAU deployment with 10 coordinator replicas and the default 1-hour TTL drives ~280 master RPS in the steady state. A single-pool webserver-on-Postgres handles that with margin.
• Coordinator memory. Each cache entry is ~80 bytes. 100k entries × 10 replicas = ~80 MB cluster-wide. Negligible relative to the coordinator's other memory needs.
• Cache cold-start after restart. A coordinator that just restarted has an empty cache; every active subject's first Search round-trips to the master. The master returns existing for in-window subjects, no cap impact — just one extra round-trip per subject per pod restart.
• Cache TTL trade-off. QUERY_COORDINATOR_SEATS_CACHE_TTL controls the TTL. Shortening it cuts the over-admission window in extreme races but increases master RPS. Lengthening it does the opposite. The default 1 hour stays well under any sane MAU window (default 30 days), so the master's ledger row gets refreshed many times before it can fall out of window.

Master unreachable. The gate is fail-closed: a coordinator that can't reach the master refuses new-subject Search calls with ErrSeatLimitExceeded (gRPC ResourceExhausted / HTTP 429 over MCP). Subjects whose admit decision is still cached (1-hour TTL) keep working — only first-time hits during the outage are denied. Bring the master back up before the cache empties (or restart the coordinator) and traffic recovers without operator intervention.

When seat traffic outgrows a single master process (typically beyond ~1000 master RPS) the upgrade path is a shared cache (Redis / Valkey / KeyDB) in front of the master's active_subjects reads — a drop-in optimisation, no architectural change required.