Obzly
Architecture

One image, five roles

The single container is the smallest shape of one design. Set SERVICE_MODE and the same image runs each role on its own, so each part scales and fails independently.

A Site

A load balancer sends traffic to stateless api replicas. Ingest goes to three JetStream brokers, then workers write Parquet to object storage. Queriers read object storage. Postgres and Valkey sit beside them.Loadbalancerapi × NUI · REST · OTLPquery coordinatorstatelessingestbroker × 3NATS JetStreamone per zone3 replicasworker × NParquet writescompaction · retentionquotas · uploadsObjectstorageZSTD Parquethour-sortedquerier × Nquery fragmentssegment cache on a volumefragmentsPostgresusers · catalogValkeyleases · histogram cache
Postgres, Valkey and object storage are yours. The Helm chart can run Valkey inside the release.
RoleDoesScales
apiUI, query API, OTLP/HTTP and gRPC ingest, query coordinatorStateless, any number behind a load balancer
workerWrites ingest to Parquet, compaction, retention, quotas, uploadsIngest and compaction shards are shared across workers
brokerNATS JetStream ingest bufferThree nodes, one per zone
querierRuns query fragments, with its own segment cacheStateless, add replicas for query throughput
control-planeOne login and UI in front of many Sites; organizations, SCIM, shared channels, Synthetics monitors, cloud connectionsOne per company, in front of every Site
obzly-probeSeparate image: runs Synthetics checks and cloud scans from your locationsProbe groups of any size; failover within a minute

Query path

A search goes to an api replica, which uses Postgres file statistics to skip files, then splits the rest into fragments sent to queriers by rendezvous hashing. Bloom filters skip segments that lack a searched ID.Searchorder 8f3a… · 7 daysapi coordinatorplan from catalog statsskip files that can't matchnewest-first windowsscan budget checkrendezvoushashingquerier Awarm cachequerier Bwarm cachequerier Cretried on failureSegments · newest firstreadbloom filter: skipgreyed = skipped without reading
Every segment keeps going to the same executor, so caches stay warm. Rows stream back as each window is read, and the search stops once the page is full. Result counts come from the histogram, whose finished hours are cached in Valkey.

Multi-site

One control plane holds identities, organizations and access grants, and proxies to three Sites. Each Site keeps its own data.PeopleSSO · SCIM · MCPcontrol planeone login and UI · organizations · teamsper-Site access grants · plansbootstrap admin: superadminper-user Site tokenSite · northapi · worker · brokerits own bucketSite · southapi · worker · brokerits own bucketSite · stagingsingle containerlocal disk
Each Site is a normal installation with a couple of extra settings. Its query and admin API is reached through the control plane rather than exposed directly. A Site's local admin stays as a break-glass account.

Synthetics & cloud

A probe group pulls monitors and cloud connections from the control plane, checks your endpoints and scans cloud APIs read-only, and sends results as OTLP to a Site project, where the inventory is built. The control plane reads results back through the Site.control planemonitors · probe groupscloud connections · channelsstores no resultsconfig · pull every 180 sprobe group · edge-northpod-1pod-2pod-3one live instance per monitorfailover within a minuteYour endpointsHTTP · TLS · TCP · DNS · ICMPCloud APIs, read-onlyAWS 111122223333 · Azureresults · OTLPSite · northproject: syntheticsalert rules per monitorinventory every 5 mincloud VMs join agent hostsread throughthe proxy
Probes push results straight to a Site, so the control plane never carries ingest. Cloud credentials stay with the probe group: its own identity, a role with an external ID, or a short-lived token the control plane signs.

Scaling & HA

Durable ingest

A batch is acknowledged only after the JetStream buffer has stored it on three replicas. Rows are searchable before they're written to Parquet.

Backpressure

When the buffer is full, clients get 429 or RESOURCE_EXHAUSTED and OTel Collectors retry.

One owner per shard

Leases give each ingest and compaction shard exactly one worker. Rolling updates drain readiness first.

Storage layout

Parquet is compacted in levels into hour-sorted files of one day each. A partitioned Postgres catalog keeps per-file statistics.

Cache warmup

Queriers keep their cache on a volume, pre-fetch each segment they own as it's uploaded, and warm up on start.

Self-monitoring

Every role serves Prometheus metrics on an internal port and traces its own requests into a system project.

Inventory per Site

A worker on each Site rebuilds the inventory every 5 minutes from data already stored. The control plane keeps none of it and merges the Sites in your browser.

Alerts survive the control plane

Sites fetch the shared channels and email relay every 5 minutes, sealed to their own key, and keep the last copy if the control plane is unreachable.