Backend services

Zy now supplies backend tools through its Go host. The interpreter remains a tree-walk engine. These features provide a practical foundation, not proof of production capacity or a community package ecosystem.

Prepared execution

core.Prepare parses an entry and all imports once. PreparedProgram.Run creates fresh module state from the immutable snapshot. Requests cannot share mutable globals and no longer scan/read/parse imports per request. Restart to load code changes. Top-level initialization still executes for each request; keep side effects inside handlers.

Local Windows/amd64 benchmark on an AMD Ryzen 5 7535HS, Go 1.26:

This microbenchmark measures module initialization, not HTTP throughput. Reproduce with go test ./core -run '^$' -bench BenchmarkImportedModuleExecution -benchmem. The backend’s tools/load.py separately measures bounded HTTP workloads.

Accounts, sessions, and roles

AUTH_ENABLED=true is the default. ENABLE_REGISTRATION=false disables public signup. Provision an account with zy user:create --email=person@example.com --role=admin --password-stdin; supply the password through stdin, not arguments. Emails normalize to lowercase and have a unique database index. Passwords require 12..72 UTF-8 bytes and use bcrypt cost 12. Public registration creates members regardless of any submitted role.

Access tokens live 15 minutes and refresh sessions 30 days from issuance. Credentials are random opaque tokens; only SHA-256 token hashes are persisted. Refresh consumption uses an atomic deletion: concurrent attempts cannot both succeed. The old access token is revoked too. Password changes increment an account version, invalidating existing access/refresh sessions immediately. Disabled accounts cannot authenticate. Expiry is checked during authentication; the TTL index only cleans up records.

Auth bodies are limited to 16 KiB, expensive password work to four concurrent operations, and auth attempts to 10/minute/direct client IP. User access tokens and the existing automation service key both authenticate /api requests. The service key acts as admin. Require roles before handler execution with:

http.route("POST", "/api/v1/projects", create_project, {"roles": ["admin"]})

Role restrictions require an /api/ path. Roles are admin and member. request.user exposes id/email/role/kind, never password/session data. auth.has_role(request, role) is also available. Roles do not impose resource ownership: tenant/user boundaries must be defined by the application. The sample backend is a shared workspace, with member reads and admin mutations.

No email verification, MFA, cookie sessions, or password-reset mail is supplied. Registered email is not treated as verified. Use HTTPS and an appropriate secure client token store. Database errors are secret-safe 503 responses.

Queries, aggregation, and transactions

mongo.find(collection, filter, limit, offset) and mongo.count(collection, filter) support field equality and $eq/$ne/$gt/$gte/$lt/$lte/$in/$nin/$exists plus $and/$or. id converts to ObjectID. Filter depth is capped at 16, logical branches at 32, and membership lists at 100. Arbitrary operators, regex evaluation, and server-side JavaScript are rejected. Find limits remain 1..100 and offsets 0..100000. Signed 64-bit database integers preserve precision.

mongo.aggregate(collection, stages) accepts 1..20 stages: $match, $sort, simple 0/1 $project, $skip, $limit, and $group with $sum/$avg/$min/$max. Group IDs are null or field references; operands are integers or field references. The runtime appends a 100-result limit, disables disk spill, and excludes writes, joins, and arbitrary expressions. Aggregation retains BSON/Extended JSON result shape, including grouped _id values.

let results = mongo.transaction([
    {"op": "update", "collection": "projects", "id": project_id, "document": {"status": "completed"}},
    {"op": "insert", "collection": "events", "document": {"event": "completed", "project_id": project_id}}
])

Transactions contain 1..25 insert/get/update/delete operations, with an eight-second deadline. They use the driver’s transaction mechanism and require a replica set/sharded deployment such as Atlas. There is no non-atomic fallback. Batches contain only database work, so retries cannot repeat arbitrary Zy callbacks. Most database calls retain a five-second deadline. Dependency failures return 503 and expired handlers 504. Validate application data before database calls.

Migrations

zy migrate executes migrations/*.zy in lexical order. A MongoDB lease excludes other migration runners for three minutes; the run deadline is two minutes. The ledger stores names, source/import checksums, and timestamps. Changed applied files or dependencies are rejected. mongo.create_index(collection, fields, unique) creates bounded indexes with direction 1/-1.

Migrations are forward-only. Use idempotent operations: a crash between a write and its ledger record can require inspection/retry. Data batches can use transactions, but index DDL is not included in a transaction. Failed migrations are not silently marked applied. Runtime collections use _zy_ prefixes, which ordinary MongoDB language APIs cannot access.

Durable jobs

jobs.enqueue(name, payload) persists a bounded MongoDB job and returns its ID. jobs.status(id) returns safe name/state/attempt metadata or null. Run zy queue:work, which loads worker.zy with a fn handle(job) entry point. The job provides id, name, payload, and attempt.

Claims are atomic, leases last 60 seconds, and handlers have a 30-second deadline. Owner tokens prevent stale acknowledgements. Failed jobs retry with exponential delay, up to five attempts; exhausted jobs become failed. Expired leases can be reclaimed. Delivery is at least once: handlers must be idempotent. Completed/failed records expire after 30 days. --once handles at most one available job, does not wait for delayed jobs, and exits nonzero on handler failure. Workers prepare source once and require restart for code changes. Shutdown interrupts the handler and persists its retry state before exit.

Cache, logs, metrics, and traces

cache.set(key, value, ttl_seconds), cache.get(key), and cache.delete(key) provide a process-local JSON cache. Keys: 1..128 bytes; values: 64 KiB; TTL: 1..3600 seconds; capacity: 1024 entries and 16 MiB. Expired and least-recently-used entries are evicted. Reads return detached values; misses return null. This is not a distributed consistency layer; define invalidation in application code.

log.info/warn/error(event, fields) emit structured, trace-linked events; field objects are capped at 16 KiB. Avoid deliberately logging secrets. Automatic request logs exclude headers, bodies, and connection strings. Valid W3C traceparent IDs continue with fresh server span IDs returned in responses. This is trace correlation; no OpenTelemetry exporter is installed.

Admin/service credentials can scrape /ops/metrics for Prometheus request/error counters and latency histograms. Metrics are per process and use bounded, label-free series. Health probes remain public. MAX_CONCURRENT_REQUESTS defaults to 128 (range 1..4096); admission overflow returns 503. Caches, metrics, concurrency gates, and IP/auth rate limits are per process. Forwarded IP headers are not trusted; enforce client limits at the proxy when traffic is proxied.

Verification and deployment

Unit tests cover prepared-source isolation, auth expiry/revocation/refresh races, roles, query rejection, cache bounds, tracing, HTTP limits, and graceful draining. Set ZY_TEST_MONGODB_URI to run TestMongoIntegration, which creates and drops an isolated random zy_test_* database. It verifies real filtering, aggregation, transaction rollback/commit, persistent identity, job leases/retries, migrations, and Zy worker execution. Never point custom test cleanup code at an application DB.

The separate backend application includes a bounded load probe, Docker/Compose configuration, and Linux API/worker supervisor and HTTPS proxy templates. It uses the installed Zy executable or an independently built runtime image, without requiring a sibling checkout or rebuilding interpreter source. Deploy immutable sources, inject secrets, supervise API and worker separately, and test backup restoration and secret rotation. The package ecosystem, independent security review, sustained load testing, and operational experience remain work beyond these implementations.