` is a `params`-block
field of the subscribing locus (for a String-keyed topic, a
`String` field — e.g. `where key == self.name`)
Higher-shape expressions (`self.a + self.b`, method calls in the
filter, cross-locus reads) are reserved for later. The
restriction keeps the static check simple ("EXPR is a let-
bindable expression with no side effects, types to the topic's
key type") and avoids surprising semantics at registration time.
**Key stability — captured by value at register.**
A routing-key subscription captures its key value at the locus's
instantiation (or restart). Subsequent mutations to fields the
filter expression references do **not** change which messages the
handler receives. If dynamic re-keying is needed, dissolve and
re-instantiate the locus. The alternative — re-evaluating the
filter on every dispatch — would break the bus's "register once,
dispatch many" cost model and introduce ordering complexity
against concurrent `self` mutation; the capture-by-value rule is
the right default, and a `re-subscribe` API is a follow-up if a
workload demands it.
**`on_unmatched: V` — policy when no subscriber's key matches.**
A keyed publish may find zero subscribers whose `where key == X`
filter matches the message's key. Topic-level config picks the
behavior; default is `swallow` (matches today's no-subscriber
semantics on unkeyed topics).
| `on_unmatched:` | Behavior |
|---|---|
| `swallow` *(default)* | Drop the message silently. Diag visible only with `LOTUS_BUS_LOG_UNMATCHED=1` env var (per-publish stderr line citing subject + key + subscriber counts). |
| `fail` | Publish becomes a fallible expression. Caller must attach an `or` disposition: `K <- value or raise` panics via `lotus_root_panic` with a `BusUnmatchedKey` marker; `K <- value or discard` silently swallows on no-match. The err-payload-carrying dispositions (`or handler(err)` / `or fail `) are reserved for v0.2 — they require synthesizing `BusUnmatchedKey { subject: String, key_lo: Int, key_hi: Int }` as a stdlib type, which is a small follow-up. `or ` is permanently rejected: Send produces no value, nothing to substitute. |
| `fallback` | A catch-unmatched subscriber on the subject — `subscribe T as h where key == _` — receives the message. At least one such subscriber is required; cross-module resolve-time check rejects the topic otherwise. The `_` sentinel is legal only on `fallback` topics. |
Static checks at typecheck:
1. `keyed_by FIELD` — FIELD must be a declared field of the
topic's payload type; FIELD's type must be one of the table
above; the topic must not also declare a `keyed_by` via a
parent topic with a different field.
2. `where key == EXPR` — EXPR's type must match the topic's
keyed-by field type after width inference.
3. `where key == EXPR` is forbidden on topics without
`keyed_by`; rejecting prevents silent-no-match bugs from
typo'd filters.
4. `where key == _` is forbidden except on topics with
`on_unmatched: fallback`.
5. `fail` topics: every `Topic <- value` send site must carry an
`or` disposition clause (`or raise` / `or discard` at v0.1
of the impl; `or handler(err)` / `or fail ` reserved for
v0.2). The `or` clause attaches to the Send statement, not
the value expression — the parser strips it off the value's
`Expr::Or` wrapping into `Stmt::Send.or_disposition`.
Conversely, an `or` clause on a Send to a non-`fail` topic
is rejected.
6. `fallback` topics: at least one program-wide `where key == _`
subscriber must exist; checked at resolve after import
merging.
Routing keys are orthogonal to topic hierarchy (Phase 2): a
parent's `keyed_by` and `on_unmatched` are inherited by children
that don't override; children may override either independently.
A child topic re-declaring `keyed_by` must agree with the
parent's key type (subjects derived from a parent's wire prefix
share the parent's key shape — anything else makes dispatch
ambiguous on the wire).
**Runtime — `lotus_bus_entry_t` extension.**
The bus router's subscriber-entry struct
(`crates/hale-codegen/runtime/lotus_arena.c`, around line 4034)
gains a tri-state filter and a u128 key value:
```c
typedef struct {
/* ...existing fields: subject, self_ptr, handler, etc... */
uint8_t key_filter_kind; /* 0 = no filter (receive-all)
* 1 = specific key
* 2 = catch-unmatched (`_`) */
uint64_t key_lo; /* i64 key, or low half of i128 */
uint64_t key_hi; /* 0 for i64 / narrower types */
} lotus_bus_entry_t;
```
Dispatch (`lotus_bus_local_dispatch_keyed`):
```c
int matched_specific = 0;
for (entry in g_bus_entries with matching subject):
if (entry.key_filter_kind == 1
&& entry.key_lo == msg.key_lo
&& entry.key_hi == msg.key_hi) {
fire(entry);
matched_specific = 1;
} else if (entry.key_filter_kind == 0) {
fire(entry); /* unkeyed receive-all */
}
if (!matched_specific) {
for (entry in g_bus_entries with matching subject):
if (entry.key_filter_kind == 2) fire(entry);
}
```
Walk cost is O(N_subscribers_on_subject) for the specific pass,
with a second pass only when there's no specific match (fallback
case). For workloads with thousands of keyed subscribers per
subject, a per-`(subject, key_lo, key_hi)` open-addressing index
can be added later — YAGNI until a workload demands.
Two new runtime symbols:
```c
void lotus_bus_register_keyed(
const char *subject, void *self,
lotus_handler_fn handler,
/* ...existing... */,
uint8_t key_filter_kind,
uint64_t key_lo,
uint64_t key_hi);
int lotus_bus_dispatch_keyed( /* returns match count;
* `fail` topics check this */
lotus_bus_queue_t *queue,
const char *subject,
const void *payload, size_t payload_size,
uint64_t key_lo, uint64_t key_hi,
lotus_serialize_fn serialize_fn);
```
Existing `lotus_bus_register` / `lotus_bus_dispatch` stay as
the unkeyed entry points (compat for unkeyed topics).
**Backward compatibility.** Topics without `keyed_by` and
subscribers without `where key ==` behave exactly as today —
the new fields default to `key_filter_kind = 0` (receive-all)
and the dispatch dispatches uniformly. Existing programs need
no source change to keep working; new programs opt in
per-topic.
**v0.2 — err-payload Send dispositions.**
On `on_unmatched: fail` topics, all four `or` disposition shapes
are now supported:
- `or raise` — no-match panics via `lotus_root_panic` (v0.1
shape; unchanged).
- `or discard` — no-match silently swallowed (v0.1 shape;
unchanged).
- `or ` — evaluates `` for side effects on no-match,
with `err: BusUnmatchedKey` in scope. The expression's value
is discarded (Send is statement-level). Canonical use:
`or log_unmatched(err)` — a free fn that takes the err
payload and logs / metrics / etc.
- `or fail ` — only legal inside an enclosing
`fallible(E)` fn. On no-match, evaluates `` (with
`err: BusUnmatchedKey` in scope), stores it into the
enclosing fn's err slot, and diverts to the fn's err-exit
path. Symmetric to `or fail` on fallible-method calls.
`BusUnmatchedKey` is a synthesized stdlib type, injected into
scope when any topic declares `on_unmatched: fail`. Layout:
```hale
type BusUnmatchedKey {
subject: String; // wire subject of the failing publish
key_lo: Int; // low 64 bits of the unmatched key
key_hi: Int; // high 64 bits (0 for i64 keys)
}
```
Codegen allocates a fresh `BusUnmatchedKey` in the current
arena on the no-match branch and binds it as `err` for the
disposition expression's lowering. Mirror of the existing
`KeyError` / `IndexError` / `IoError` synthesis pattern.
**Out of scope at v0.1 (explicit non-goals):**
- Multi-field / tuple `keyed_by` (`keyed_by (sym_id, side)`).
Apps that need compound keys pack into a `Decimal` field in
user code. Eligible for a v0.2 sugar once a workload's
ergonomics surface the friction.
- Wildcard key sets (`where key in [1, 2, 3]`).
- Range predicates (`where key > 100`). Equality match is the
workload-driven sweet spot; broader predicates would defeat
the O(1)-dispatch cost model.
- String routing keys. String-equality match defeats the
perf goal; only int-shaped scalars at v0.1.
- Method blocks on enums to make enum-typed routing keys
ergonomic. Treated as a separable language feature; users at
v0.1 use bare Int / Decimal or existing no-payload enums.
- Per-publish key override (publishing with a key value that
isn't derived from the payload field). Tied to the payload's
identity so wire-format consumers see a consistent key.
- Cross-process keyed dispatch (over the remote-fanout path).
v0.1 ships keyed dispatch for the intra-process bus only;
remote subscribers still fanout per-subject and filter in
their own bus router after deserialize.
- `or ` on Send for SUCCESS-value substitution.
Send produces no value to substitute. v0.2's `or `
disposition on fail topics is a side-effect handler call
(evaluates `` with `err` in scope, discards the value),
not a value substitution.
## Placement block (F.31)
The `placement { }` block on `main locus` controls per-locus
thread placement, parallel to `bindings { }` for bus topology.
Placement is a deployment seam — same library, different
placement entries, different binary behavior. See
`spec/decisions.md` § F.31 for the intrinsic-vs-
deployment axis the block sits on, and `spec/runtime.md` §
"Placement classes" for the runtime semantics.
### Syntax
```hale
main locus App {
params {
gateway_a: Gateway = Gateway { venue: "venue-a" };
gateway_b: Gateway = Gateway { venue: "venue-b" };
metrics: MetricsServer = MetricsServer { port: 9100 };
ui: Renderer = Renderer { };
}
placement {
gateway_a: pinned(core = 1);
gateway_b: pinned(core = 2);
metrics: cooperative(pool = io);
ui: cooperative(pool = render);
// unspecified main-locus params → cooperative(pool = main)
}
}
```
### Type-check rules
> These placement/bus rules are part of the broader compile-time
> verification surface; `spec/verification.md` is the canonical
> catalog of every static check (with severities and enforcing passes).
1. **`placement { }` is `main locus` only.** Any other locus
declaring `placement { }` is a parse error (same shape as
`bindings { }`).
2. **Keys reference main-locus `params` field names.** The key
on the left of each `placement_entry` must match a declared
`params` field on the enclosing `main locus`. Unknown
field name → typecheck error pointing at the params block.
3. **Field values are locus types.** A placement entry on a
non-locus field (`port: Int`, `host: String`) is a typecheck
error — placement applies only to locus instances.
4. **At most one placement entry per field.** Duplicate keys
are a parse error.
5. **Pool names use snake_case Idents.** The set of pool names
is inferred from `cooperative(pool = X)` references across
all placement entries in the bundle. Pool `main` is always
available; it refers to the program's main OS thread.
6. **Locus-pinning compatibility.** A locus placed `pinned` is
subject to the existing pinned-class restrictions (no
`accept(c: Child)` accept-method, no `closure` declarations
in v1). These restrictions move from the locus declaration
site (pre-F.31) to the placement site: the typechecker walks
each placement entry and applies the relevant restriction to
the named locus type. A locus that uses neither feature can
be placed either cooperative or pinned at the deployment's
discretion.
7. **Dead bus receiver (error).** A locus that declares
`bus { subscribe ... }`, is placed `cooperative(pool = X)` with
`X != main` (and not `where async_io`), **and** whose `run()`
makes a known-blocking stdlib call is rejected. A cooperative
locus receives cells only while its pool thread is free to run
the dispatch (the cross-process transport reader dispatches
into the handler set; the in-process cooperative queue is
drained at yield points). A blocking call monopolizes the pool
thread, so the dispatch never runs and the subscriber's handlers
never fire. **Placement alone is not the condition** — an
event-driven subscriber that yields (handlers plus a
`time::sleep` loop, or `where async_io`, which parks) receives
fine and is *not* flagged; only the blocking-and-subscribing
combination is rejected. A subscription to a topic the locus
also *publishes* is also spared (an intra-locus
self-publish→subscribe is devirtualized to a direct call). Fix:
`pinned` (own thread + a mailbox drained at sleep/yield) or keep
`run()` non-blocking. (Corrected 2026-06-03: an earlier form
rejected on placement alone and over-fired on event-driven
non-main cooperative subscribers, which receive reliably.)
8. **Blocking syscall on a cooperative pool (warning).** A locus
placed `cooperative(pool = X)` *without* `where async_io`
whose `run()` calls a known-blocking stdlib op (tcp/tls recv,
accept, `process::run`/`wait`) but is *not* a dead receiver
(rule 7) gets a **warning** (not an error — hale's only
non-fatal diagnostic). A blocking call holds the pool's OS
thread for its whole duration, stalling every other locus
scheduled on that pool and the pool's bus drain. The fix is
`pinned` (its own thread — the prescribed shape for blocking
I/O) or `cooperative(pool = X) where async_io` (parks on I/O
readiness). It is a warning rather than an error because a
single-purpose blocking server with nothing co-scheduled is
legitimate. Detection is **interprocedural**: it follows the call
graph, so a `run()` that blocks indirectly — through a helper fn
or a `self.method` it calls (transitively) — is flagged too, with
the diagnostic naming the offending call. It remains best-effort
at the edges (blocking via a *handle* method like `stream.recv`,
or across a cross-locus `self.field.method()` hop, isn't traced).
Note rule 7 (the dead-receiver *error*) stays **direct-call-only**
— it is not widened onto indirect paths, so the higher-stakes
diagnostic keeps its precision.
9. **Orphan bus topic (warning).** In a closed-world program (a
`main` locus present), a bus subject — a declared `topic` or a
literal string — wired to only one end is flagged: *published with
no subscriber* (the cells go nowhere) or *subscribed with no
publisher* (the handler can't fire), and a declared topic touched
by neither is *dead wiring*. Suppressed when the other end is
plausibly external: a **transport binding** (`bindings { T: ... }`
implies a cross-process peer), a **wildcard** subscriber/publisher
covering the subject (`log.**` covers `log.app`), a **cross-seed**
reference (`alias::Foo` — the other seed owns the other half), or
the same locus being both publisher and subscriber. The closed-
world gate is why this is skipped for library seeds (no `main`):
their consumers are downstream, out of the bundle. (GH #18 #4.)
10. **Bus cycles.** An edge `S →(L) D` exists when locus `L` subscribes
subject `S` with a handler that sends to subject `D`. A cycle in
this graph is a publish→subscribe→publish loop, and the dispatch
model splits it two ways:
- A **cross-locus** cycle (edges from ≥2 loci) hops between loci
through the cooperative *queue* (drained at yield) — it spins
the queue / livelocks → **warning**.
- An **intra-locus** cycle (every edge in one locus) is
intra-locus self-dispatch, which is devirtualized to a direct
synchronous call (rule 7), so it recurses on one thread without
bound → stack overflow → **error**. To keep the error precise,
only **unconditional** sends form intra-locus edges: a
self-republish guarded by an `if`/`match`/loop is a terminating
state machine, not unbounded recursion, and is not flagged.
(GH #18 #4.)
11. **Bus backpressure (warning).** A locus that publishes to the bus
inside an **unbounded** `while true` loop carrying no flow-control
or exit point — no cooperative `yield`, no `time::sleep`/`tick`
throttle, no input-pacing blocking `recv`, no `break`/`return` —
has no backpressure: it posts cells faster than any subscriber can
drain, so the queue and the payload arena grow without bound. A
full producer-vs-consumer rate analysis is undecidable, so this is
a deliberately narrow structural heuristic (warning): only literal
`while true` loops are considered (bounded `for`/`while cond`
loops never are), and any flow-control point anywhere in the loop
body clears it. (GH #18 #4.)
12. **Bus subject type-mismatch (error).** Every publish/subscribe
site addressing the same **literal** subject string must declare
the same `of type` payload — otherwise a subscriber decodes the
publisher's bytes as the wrong type at runtime. A declared `topic`
is already unified by its declaration (and `of type` is forbidden
on topic refs), so this closes the literal-subject gap.
Grouping is by *exact* subject string, which excludes wildcards
(`log.**` and `log.app` are different strings, never
cross-compared). The fix is to declare a `topic` (one payload
type, fixed in one place) or align the `of type` annotations.
(GH #18 #4.)
13. **Empty / degenerate `pinned(cores = …)` (error).** A cpuset
affinity spec that selects no cores is rejected statically: an
exclusive range whose upper bound is `≤` its lower (`4..4`,
`8..4`), an inclusive range that runs backwards (`8..=4`), or a
set with a duplicated element (`{2, 4, 2}`). Bounds/elements are
integer literals — placement is closed-world — so the selected
core list is known at compile time and a spec that reduces to
"no cores" (or a redundant one) is an authoring error, not a
deploy-box question. Whether the selected cores *exist* on the
target stays best-effort at runtime (out-of-range indices are
skipped, exactly as `pinned(core = N)` degrades on a smaller
machine). Linux-only affinity; a no-op on other hosts.
(Topology Phase 1a, 2026-07-04.)
14. **`topology { }` consistency + `pinned(node/l3)` resolution
(error).** The declare-only `topology { }` block is validated
statically: NUMA `node` ids must be unique; L3-domain names
must be globally unique (they're referenced by `pinned(l3 =
name)` without node qualification); each `cores` spec must be
well-formed (rule 13); a core may belong to at most one L3
domain (overlap is ambiguous affinity); and a domain core may
not overlap a `reserve`d range (reserved cores are held back
for the OS / main). A `pinned(node = N)` / `pinned(l3 = name)`
placement entry must reference a domain the block declares —
using either with no `topology { }` block, or naming an
undeclared node/domain, is an error. Resolution is
closed-world (ids and domain cores are literals), so the
selected core set is known at compile time. Whether those
cores exist on the deploy box stays best-effort at runtime.
A `pinned(node/l3)` locus gets **thread + memory
co-location**: its thread is affinity-masked to the domain's
cores (Phase 1a cpuset path) *and* its arena — including
method-scratch sub-regions — is `mbind`-bound to the node, so
its working set lives on the node its thread runs on.
Node binding is a raw `mbind` syscall (no libnuma dependency),
Linux-only and best-effort (falls back to first-touch when
the node can't be honored); non-node arenas are unchanged.
(Topology Phase 1b, 2026-07-05.)
15. **`replicas = K` (error on `K < 1`; pinned-only).** A
`pinned(..., replicas = K)` entry fans the field into K
single-threaded instances — replica `i` pinned to one core of
the affinity set (round-robin), each on its own OS thread.
`K` must be `>= 1` (`0` / negative is rejected). `replicas` is
valid only on `pinned`: K cooperative loci on one pool would
share a single thread (not parallel), so `cooperative(...,
replicas = K)` is rejected at parse with guidance toward the
pinned form. The point is that parallelism comes from *more
single-threaded units*, never a multi-worker pool — each
replica is its own single consumer, so the lock-free rings, bus
devirtualization, and single-threaded-method guarantee all hold.
Replicas compose with `node`/`l3` (each replica's arena binds to
the target node) and are non-addressable (no `field[i]` surface;
they are bus-subscribing or run-loop workers). All K are joined
and dissolved at parent teardown. (Topology Phase 1c, 2026-07-05.)
### Single-threaded-method invariant
A locus's methods may be invoked only on the OS thread that
owns the locus's placement's pool. This is enforced at
typecheck via a static call-graph walk starting from each
top-level placement entry:
1. Seed each placement entry with its pool: `gateway_a`
→ pinned (own thread), `metrics` → cooperative pool `io`,
etc.
2. For each method call expression `recv.foo(args)`, determine
the receiver's pool from the receiver's static type and the
surrounding pool context.
3. Cross-pool direct method calls are rejected with a focused
diagnostic naming both pools and pointing at the
`placement { }` entries that picked them.
4. Bus publishes (`Topic <- payload;` / `"subj" <- payload;`)
are unrestricted — they route through the substrate's
cross-thread dispatch machinery (the existing m28b
condvar+memcpy mailbox path generalized to cooperative
pools).
This invariant is the substrate enforcement that makes M:N
safe. Without it, multi-pool deployments would silently race
on locus arenas (which are unsynchronized bump allocators by
design).
### Nested instantiation
Loci instantiated nested in another locus's body (`birth` /
`run` / lifecycle methods, let-bound children, or `params`
fields of non-`main` loci) inherit their containing tower's
pool by construction:
```hale
main locus App {
params {
gw: Gateway = Gateway { };
}
placement {
gw: pinned(core = 1);
}
}
locus Gateway {
params {
// Cache instantiated nested in Gateway's params —
// inherits Gateway's pool (the pinned thread). No
// placement entry is permitted on `cache` at any
// main locus.
cache: Cache = Cache { };
}
}
```
Placement entries on nested fields are a typecheck error —
placement is a top-level main-locus surface only. To run a
nested locus on a different pool, hoist it to a main-locus
sibling.
### Default placement
Main-locus `params` fields with no explicit `placement { }`
entry default to `cooperative(pool = main)`. The pre-F.31
shape (`: schedule cooperative` on every cooperative locus
declaration, with a single shared main thread) is exactly the
behavior a program without any `placement { }` block
receives. Existing programs that don't declare placement see
no observable change.
### Pool inference
The set of cooperative pools is the union of `X` values
appearing in `cooperative(pool = X)` references across all
placement entries, plus the implicit `main` pool. The runtime
spawns one OS worker thread per inferred pool name beyond
`main`. No `threads { }` declaration block at v1 — pools are
named purely by use site, and the runtime materializes them
on demand at startup.
## Bus subscription dispatch
A `bus { subscribe SUBJECT as HANDLER of type T; }` declaration
wires:
1. The runtime registers HANDLER as the receiver for SUBJECT
on the bound transport.
2. Inbound messages on SUBJECT are decoded as `T`, then
HANDLER(payload) is invoked.
3. HANDLER runs in the locus's scheduler context. It may
call `publish(SUBJECT, msg)` to emit responses (subject
to `bus { publish ... ; }` declarations).
4. HANDLER yields naturally on completion; scheduler returns
to other loci.
If HANDLER panics:
- The current message is dropped.
- `on_failure(self, BusHandlerFailure { subject, payload, err })`
invoked on the parent if any.
- The subscription itself is *not* removed; future messages
continue to dispatch.
### Payload type — primitives + nested structs + String
The wire format supports primitives (`Int`, `Float`, `Bool`,
`Decimal`, `Duration`, `Time`, `String`), `Bytes`, and
**nested user struct types** (`type T { ... }`) recursively
composed. A bus payload may carry a struct whose fields are
primitives, Strings, Bytes, or other nested structs, at any
depth. Serialize walks the field tree in declaration order;
deserialize allocates each nested struct in the lazy global
payload arena and recurses. Arrays, tuples, and enums as bus
payload fields are post-v1 polish.
## Closure-test evaluation
For each `closure NAME { LEFT ~~ RIGHT within TOL; epoch
... }`:
1. At each declared epoch boundary, runtime evaluates LEFT
and RIGHT in the locus's scope.
2. Computes `|LEFT - RIGHT|`.
3. If `<= TOL`: closure passes silently.
4. If `> TOL`: flips the locus's "exploded" flag; emits a
typed `ClosureViolation` event.
Epoch boundaries:
- `epoch dissolve` (default): fires once, as part of dissolve
sequence.
- `epoch tick`: fires on each runtime tick (configurable
cadence).
- `epoch duration(d)`: fires every `d` of monotonic time.
- `epoch birth`: fires once, after birth completes.
- `epoch explicit`: fires only when user code calls
`epoch_advance(NAME)`.
- `epoch inline` (F.27, v1.x-VIOLATE): never fires
automatically; fires only when user code executes
`violate NAME;`. The closure body has no assertion (no LEFT /
RIGHT / TOL to evaluate). See "Inline closure violation"
below.
### Per-epoch field reset (F.34, v1.x-WINDOWED)
A closure paired with `epoch duration(N)` may declare
`resets_per_epoch(field1, field2, ...);`. The named locus fields
are zeroed by the runtime **after** the assertion fires at each
duration boundary. Ordering matters: the assertion sees the
window's accumulated value; the reset prepares the next window.
```hale
closure low_corrupt_rate {
self.corrupt_per_min ~~ 0 within 10;
epoch duration(1m);
resets_per_epoch(corrupt_per_min);
}
```
Restrictions enforced at typecheck:
- The closure MUST declare `epoch duration(N)`. The clause is
rejected on `tick` / `birth` / `dissolve` / `inline` /
`explicit` — other epochs either don't recur or recur too
fast for a rate-budget framing.
- Each named field MUST be declared on the enclosing locus and
MUST have numeric type (`Int`, `Uint`, `Float`, `Decimal`).
Booleans, strings, and structs are rejected — zero is not a
meaningful reset value for them.
User code increments / decrements the field as the window
accumulates. The closure assertion is the structural contract
(rate bounded by a per-window budget); `resets_per_epoch`
keeps the substrate honest about which window the counter
belongs to without forcing the user to maintain a `last_reset_at`
field or a parallel pre-fire hook.
## Inline closure violation
(F.27, v1.x-VIOLATE.) Inline closures provide a pull-only
structural-failure channel for locus method bodies that catch a
value error and want to escalate it. The declaration carries no
assertion; the optional `captures:` clause names locus fields
whose values are snapshotted into the ClosureViolation payload
at fire time.
```hale
closure fatal_io { captures: last_error; epoch inline; }
```
`violate NAME;` (optionally `violate NAME with EXPR;`) fires the
closure synchronously at the call site:
1. Runtime synthesizes a `ClosureViolation` value carrying:
- `locus`, `closure` — string names of the failing locus and
the inline closure (always present).
- The captured fields named in the closure's `captures:`
clause are NOT materialized on the `ClosureViolation`
struct, which has a fixed shape. The access pattern for
captured state is to read the frozen child through the
child handle in `on_failure(c, err)` — see "Reading the
audit state" below.
- If `with EXPR` was given, EXPR is evaluated for side
effects (and to detect typecheck errors on the payload
type) but no `payload` field is materialized on the
`ClosureViolation`.
- The assertion-shape fields (`left`, `right`, `tolerance`,
`diff`) are NOT populated for inline violations.
2. The locus's exploded flag is set (same as the auto-epoch
path; downstream observers can't tell from the flag whether
the fire was auto-epoch or inline).
3. The synthetic `__drain_requested` field on the locus is
set. Readable from user code as `self.draining`.
4. The parent's `on_failure(child, ClosureViolation { ... })`
handler runs — same routing as for auto-epoch closure
violations.
### Reading the audit state
The portable access pattern in `on_failure(c, err)` is to read
the child's frozen locus state through the child handle:
```hale
on_failure(c: Child, err: ClosureViolation) {
log::error(err.closure, " ", c.last_error, " fd=", c.conn_fd);
}
```
`violate` is divergent — the method body's remaining statements
do not execute, so the child's locus state is frozen at the
violate moment. `c.last_error` reads exactly the value the
violate site observed.
The `ClosureViolation` value carries only `err.locus` and
`err.closure`; it does not materialize the captured fields.
Source that reads `err.last_error` will typecheck
(`ClosureViolation` admits unknown fields permissively at
field-access time) but will fail to link / run — read captured
state through the child handle (`c.last_error`) instead.
The `violate` statement is divergent: the typechecker treats it
as `Never`, the same as `fail` in fallible fn bodies and
`bubble` in `on_failure`. No statement after `violate` in the
same block is reachable; the typechecker does not require a
trailing `return` on a `violate` branch.
### `birth_check` synthesis hook
(F.27 v2, 2026-05-20.) A declarative form for construction-time
invariants:
```hale
locus L {
params { x: Int = 0; }
closure invariant_broken { captures: x; epoch inline; }
birth() { /* set up state */ }
birth_check { self.x < 0 } -> violate invariant_broken;
}
```
After `birth()` body completes and birth-epoch closures have
fired, each declared `birth_check` clause's `cond` expression is
evaluated. A `true` result fires the named closure with the
locus's fully-constructed state — every field reads its
declared post-birth value, so the on_failure handler's
capture-snapshot sees coherent state. Multiple clauses evaluate
in declaration order; the first to fire short-circuits the
rest.
Why a separate clause vs. calling `violate NAME;` inside the
birth body: a violate mid-birth leaves the locus partially
constructed (some fields set, others at defaults) when the
on_failure handler reads captures. `birth_check` runs the body
to completion before the check fires, so the post-birth invariant
of "every field has its declared value" holds at violation time.
The runtime-routing semantics are otherwise identical to a
regular `violate` (drain_requested set, parent on_failure
absorbs or process exits non-zero with diagnostic). The codegen
emits the check + violate routing INLINE at the instantiation
site, branching to a continuation block on absorbed violations
rather than returning from the caller's fn — the absorbed-then-
continue contract matches what users expect when wrapping the
instantiation in a parent that handles the failure.
The check expression is read-only against `self.X` fields; the
closure name must resolve to a declared epoch-inline closure on
the same locus, same constraint as a regular violate.
### `self.draining`
While the locus is draining, the synthetic `self.draining`
field reads `true` from any locus method body. The canonical
use is to suppress downstream sends after escalation:
```hale
let r = expr or self.handle_io(err);
if !self.draining { Result <- r; }
```
`self.draining` is the only synthetic field exposed by name to
user code; `__drain_requested` is internal-only.
### Rejection contexts
`violate` is rejected at typecheck in:
- **Free fn bodies.** No `self` to resolve the closure name
against. A free fn helper called from a locus method body
cannot violate transitively: `violate` is lexically scoped to
the locus method body it appears in.
- **`on_failure` body.** Use `bubble(err)` — `on_failure` is the
parent-side handler for child failures; re-firing a self-
closure from there mixes the two channels.
Allowed everywhere else that has `self`: named locus method
bodies, bus-handler methods (`subscribe X as foo` → `fn foo`),
`run()`, lifecycle methods (`birth()`, `dissolve()`, `drain()`),
mode-method bodies. The same body shape gets the same primitive.
## Perspectives: contract, `serves`, and the slot (Phase 2a)
A `perspective P { ... }` is a **contract** — a set of bodyless
`fn` signatures that form a stable ABI boundary — plus a
program-global, live-rebindable **slot** that holders dispatch
through. Phase 2a ships the contract, conformance,
the slot type, and dispatch; the live swap (`reperspective`)
followed in Phase 2b + 3 (see "The live swap" below).
```hale
perspective Router {
fn route(code: Int) -> Int; // bodyless contract signature
fn health() -> Int;
}
locus RouterV1 : serves Router { // declares conformance
fn route(code: Int) -> Int { return code + 100; }
fn health() -> Int { return 1; }
}
locus Gateway {
params { router: perspective(Router) = RouterV1 { }; } // holds the slot
fn handle(c: Int) -> Int { return self.router.route(c); } // calls through it
}
```
**`serves` conformance (error).** A `locus L : serves P` must
provide every contract method P declares — matching arity, param
types, and return type — **and** (Phase 2c) every bus edge P's
contract declares: a `bus { subscribe/publish ... }` block in the
perspective is part of the ABI, so a serving impl must subscribe /
publish each named subject. A missing or mismatched method, a
missing bus edge, or `serves` naming an unknown / non-perspective
symbol, is a typecheck error. Live-swapping a bus-backed
perspective (re-pointing its subscriptions) is a follow-up; until
then `reperspective` on such a perspective is rejected. This is the perspective analog of interface
structural satisfaction (and reuses its shape). The synthesized
`is_stable` (from `stable_when`) is not a contract method the impl
must provide.
**The slot type `perspective(P)`.** A holder programs against
`perspective(P)`, never a concrete impl. It is a handle: at the
LLVM level a single pointer stored in the holder's field, but
dispatch does **not** read that field.
**One global slot (1-1, not 1-N).** Each perspective P has exactly
one program-global slot — a `{ data, vtable }` cell (the interface
fat-pointer layout) named `__persp.`. Every holder of
`perspective(P)` funnels through it: `self.router.route(x)` loads
the global slot, indexes the vtable at the contract-method
position, and indirect-calls with `data` (the current impl's
self) as the implicit receiver. Because the interop is closed-
world and 1-1, the compiler sees every call site and there is
exactly one target — which is what makes the Phase-2b swap a
single atomic store that redirects the whole program. (Contrast
`interface`, which is a *per-value* fat pointer, many impls, no
global slot.)
**Designation.** A `perspective(P) = Impl { }` field default
*designates* the slot: it instantiates `Impl` (an owned child of
the holder, torn down normally) and stores `{ impl_self, vtable(Impl,
P) }` into the global slot. The field itself stores the impl's
self-pointer for ownership; the slot holds the same pointer plus
the vtable for dispatch.
**Cost.** Steady state is one load + one predicted indirect call
per call into a perspective — near-direct. The mechanism is
Linux/native-agnostic (no new runtime dependency); a program that
declares no perspectives pays nothing.
### The live swap: `reperspective` (Phase 2b + 3)
`reperspective self. as ;` is the live redeploy. It
re-points the perspective's global slot — identified by the
`self`-field's `perspective(P)` type — at `Impl` (which must
`serve P`). Because every holder funnels through the one slot, the
swap redirects **every** call site at once: the same
`self.router.route(...)` resolves to the new impl immediately after.
- **State-preserving (Phase 3).** The slot is `{ data, vtable }`:
`data` is the running state (an arena-backed struct), `vtable` is
the code. They are already separate, so the swap is a single store
of the new impl's vtable into the slot — `data` is untouched and
the new impl's methods continue on the **same live state** (the
note's layout-identity "zero migration"). No re-instantiation, no
birth defaults, nothing to tear down.
- **Footprint identity (soundness).** The vtable swap is layout-safe
only if the new impl's field offsets match the retained state, so
the typechecker requires **every** impl of a perspective to share
one footprint (same params, by name and type, in order). A
footprint *change* is the `migrate` case — rejected for now with an
actionable diagnostic rather than silently reinterpreting bytes.
- **Rebind authority.** The statement runs on the locus that owns
the slot (`self.`), never a mere caller — the ownership
tree is the redeploy authority. The typechecker requires the field
to be a `perspective(P)` param of the current locus and the new
impl to `serve P`.
- **Bus edges swap too (Phase 2c-runtime).** When the perspective
declares a bus surface, the swap also re-points its subscriptions:
it tombstones the current impl's registrations on the shared slot
`data` (`lotus_bus_quarantine_self`) and re-registers the new
impl's handlers on that same `data`. A message published after the
swap dispatches to the new handler, operating on the carried
state. Cooperative dispatch is deferred (a publish captures the
handler current at that moment; handlers run at drain), so the
swap boundary is respected per message. Perspective impls are
designated (never `placement`-pinned), so they are cooperative —
the re-registration routes through the global queue, no mailbox
hand-off. (Cost: tombstoned entries are skipped, not compacted — a
bounded per-swap cost.)
## Perspective hot-load
> **Status:** Phase 2b + 3 + 2c ship the `reperspective` swap
> (above): the atomic slot re-point, state-preserving across impls
> of one footprint, re-pointing sync dispatch AND bus subscriptions.
> A footprint-changing `migrate`, and the bus-arrival / decode /
> `stable_when` / drain flow below (transport-driven redeploy from
> the wire), remain the aspirational path.
For each `perspective P { ... }` instance currently active:
1. New perspective arrives via bus (or explicit
`load_perspective(P, bytes)` call).
2. Runtime decodes against P's compiled-in schema. Type-
mismatch → reject; emit `PerspectiveDecodeError`.
3. Validates `stable_when` predicate. If false → reject;
emit `PerspectiveNotStable`.
4. Atomically swaps the active perspective:
- Pause all readers (readers within the locus see the
pre-swap perspective).
- Replace.
- Resume.
5. Emit `PerspectiveLoaded` event.
Old perspective is freed only after the swap completes; no
torn read possible.
## Recovery primitives
### `restart(child)`
1. Schedule child for dissolution.
2. Once dissolved, instantiate a new child with the same
declared params.
3. New child's birth runs; old child's state is gone.
### `restart_in_place(child)`
1. Set child's "restarting" flag.
2. Wait for current handler / mode invocation to complete
(cooperative yield point).
3. Reset locus to post-birth state, preserving the arena.
4. Re-run birth().
5. Mark restart complete.
Useful for transient failures that don't invalidate the
locus's structural commitments (e.g., the locus's k_max is
fine; just had a bad message).
### `quarantine(child) [for d]`
1. Pause child (no new messages dispatched, no new accepts
permitted, no run scheduling).
2. Preserve arena and state.
3. If `for d` clause given, automatically restart after `d`.
4. Otherwise wait until parent explicitly resolves.
### `reorganize(child, ...)`
Reserved syntax. Semantics TBD; expected: relocate child's
sub-children to a sibling. Not in v0.
### `bubble(err)`
Re-raise the error to self's parent. Equivalent to:
```
on_failure(c, err) { bubble(err); } // for self's failures
```
Fully traverses the lotus tower upward until a handler
absorbs.
### `dissolve(child)`
Force-dissolve child immediately. Skips drain; closure tests
at non-dissolve epochs do not fire on this path. Used for
forced-shutdown scenarios.
## Drain cascade (whole-process)
SIGINT or SIGTERM:
1. Signal handler in the runtime root locus calls
`drain(self)` on itself.
2. Drain cascades depth-first to all children of root.
3. Each child cascades to its children, etc.
4. Leaves drain first; in-flight messages complete; bus
subscriptions stop accepting.
5. Each parent waits for all its children to drain before
draining itself.
6. Root drains last.
7. Runtime tears down schedulers, bus router, allocator.
8. Process exits 0.
## Closure-failure cascade
A closure violation at any epoch:
1. Runtime emits `ClosureViolation` event.
2. Locus's exploded flag is set.
3. Subsequent epochs may also fail; flag persists.
4. At dissolve, parent's `on_failure(self, ClosureViolation
{ ... })` invoked.
5. Parent's policy decides: absorb, recover, bubble.
6. If bubbled, propagates to grandparent; recursively until
absorbed or reaching root (process exit).
`epoch inline` closures (F.27) take the same cascade path with
one addition: at step 2 they also set `__drain_requested`, so
the locus enters drain at the next cooperative yield rather
than continuing on its current epoch. The drain initiation is
the only divergence from the auto-epoch cascade; routing to
parent's `on_failure` at step 4 is identical.
## Scheduler dispatch
Per `runtime.md`: multi-scheduler cooperative.
1. N schedulers, one per CPU core (configurable).
2. Each scheduler holds a queue of runnable loci.
3. Scheduler picks a locus, runs it until cooperative yield.
4. At yield, scheduler picks next.
5. Cross-scheduler communication via bus (typed messages).
6. Loci may be migrated between schedulers transparently for
load balancing.
## Failure-traversal flow
Failures flow upward:
1. Child failure → child's parent's `on_failure`.
2. Parent's handler decides absorb / restart / bubble /
quarantine / dissolve.
3. If bubble, → grandparent's `on_failure`.
4. If reaches runtime root, process exits with structured
error report.
Failures never flow laterally (sibling-to-sibling) — the
framework's vertical-only-flow expressed at the runtime layer.
## Fallible call semantics (v1.x-FORM-1; PR6 reframe)
Hale carries two **orthogonal** failure channels:
- **Closure-violation channel** — structural failure of a
locus's closure (its assertion / invariant) fires
`Signal::Bubble(ClosureViolation)` and routes through the
existing `bubble` / `on_failure` machinery. See **F.9**.
This is the *substrate-facing* channel: it expresses
"a locus's promised invariant broke" and propagates
vertically through the locus tower per the failure-
propagation-upward mechanic.
- **Value-error channel** — value-level `fallible(T)` returns
are an *addressing protocol* between immediate caller and
fallible callee. They don't constitute a separate runtime
mechanism at intermediate frames; they propagate by sret
+ path-indicator through the static call stack, addressed
at each level by a required `or` clause. This is the
*application-facing* channel: it expresses "this
call-by-call computation might fail; address it inline."
The two channels meet at exactly one place: the implicit main
locus's root boundary (see "Process exit" below). Everywhere
else, the channels are independent. See
`spec/design-rationale.md`.
### Where each channel lives (declaration sites)
The two channels are realized through different declaration
sites. The mapping is canonical, not advisory:
- **`fallible(E)` may be declared on:**
- **Free fns** — pure application-layer computations whose
failure shape matters call-by-call.
- **Stdlib-synthesized methods on `@form(...)` containers**
(`@form(vec).get` / `.pop`, `@form(hashmap).get` /
`.remove` / `.key_at` / `.entry_at`,
`@form(ring_buffer).pop`). Application-layer storage
substrate: the container's role is application-layer
data, not locus-structural participation in the
substrate's lifecycle.
- **User-declared `fn` member fns on a locus**
(open-question #24, shipped 2026-05-25 in two phases:
MVP at `d565d6f` with value-only payloads, v0.2 at
`98910b9` extending to heap-bearing payloads via the
TLS caller-arena snapshot non-fallible heap-returning
methods already use). The narrowed rule recognises that
a `fn` member fn is *not* substrate-orchestrated — its
callers hold a frame, can address the error channel
inline, and the value-error path doesn't conflict with
the closure-violation channel.
- **`fallible(E)` may NOT be declared on:**
- **Lifecycle methods** (`birth` / `run` / `accept` /
`drain` / `dissolve` / `on_failure`). The substrate
orchestrates these — bus dispatch invokes the handler,
parent invokes `accept`, runtime invokes `run` — and
there's no caller frame in user code to address a value
error. Physically rejected at the AST level
(`LifecycleDecl` carries no `fallible` field).
- **Mode methods** (`bulk` / `harmonic` / `resolution`).
Same shape: AST doesn't carry the field.
- **Closure assertions.** Substrate evaluates the
assertion at the epoch boundary; there's no caller in
the expression's frame to address a value error.
Closures route failure via their own structural channel
(assertion firing → `on_failure`), not a value channel.
- **Bus-subscribed handlers.** Verified at the
`subscribe ... as handler` site rather than at the fn
decl: a fn that's `fallible(E)` by declaration may not
also be referenced by a `subscribe` entry. Bus dispatch
has no caller frame; subscribing a fallible fn would
have nowhere to send `out_err`. The typecheck
diagnostic fires at the subscribe site, naming the
handler fn.
The rule is **two-channel separation at substrate-facing
surfaces**, not "no fallible on any locus method." The
load-bearing constraint is *who's the caller* — when the
substrate orchestrates a method (lifecycle, mode, closure
assertion, bus handler) there's no caller frame to address
the error channel, so `fallible(E)` would describe a
contract that cannot be satisfied. User-declared `fn`
members called from inside a method body or from another
locus's method body have an addressable caller; they carry
`fallible(E)` like free fns do, with the same `or`
disposition surface.
Example shape post-narrowing:
```hale
type ParseError { msg: String; }
locus Reader {
// Allowed: user-declared `fn` member with fallible(E).
// The body can `fail ParseError { msg: ... }` or call
// other fallible functions and propagate via `or raise`.
fn parse_message(b: Bytes) -> Message fallible(ParseError) {
// ...
if bad { fail ParseError { msg: "bad header" }; }
return Message { ... };
}
// Allowed: lifecycle method calling a fallible member fn
// and addressing the error inline.
run() {
let m = self.parse_message(b) or default_message();
// ...
}
}
```
The earlier v0 rule (blanket "no fallible on locus methods")
was narrowed because the friction signal across multiple
apps and libraries showed devs extracting free fns just to
get a value-error channel back — losing `self` ergonomics
and splitting closely-related code across two top-level
decls. See `notes/open-questions.md` § #24 for the
narrowing's full reasoning and the rejected alternatives.
### `fail` statement
`fail ;` inside a fallible fn body:
1. Evaluates `` to a value `v` (typed as the fn's
declared payload type E).
2. Exits the enclosing fallible fn body via the error path.
3. The caller's `Expr::Call` sees the result as
`FallibleErr(v)` — a tagged value the immediate caller's
`or` clause is required to address.
`fail` outside a fallible body is a typecheck error;
statement-position recognition is also parser-gated to a
fallible-body scope (so `let fail = 0;` outside such a body
stays admissible).
### `or` disposition
` or ` evaluates ``. If the result is a
non-error value, that value is the expression's value
(disposition is a no-op). If the result is `FallibleErr(p)`:
- **`or raise`** — propagate the error one frame up the
static call stack. Inside a fallible(E) fn, this writes `p`
into the enclosing fn's error sret slot and exits via the
enclosing fn's error path; the enclosing caller's `or`
clause then addresses the error in turn. The closure-
violation channel is **not** entered. (An application may
later promote a value error to a closure violation
explicitly, but no such syntax exists in v1.)
- **`or `** — binds `err` to `p` in scope and
evaluates ``. Its result is the expression's
value. Type must match the success type. `` may
itself be a call (`or handler(err)`); the identifier `err`
in the fallback expression resolves to the typed payload.
- **`or discard`** (added 2026-05-16) — swallows the error
and produces Unit. The underlying call's success type MUST
be Unit; the typechecker rejects `or discard` on
value-bearing calls with a message pointing at
`or ` or `or raise`. Sugar for the previously-
idiomatic `or noop(err)` pattern with a no-op handler fn.
- **`or fail `** (added 2026-05-17, B3 / G6) —
symmetric to `or raise`, but the caller picks a fresh
payload of the enclosing fallible fn's declared error type
instead of forwarding the inner call's payload verbatim.
Lets a caller translate one error shape into another
inline (`std::str::parse_int(s) or fail AppErr { msg: "bad
number" }`) rather than bouncing through a helper fn. Same
divergence rule: chain value type collapses to the inner
success type. Typechecker rejects outside a fallible fn
body with a hint to use `or raise` or `or `.
Chains are right-associative: `a() or b() or raise` reduces
the value to the success type level by level.
### Process exit
The runtime ends a program against its will via one of two
boundary events at the implicit main locus's root:
1. **Closure-violation escape (F.9).** If a closure violation
bubbles past every `on_failure` handler back to the
runtime root, the process exits with the violation's
payload as the structured error report.
2. **Value-error escape (PR6).** If an `or raise` reaches
the implicit main locus's body with no enclosing
`fallible(E)` frame to absorb it, the value error escapes
the locus's body. The runtime panics via
`lotus_root_panic(payload, size, typename)` — today
dprintf to stderr (`"Hale panic: unhandled
escaping main locus"`) + `exit(1)`. Architecturally this
is the seat for a future routing-through-main-locus-
`on_failure` extension: when (if) the main locus declares
`on_failure`, the runtime will route the synthesized
ClosureViolation through that handler before falling out
to the dprintf+exit fallback. Until then the boundary
collapses both channels to the same exit shape.
Both paths preserve the framework's vertical-only-flow: every
failure exits through the top of the recursion, never
laterally.
## Cross-seed namespace resolution (v1.x-IMPORT)
A file may declare `import "" as ;` at the top.
References to library decls go through the alias as
`alias::Name`. Resolution is two-step:
1. **Parse / merge.** The CLI resolves each import's path (per
`spec/projects.md` "Resolution order"), parses every `.hl`
file in the resolved target, applies the auto-mangler with
a stable path-derived `` + each file's stem, and
merges the mangled items into the importing program's item
list. A per-build path-rename table is built mapping
`["", ""]` to `__lib___` —
the `` is the importer's local namespace choice; the
`` is the lib's canonical path identity, so two
consumers importing the same lib under different aliases see
identical mangled symbols.
2. **Codegen lookup.** Codegen's qualified-name resolution
consults three tables in order — static `STDLIB_PATH_RENAMES`,
static `MOA_PATH_RENAMES`, and the per-build import table —
when lowering any path-qualified type expression, struct
literal, or method receiver. The first matching table wins.
Cross-seed references in user code (`foo::Bar`) and intra-seed
references inside the imported library (bare `Bar` from a file
that uses a type declared in a sibling file) BOTH resolve to
the same mangled symbol. The mangler builds a unified rename
map across every file in the imported library before
rewriting, so `greet.hl`'s reference to a `Formatted` type
declared in `format.hl` rewrites to the same
`__lib__format_Formatted` symbol that `format.hl`'s
decl ends up at.
Local bindings (`let`, `let mut`, fn params, lifecycle params,
for-loop vars, pattern bindings, generic params) shadow
top-level names per ordinary lexical scope; the mangler's
scope-aware walker leaves shadowed references unrewritten.
**Per-importer scoped imports (A4).** Imports
declared inside imported library files **are** followed
transitively by the resolver, but each library's imports land
under that library's own alias namespace — they do not become
visible to the top-level program. So library A importing
library B exposes A's surface to the importer; B is reachable
only through A's API surface (or by the importer re-declaring
its own `import "lib/B" as ...;`). This replaces the prior
strict barrier (which rejected transitive imports outright) and
unblocks composition without leaking dependency identity. See
`spec/projects.md` for the rationale and per-alias scoping rules.
**`hale run` interaction.** `hale run` compiles through the same
codegen path as `hale build`, and — as of WS3.3 —
the same *import* path: both the single-file and directory forms
(`hale run ./dir`) resolve cross-seed imports, build the per-build
path-rename table, and rewrite qualified `alias::Name` references
identically. A directory `run` produces the same resolved program
as the corresponding `build` and execs it. (Previously the
directory `run` form bundled files without the rename table, so
cross-seed imports only worked under `build`; that gap is closed.)
## Region lifetime guarantees
Per `memory.md`:
- A locus's region is freed atomically on dissolve.
- Sub-regions are freed before the parent's region (drain
cascade ensures this).
- No pointer-into-a-freed-region is reachable after region
release (compile-time-checked + region-lifetime-checked).
> Forward-looking / deferred items for this area now live in the
> decision log — see [`decisions.md` § Deferred & future
> work](./decisions.md#deferred--future-work).
================================================================================
## Type system
## source: spec/types.md
================================================================================
# Type system
This document specifies Hale's type system: what types exist,
how they relate, what the compiler verifies. Where the grammar
(`grammar.ebnf`) tells you what's syntactically valid, this
document tells you what's *meaningfully* valid.
## Primitive types
| Type | Repr (v0 codegen) | Notes |
|---|---|---|
| `Int` | i64 | Signed; default for integer literals |
| `Uint` | i64 | Unsigned at the type level; codegen lowers as i64. Parser-recognized; full lowering pending a workload that exercises the unsigned-arithmetic distinction. |
| `Float` | f64 | IEEE 754 double; default for float literals |
| `Decimal` | i128 | Mantissa with implicit scale 9 (`mantissa × 10^-9`). Distinct from `Float` at the type level; same-shape arithmetic with scale-adjusted mul/div. Suffix `d` on literals (`1.50d`). Real arbitrary-precision deferred. |
| `String` | ptr (NUL-terminated) | UTF-8 bytes, C-style NUL-terminated. Single-pointer ABI to fit return-by-value through the m49 calling convention. Embedded NUL truncates — use `Bytes` for binary content. Allocated in the caller's arena (or the lazy global payload arena for stdlib returns whose lifetime needs to outlive the call). |
| `Bool` | i1 | `true` / `false` |
| `Time` | ptr (string-shaped, v0) | v0 codegen stores `Time` as a pointer to the literal's source-spelling String — a placeholder shape that the typechecker keeps distinct from `String`. Real `i64`-since-epoch lowering deferred. |
| `Duration` | i64 | Nanoseconds. Suffix literals (`5s`, `100ms`). |
| `Bytes` (m89) | ptr → `[i64 len][u8 data[len]]` | Binary-safe. Single-pointer ABI like String, but the underlying blob carries an explicit length prefix so embedded NUL bytes don't truncate. `len(b)` reads the prefix. Distinct from `String` at the type level; the typechecker keeps them apart. Operations: `std::io::fs::read_bytes` (m89), `Stream.send_bytes` (m89), `Stream.recv_bytes` (Phase 2g; both `fallible(IoError)` since #209), `std::bytes::at` / `std::bytes::slice` / `std::bytes::from_string` (Phase 2g), `std::str::from_bytes` for the inverse direction (Phase 2g). |
| `BytesView` (F.30) | `lotus_view_t { src: ptr, epoch: i64 }` (16 bytes, by-value; SysV AMD64 returns in `rax`/`rdx`) | Non-owning view over a `BytesBuilder`'s buffer. Returned by `BytesBuilder.view()`. `src` is the builder pointer; `epoch` snapshots the builder's `mutation_epoch`. The underlying Bytes-shaped data pointer (`buf - 8`) is *recomputed* at read time by `lotus_bytes_view_data`, so the view itself doesn't allocate or carry the data pointer. Coerces implicitly to `Bytes` at function-argument READ positions (e.g. `std::bytes::at(view, i)`, `len(view)`, user-defined fallible-fn args, self/external/interface method args, monomorphized-generic args); codegen emits a call to `lotus_bytes_view_data` which checks the stamped epoch against the builder's live epoch, panics on mismatch (F.30b mutation-while-view-live guard), and returns the recomputed data ptr on the OK path. Rejected at `Bytes`-typed storage sites — callers wanting owned storage must `std::bytes::clone(view)` for a deep-copy into the caller's arena. Storage typed `BytesView` is allowed; a `String` / `Bytes` literal at a `BytesView`/`StringView` storage default is wrapped via `lotus_view_from_static_data` with `epoch == LOTUS_VIEW_EPOCH_STATIC = -1` — the unpack helper sees the static sentinel and returns `src` directly without an epoch check (the literal lives in the global string table at program-lifetime, so there's no source builder to check against). 2026-05-22 PM: ABI compacted from a 24-byte heap-allocated struct to this 16-byte by-value shape; no arena allocation per `view()` call. |
| `StringView` (F.30) | `lotus_view_t { src: ptr, epoch: i64 }` (16 bytes, by-value) | Non-owning view over a `BytesBuilder`'s NUL-terminated buffer. Returned by `BytesBuilder.text_view()`. Symmetric companion to `BytesView`: same layout, same epoch guard, same static sentinel. Coerces to `String` at read sites via `lotus_str_view_data` (which recomputes the C-string pointer as `b->buf` — the `buf[len] == '\0'` invariant maintained by every mutating op makes this well-formed); rejected at `String`-typed storage; `std::str::clone(view)` upgrades to owned. |
| `BytesMut` (#3) | raw `{ptr, len}` window (by-value; no `[i64 len]` prefix) | A non-owning **raw writable/readable window** — distinct from `Bytes` (which carries a length prefix the handle points *into*) because `BytesMut` is a bare `{ptr, len}` pair over memory owned elsewhere. Handed out by the zero-copy ring producer (`Topic.write(max) { w => … }` binds `w: BytesMut` over the reserved slot) and by `std::io::MirrorRing` (`readable()` / `writable()` return a `BytesMut` over the live / free region). Read it zero-copy with the `_raw` siblings of the binary-pack family (`std::bytes::read_*` / `at` / `find_byte` accept a `BytesMut` directly — length is the window length, not a prefix); write into it with the binary-pack writers (`std::bytes::write_*`). The window is valid only until the next ring commit / mirror advance — no epoch guard, so the lifetime discipline is the caller's. |
**FnPtr (m80):** First-class function values, type-spelled
`fn(T1, T2) -> R` (or `fn(T1, T2)` for void-returning). LLVM
lowering is `ptr` (raw fn pointer); calls go through
`build_indirect_call` with an LLVM `FunctionType` synthesized
from the FnPtr's args/ret at the call site. The implicit
`__caller_arena: ptr` first param of every user fn (m49 calling
convention) is also expected on the FnPtr's call ABI —
indirect calls prepend it before user-visible args. See
`stdlib/io_tcp.hl` for the canonical use:
`Listener.on_connection: fn(std::io::tcp::Stream)`.
**FFI-portable subset (Stage-1 FFI):** the primitive
type set above carries an additional axis of distinction at the
`@ffi("c")` boundary: which types have a stable C-ABI mapping.
`Int` / `Float` / `Bool` / `Duration` / `Time` / `String` /
`Bytes` / `BytesView` / `StringView` may appear in `@ffi`
parameter and return positions; `Decimal` / `Uint` are
typecheck-rejected (platform-variable ABI or Hale-internal).
See [`spec/ffi.md`](./ffi.md) for the full marshalling table and
the lifetime contract.
## Compound types
| Construct | Form | Notes |
|---|---|---|
| Slice / array | `[T]` or `[T; N]` | Dynamic or fixed-size |
| Bounded collection | `bounded[T; N]` | Fixed-capacity counted list, INLINE in its containing type/params (`{ i64 len, [N x T] }`). See § "bounded[T; N]" below. |
| Tuple | `(A, B, C)` | Fixed-size heterogeneous |
| Struct | `type Foo { x: Int; y: Int = 0; }` | Named record. Each field can declare a default value (`= expr`); literals omitting a defaulted field fill it from the default at instantiation time. |
| Enum | `type Foo = enum { A, B(int) };` | Tagged union (sum type) |
| Function | `fn(A, B) -> C` | First-class function values |
| Generic | `Foo` | Parametric over type T |
## Projection-class types
`Rich`, `Chunked`, `Recognition` are **language-native
generic constructors**. The compiler recognizes them and selects
allocator + implementation strategy based on which projection-
class wrapper a value carries (per `memory.md`).
The constraint `` (per F.2) is a built-in
"any-of-three" constraint: T must instantiate to `Rich`,
`Chunked`, or `Recognition` for some U. No trait system
required.
```
fn process(input: P) -> P { ... }
```
The compiler monomorphizes per concrete `P` instantiation.
## Locus types
A `locus L { ... }` declaration introduces a *locus type* L.
Locus types have:
- A set of **params** (name, type, default value or
`: inferred`); these are also the locus's mutable state (per
F.3 / §3 in decisions).
- Optional **contract** (expose / consume entries).
- Optional **capacity slots** (F.22 — `pool X of T;` / `heap Y
of T;` declarations naming slots 1..N beyond the implicit
slot 0 / Arena).
- Optional **lifecycle methods** (`birth`, `accept`, `run`,
`drain`, `dissolve`, `on_failure`).
- Optional **mode declarations** (`bulk`, `harmonic`,
`resolution`).
- Optional **bus interface** (subscribe, publish).
- Optional **closure tests**.
- Optional **member fns**.
Instantiating a locus type produces a **locus handle** of that
type, allocated as a region within the enclosing scope (per
`memory.md`).
## Capacity-slot cell handles (F.22)
`Cell` is the value type returned by `acquire()` (Pool slots)
and `alloc()` (Heap slots), and accepted by `release(c)` /
`free(c)`. It is **not** user-spellable in source — there is no
`let x: Cell = ...;` syntax. The type appears only at
typecheck and codegen.
| Aspect | v1 behavior |
|---|---|
| LLVM repr | `ptr` — a typed pointer to T's struct layout |
| Element type | The boxed inner type carries T from the slot's `of T` declaration; `Cell` and `Cell` typecheck distinctly. |
| Validity surface | Round-trip only: a value can flow through `let`-bindings, get re-supplied to `release` / `free`, and live inside the locus body. |
| Forbidden ops | println, arithmetic, comparison, fn-return-boundary crossing. Each rejects with a focused build-time diagnostic. |
| Field access (v1.x-2) | Struct cells support `cell.field` reads and `cell.field = v` writes; lowers to struct GEP + load/store. Primitive cells (`Cell` etc.) reject field access with a focused diagnostic. |
| Slot-of-origin tracking (v1.x-5) | `Cell` carries both T AND the originating `(locus, slot)` pair. Releasing a cell into a different slot than it came from is a hard error at codegen, with a diagnostic naming the originating slot. |
## Perspective types
A `perspective P { ... }` declaration introduces a *contract
type* P — a set of bodyless `fn` signatures (optionally a
`bus { subscribe/publish ... }` surface) that form a stable ABI
boundary. Holders program against the **slot type**
`perspective(P)` — never a concrete impl — and dispatch through a
single program-global, live-rebindable slot. A locus
`L : serves P` provides the contract and can be swapped in behind
the slot at pointer-flip cost via `reperspective`. An optional
`stable_when { ... }` predicate feeds the (aspirational)
transport-driven hot-load path. See
[`semantics.md` § Perspectives](./semantics.md) for the full model
(contract / `serves` / slot / live swap).
## Interface types (F.20)
An `interface I { fn ...; ... }` declaration introduces a
**structural interface type** I — a named set of method
signatures. A locus L satisfies I iff for every method in I, L
declares a method with the same name, the same arity, compatible
param types, and a compatible return type. Satisfaction is
**implicit**: there is no `impl I for L` declaration.
Interface types appear in fn parameter positions:
```
fn render(sink: Sink) {
sink.line("hello");
}
```
The structural-impl check fires at every call site where a fn
declares an interface-typed param: missing-method, arity-
mismatch, param-type, or return-type mismatches all produce
typed diagnostics at typecheck time.
**v0.1 scope (Phase A + Phase B).** Interface declarations
parse, register, and the typechecker enforces the structural
rule (Phase A, shipped 2026-05-10). **Codegen vtable dispatch
(Phase B) is shipped 2026-05-11.** Interface values are fat
pointers `{data, vtable}` allocated in the current arena; the
data slot holds the underlying locus pointer (same single-ptr
ABI as `LocusRef`) and the vtable slot holds a per-(locus,
interface) static global of fn pointers indexed by interface-
method declaration order. A locus flowing into an interface
slot coerces at the call site; method calls on an interface
value lower as indirect calls through `vtable[i]` with the
data pointer passed as the implicit self arg.
Interface values are usable as fn parameters, fn returns, locus
param / field values, and `@form(vec)` cell elements. Method-call
receivers, polymorphic return through control flow, and
pass-through aliasing (the original instantiator's binding and
the returned binding refer to the same underlying locus) all
work end-to-end. The `std::text::Sink` stdlib migration (split
`Sink` into `StdoutSink` / `StringSink` / `FileSink`
loci behind one `Sink` interface) shipped 2026-05-11; see
`spec/stdlib.md` and `crates/hale-codegen/tests/sink_polymorphism.rs`.
The implicit LocusRef → Interface coercion fires at the
following positions:
- **Free-fn arguments.** `fn render(s: Sink)` accepts any
LocusRef of a locus satisfying `Sink`.
- **Locus-method arguments.** `fn add(t: Tower)` on a `Registry`
locus accepts the same coerce (added 2026-05-18).
- **Returns.** `return r;` from a fn declared
`-> Sink` builds the fat pointer at the return site.
- **`type` field initializers.** `TowerEntry { t: r }` where
field `t` is interface-typed coerces the LocusRef `r`
(added 2026-05-18).
- **Locus `params` / field initializers.** Same shape as
above for locus param defaults and `locus L { params { t:
Tower; } }` slots.
- **`@form(vec)` cell `push`.** A `Registry @form(vec) of
Tower` accepts pushes of any satisfying LocusRef.
- **`or ` fallback expressions.** When a fallible
has success type `Interface(I)`, an `or fallback`
expression of LocusRef type satisfying `I` coerces
(added 2026-05-18). E.g. `lookup(...) or Hello { }`
where `lookup` returns `Greeter fallible(...)`.
- **let-binding ascription with composite type (G20).** `let arr: [Greeter; 2] = [Hi {}, Hey {}];`
coerces each element through the ascription's element type;
same shape for `let pair: (Greeter, Greeter) = (Hi {}, Hey
{});` and `let arr: [Greeter; 3] = [Hi {}; 3];`. The codegen
routes the RHS through `lower_expr_into(expr, hint)` so the
composite-element coerce fires per position. The single-
element let-ascription case is still permissive-via-Unknown
(interface names resolve to `Ty::Unknown` at typecheck per
`collect_known_names`); composite-element coercion picks up
where the single case's implicit handling leaves off.
The return path uses two cooperating mechanisms: at the return
site, an implicit locus → interface coercion builds the fat
pointer, and the locus-instantiation routing extension (the same
m90 shape that handles `-> LocusRef(L)` returns) routes any
instantiation of a satisfying locus inside an `-> Interface(I)`
fn body to the program-lifetime payload arena. The fat-pointer
struct itself is then deep-copied into the caller's arena by
`emit_return_value_deep_copy`. Single-element coverage is in
`crates/hale-codegen/tests/interface_return.rs`.
**Composite-construction coercion (G20).** Interface
elements inside fixed-size arrays, array-repeat literals, and
tuples are now coerced at the construction site when the
destination type is known. The let-RHS with a composite
ascription is the wired entry point:
```hale
let arr: [Greeter; 2] = [Hi { }, Hey { }];
let arr3: [Greeter; 3] = [Hi { }; 3];
let pair: (Greeter, Greeter) = (Hi { }, Hey { });
```
The codegen propagates the ascription's element type through
`lower_expr_into(expr, hint)` so per-position
`coerce_to_interface` fires before the array's "mixes element
types" check would otherwise reject heterogeneous LocusRefs.
Tests live in
`crates/hale-codegen/tests/interface_in_composites.rs`.
Once the let-RHS coerces, the array's static type is already
`Array` (or the tuple's positional types are
`Interface`), so flowing it to fn-args, struct fields, or
return positions of the same type goes through plain type
matching — no further coercion is needed at the consumer
side.
**Still deferred:** locus-routing across nested return positions
— a fn declared `-> [Greeter; N]` instantiating loci inline in
its return expression still aliases the fn's stack frame.
Closing that needs the m90 routing extension to fire on nested
locus instantiations inside composite returns, the same gap that
governs tuple-of-`LocusRef` escape today.
F.11 child acceptance (`accept(c: ConcreteLocus) { ... }`) is
intentionally NOT in the coerce list above. The child-accept
mechanism keys dispatch by exact concrete locus name across
substrate sites — accept fn signature, the `self.children`
storage layout, and the parent's accept-dispatch table — so
`accept(c: Iface)` is a multi-system change, not a coercion
wire-up. Single-accept-type per parent is the v1 design.
Interfaces have no default methods at v0; the body is signature-
only. No interface inheritance, no multi-interface bounds on
generics, no interface equality. F.21 sketches a paired
substrate-aware (cascading-dimension) interface form for the
n-dim growth case; not implemented at v0.
## Type compatibility
### Subtyping
Lotus is **invariant** at the type level — no implicit subtyping.
A `Rich` is not assignable to a `Chunked` even though
both wrap `int`; explicit conversion required.
Exception: contract-graded subtyping (next section).
### Numeric coercion: Int → Float (Phase 2c)
A single documented one-way widening fires at the following
surfaces:
- **let-binding type ascription:** `let nf: Float = self.n;`
where `self.n: Int` widens `n` to a Float at the binding
site (codegen `sitofp`).
- **fn-arg coercion:** when a parameter is typed `Float` and
the call-site argument is typed `Int`, the argument widens
at the call site. Same rule applies to user-declared fns
and to stdlib path-calls (`std::math::sqrt(n)` with `n: Int`
works without `2.0` literals).
- **binary-op promotion** (B13 / G30): when exactly
one side of a numeric binop is `Int` and the other `Float`,
the `Int` side widens to `Float` and the op produces `Float`.
Same rule covers comparison ops (`<`, `>=`, `==`) so
`i < 0.5` typechecks. Symmetric — either side can be the
one that widens.
- **user-type field init:** assigning an `Int` value into a
`Float`-typed struct field at literal-init time widens at the
store, mirroring fn-arg coercion. Lets a config bundle
declare `timeout: Float` and accept an `Int` from the caller
without sprinkling `Float(n)` casts.
The widening is **strictly one-way**. `Float → Int` narrowing
remains explicit (round + cast). `Decimal` never participates
in implicit cross-type conversion. The rule was added 2026-05-11
as part of the float-surface-gaps friction-log resolution; see
F.23 in `spec/decisions.md` and the Phase 2c entry in
`spec/stdlib.md`.
#### Explicit numeric conversions
Where the implicit widening above does not apply — most often a
`Float → Int` narrowing, or an `Int → Float` conversion needed in
the middle of an expression rather than at one of the coercion
surfaces — there are two explicit forms, both round-toward-zero
for the narrowing direction (LLVM `fptosi` / `sitofp`):
- **The `Int(x)` / `Float(x)` casts** — the idiomatic in-language
form. `Int(f)` narrows a `Float` to an `Int` (truncates toward
zero); the cast is opt-in, so there is no silent `Float → Int`.
- **`std::math::int_to_float(i: Int) -> Float` and
`std::math::float_to_int(f: Float) -> Int`** (WS3.1)
— the named-function spelling, callable in any expression
position. Semantically identical to the casts (`sitofp` /
`fptosi`, round-toward-zero); provided so numeric code does not
have to round-trip through ASCII (`to_string` + `parse_*`) and
so the conversions sit alongside the other `std::math` numeric
primitives. An already-correct-typed argument passes through
unchanged.
- **`std::math::round(f: Float) -> Int` and
`std::math::trunc(f: Float) -> Int`** — the Float→Int
conversions that round at a chosen mode. `trunc` is round-
toward-zero (an alias of `float_to_int`); `round` is round-
half-away-from-zero (`3.7 → 4`, `2.5 → 3`, `-2.5 → -3`),
computed as `fptosi(f + copysign(0.5, f))` via a compare/select
half-shift (no `llvm.round` intrinsic, so the path needs no
libm libcall — it lowers host-free on `wasm32`). `round` is the
spelling numeric code wants when building an integer field from
a Float quantity; `Int(f)` / `float_to_int` / `trunc` all
truncate.
### Contract compatibility
When parent declares `consume X: T` and child declares
`expose X: T`, the compiler requires:
- `X` is the same name in both.
- The child's type for `X` is a *subtype* of (or equal to) the
parent's expected type.
For v0, "subtype" is just type equality. Future versions may
admit covariant / contravariant relationships; v0 is invariant.
This is the F.8 commitment expressed as a typing rule.
### Three-way interface (F.14)
Per F.14: any function injected by L into its arena that
satisfies a contract entry must return the contract's typed
surface. The compiler verifies, for each contract `expose X: T`
declaration:
- L has either (a) a param named `X` of type `T`, OR (b) a fn
returning `T` named `X` (or an annotated impl), OR both.
- Multiple impls (the projection-class-specific case) are
permitted; all must return T.
Default-implementation rule: if no fn is annotated, the param
named X is the default implementation (read field directly).
## Mutability
Per F.E (design-rationale): bindings are **immutable by
default**. `let x = 0;` produces an immutable binding.
`let mut x = 0;` produces a mutable binding; reassignment
permitted.
Mutability is a per-binding property, not a per-type property.
There's no `Mut`; the binding either is or isn't `mut`.
For `params` / locus state, the implicit rule is that fields
are mutable through `self.x = ...` (per F.3). The locus's
state is the locus's mutable bundle.
**Reassigning a locus-typed field is a lifecycle transition.**
A field that holds a child locus (`params { conn: WsClient =
WsClient { … }; }`) can be whole-value reassigned from a member
fn (`self.conn = WsClient { … }`). Because a locus is not a plain
value — it owns a region and possibly `@ffi`-acquired resources —
this is lowered as **dissolve-the-old + construct-the-new**, not a
pointer store: the previous instance is reclaimed (its `drain` /
`dissolve` run, releasing its resources) and the new instance is
constructed into the owning locus's arena, owned by the field (so
the parent's dissolve cascade reclaims it). The field always
points at a fully-live instance. (Before this rule the new
instance was a scope-bound temporary dissolved at the method's
exit — a use-after-free; see WS1#4.) For "same instance,
reconfigure," prefer **in-place mutation** (`self.conn.url = …`),
which keeps the locus's identity and resources and is cheaper.
v1 limitation: the reassigned instance inherits the owner's pool;
reassigning a field with an explicit non-default `placement` does
not re-apply that placement.
## k_max as a typing rule
Per F.1 / F.3: the compiler computes
`k_max = B / [(1 - phi) * c + phi * sigma]` from the locus's
declared params. This determines the maximum coordinatees an
`accept()` can attach.
If params are constants (compile-time-known), k_max is a
compile-time integer. The compiler may reject `accept` call
sites that statically exceed k_max. (For dynamic params, the
runtime checks at each accept; exceeding k_max raises a
typed `KMaxExceeded` failure handled by the parent's
`on_failure`.)
## bounded[T; N] — fixed-capacity collections in types
Types are pure data, so they cannot hold a `@form(vec)` (a locus).
`bounded[T; N]` is the type-level collection: a fixed-capacity
counted buffer laid out inline as `{ i64 len, [N x T] }` — the
capacity is part of the type (K made value-level, the F.22
philosophy). Works in `type` fields and locus `params`.
Operations are GRAMMAR INTRINSICS (like `len(s)`), not methods, so
the types-have-no-methods axiom holds:
```hale
push(f, x) -> () fallible(CapacityError) // full = error;
// displacement policy
// lives in the or-arm
at(f, i) -> T fallible(IndexError)
set(f, i, x) -> () fallible(IndexError) // overwrite live slot
count(f) -> Int
clear(f) // len = 0
truncate(f, n) -> Int // len = clamp; returns it
for x in f { } // iterate live slots
```
Semantics:
- Fields auto-initialize EMPTY. Literal init and whole-field
assignment are rejected — the intrinsics are the only mutation
surface. Whole-STRUCT copies carry elements + count by
construction (the storage is inline).
- Scalar elements (Int/Float/Bool/Decimal/Duration) are flat under
`zero_copy` and travel the bus as raw bytes. Pointer-shaped
elements (String/Bytes/struct) work in-process — push/set
arena-anchor the element into the receiver's owning arena — but
are rejected in cross-process bus payloads (post-v1 polish).
- Drop-front/FIFO is the shift-left idiom: `set` live slots down,
then `truncate`.
- `CapacityError { cap: Int; count: Int }` and the shared
`IndexError` are the injected error shapes.
- The unbounded-alloc analysis treats bounded fields as bounded by
construction. `@form(vec)` remains the unbounded, locus-owned
collection: unbounded data lives on a locus; bounded data can
live in a type.
## Generics
**Generic type-expr ↔ monomorph unification:** a
generic instantiation type-expr (`Box`) resolves at typecheck
to its mangled monomorph name (`Box_Int`) — the same name codegen
synthesizes and that `Box_Int { ... }` literals produce — so
declarations and literals unify, and a `Box_String` literal in a
`Box` slot is a caught mismatch. Monomorph literal fields
validate against the template with the type args substituted, and
field reads on monomorph values type as the substituted field.
Generic params are declared with angle brackets:
```
fn map(xs: [T], f: fn(T) -> U) -> [U] { ... }
type Stack { items: [T]; }
```
The constraint syntax `` admits:
- `ProjectionClass` (built-in any-of-three; F.2)
- A specific projection class: `Rich`, `Chunked`, `Recognition`
- Concrete types: `` is illegal (use the type directly);
`` is also illegal (no trait system in v0)
V0 supports only projection-class constraints. Future versions
may add traits.
Monomorphization: the compiler emits one machine-code instance
per concrete generic instantiation (per F.1 commitment to
runtime perf over compile-time perf). Compile times grow with
generic surface; runtime is full-speed.
## Type inference
### `let` bindings
The type of `let x = expr;` is inferred from `expr`. Explicit
annotation `let x: T = expr;` overrides inference; if `expr`'s
type is incompatible with `T`, compile error.
### Function return types
If a fn omits `-> T`, the return type defaults to `()` (unit).
Explicit `-> T` is required for any non-unit return.
### Locus params
Params must declare types explicitly. `params { x: Int = 0; }`
is the full form. (Inference of param types from defaults is
not supported in v0; explicit is preferred for the `inferred`-vs-
`= default` distinction.)
Three init shapes:
- `name: T = expr;` — default. Used when the caller omits the
field; evaluated in the caller's scope at instantiation time.
- `name: T;` — **required**. The caller MUST supply the field at
the locus literal site; instantiation without it is a compile
error. Use for fields where no sensible default exists (e.g.
`Server { handler: ... }` where the handler is the whole
reason the locus exists).
**Exception (B7 / G19):** if `T` is a user-defined
`type` (struct) and every field of `T` has a declared default,
the compiler synthesizes `T { }` as the param's default. So
`params { cfg: Cfg; }` against `type Cfg { host: String =
"localhost"; port: Int = 8080; }` works without `= Cfg { }`
spelled out. Required-shape is preserved when any field on
`T` lacks a default.
- `name: T : inferred;` — F.3 inference path; compiler /
runtime determines the value.
**T may be another locus (B10 / G24).** A param
typed as a locus name (`params { db: DB; }`) stores a `LocusRef`
— a single-pointer borrow. The param-holding locus does **not**
own the referenced locus; the caller keeps it alive. Cross-decl
declaration order doesn't matter: a forward reference
(`User { db: DB; }` declared above `locus DB { ... }`) resolves
via the codegen-side `pending_locus_names` pre-pass.
Reading through the borrow (`self.db.name`, `self.db.draining`)
goes through the same field-access lowering as any other
LocusRef receiver. Synthetic fields (`self.db.k_max`,
`self.db.draining`) work on non-self receivers too (B14 / G31).
## `inferred` params
Per F.3: a param declared `: inferred` (instead of `= value`)
indicates the compiler / runtime determines the value, not the
author. The compiler treats:
- Hand-declared `= value` → prior, fixed at compile time.
- `: inferred` → unknown; resolved at compile time if possible
(constant propagation), at runtime otherwise (via the
perspective-stability machinery).
Typing-wise, inferred params have the declared type; they're
just not bound to a value at declaration time.
## Function types
Functions are first-class values. `fn(A, B) -> C` is a type;
function literals can be assigned, passed, returned.
A locus's `fn` member can be a method (takes `self`) or a free
function within the locus's scope. Lifecycle methods (`birth`,
`accept`, etc.) are not regular `fn`s — they have their own
syntax and don't take `self` (it's implicit).
## Contract subsumption
For two contracts `C1` and `C2`, `C1 ⊆ C2` iff every entry in
`C1` has a matching (compatible) entry in `C2`. This is used
for:
- Parent-child compatibility: parent's `consume` ⊆ child's
`expose`.
- Locus-type substitutability: when a child locus type is
expected, any locus type with a compatible expose-surface
may substitute.
## Vertical-only flow as a typing rule
Per F.6 / F.11 / `memory.md`: cross-locus references at the
type level are limited to the contract's typed surface.
Specifically:
- A reference to a coordinatee accessible only via
`self.children[i].x` where `x` is in the contract.
- Sibling references: not typeable. No syntax exists for
reaching from one sibling to another; if attempted via
manual pointer construction, the compiler rejects on
region-lifetime grounds.
- Grandparent references from a child: not typeable. Failure
flows through `bubble`; intent flows through the parent.
This makes the framework's vertical-only commitment a
type-system invariant, not just a convention.
## Single-threaded-method invariant (F.31)
A locus's methods may be invoked only on the OS thread that
owns the locus's placement's pool. Cross-pool direct calls
are typecheck errors; cross-pool coordination goes through
the bus.
The invariant is enforced via a static call-graph walk seeded
from the `main locus`'s `placement { }` entries:
1. Each main-locus `params` field has a pool — explicit
(`placement { field: cooperative(pool = X); }` or
`placement { field: pinned; }`) or default
(`cooperative(pool = main)`).
2. Each nested locus inherits its containing tower's pool
(see `spec/semantics.md` § "Nested instantiation"). Methods
on a nested locus run on the parent's pool's thread.
3. For a method-call expression `self.field.foo(args)`, the
receiver's pool is the pool of the *field instance*,
inferred at the call site: the enclosing locus's own
`placement { }` entry for `field` if it names one (e.g. a
`db: pinned` field on the main locus), otherwise the field
co-locates with its owner (the caller's pool). The pool is a
property of the **instance**, not the field's type — the
same locus type used as a field in two loci on two pools is
two independent instances, one per owner, each single-pool.
4. If the receiver instance's pool differs from the caller's
pool, the call is rejected with a diagnostic naming both
pools and pointing at the `placement { }` entries that
picked them.
5. Bus sends (`Topic <- v;` / `"subj" <- v;`) are unrestricted
— the runtime's cross-thread dispatch (m28b condvar+memcpy)
handles the boundary safely.
The invariant is the substrate enforcement that makes M:N
cooperative pools safe. Without it, multi-pool deployments
would silently race on locus arenas (unsynchronized bump
allocators by design). The typecheck happens once per main
locus (the placement-bearing locus is unique per binary), so
the rule applies at binary compile time rather than at every
library typecheck.
**Interaction with `LocusRef` borrows.** A locus param of
type `LocusRef(L)` carries a borrow of an `L` — but the
borrow's pool is the locus's own placement, not the borrower's
placement. So `self.db.query(...)` where `self.db: DB` and
the `DB` instance is on pool `db_pool` is a cross-pool call
from any non-`db_pool` thread, and routed through the bus.
This is the "vertical-only flow" rule generalized to
cooperative pools: cross-pool access is the same shape as
sibling access — bus only.
**Interaction with builtins / stdlib.** Free functions and
stdlib path-calls (`std::io::fs::read_file`, `std::str::*`,
etc.) are pool-neutral — they run on whichever thread calls
them. Their arena routing through `lotus_current_caller_arena`
TLS handles the per-thread isolation. The single-threaded-
method invariant applies only to locus member functions,
which are what carry per-locus arena state.
**Interaction with `@form(...)` loci.** A locus declared
with a `@form(...)` annotation (`@form(hashmap)`,
`@form(vec)`, `@form(ring_buffer)`) is **single-pool by
default** — its methods participate in the same single-
threaded-method invariant as any other locus. Plain
`@form(...)` cells have no runtime synchronization;
concurrent writers from different pools corrupt the
underlying structure.
Cross-pool access is opt-in via the `sync = ` kwarg on
the form annotation:
| Annotation | Discipline | Trade-off |
|---|---|---|
| `@form(hashmap)` | single-pool only | densest layout, no sync overhead, cross-pool calls rejected |
| `@form(hashmap, sync = serialized)` | per-map mutex (F.32-1α) | correct cross-pool; throughput bounded by lock contention |
| `@form(hashmap, sync = striped)` | cell-level CAS + per-map rwlock for grow + cache-padded cells (F.32-1β2-v2) | parallel writers; grow path serializes; rwlock overhead can outweigh parallelism on cheap-payload workloads |
| `@form(hashmap, sync = lockfree)` (optional `cap = N` hint) | cell-level CAS, no rwlock or mutex on the steady-state path; `remove` via tombstones + lazy grow with a brief migration stall (F.32-1γ-v2) | highest measured throughput on the false-sharing bench |
When a locus carries a recognized sync discipline, cross-
pool method calls into it are accepted without diagnostic —
the substrate's chosen discipline carries the safety
contract. Plain `@form(...)` (no sync kwarg) gets the same
cross-pool diagnostic as any other locus, extended with an
upgrade-path hint naming the sync kwargs.
Because a form field's pool is its instance's (rule 3 above),
two loci that each hold their own `@form` field of the same
type on different pools are two **separate**, single-threaded
structures — each accessed only by its owner's pool. Neither
is a cross-pool access, so neither is flagged, and neither
needs a `sync` discipline (there is no sharing to synchronize).
The diagnostic fires only on a genuine cross-pool access —
reaching one instance from a pool other than the one it lives
on, e.g. a form field explicitly placed off its owner. (This
is why two same-type form instances on two pools do **not**
require byte-identical twin types.)
Concrete shape: a `Registry @form(hashmap, sync = striped)
of Counter indexed_by name` shared across producer pools
(gateway loci incrementing counters) and a consumer pool
(`MetricsEndpoint` rendering Prometheus text) typechecks
clean. Without the `sync = striped`, every cross-pool
`self.registry.counter(...)` would be rejected.
Non-form receiver loci have no sync discipline available;
cross-pool coordination must go through the bus.
**Inference.** F.32-1∞ adds a closed-world inference pass
that picks a `sync` default per form-bearing locus type
from the pool-propagation graph. The explicit annotation
always overrides; the inferred pick is shown via a
compile-time diagnostic at the decl site. See
`notes/f32-cache-aware-delivery-plan.md` § F.32-1∞.
**History.** Commit `3ec6391` (2026-05-24, first cut)
admitted any `@form(...)` locus into the cross-pool-safe
set unconditionally, on the strength of a not-yet-shipped
"form ABI serializes" claim. Bench-prep for F.32-1
surfaced that the runtime had no synchronization on the
form paths; F.32-0 (this section's current state) scopes
the exemption to explicit opt-in via `sync = X`. See
`notes/f32-cache-aware-delivery-plan.md` § F.32-0.
## Closure-test typing
A `closure name { left ~~ right within tolerance; ... }`
declaration types as:
- `left` and `right` must be expressions of compatible
numeric types (integer or numeric).
- `tolerance` must be a non-negative numeric expression.
- The compiler verifies left/right resolve in the closure's
scope (which is the locus's scope; `self.x` is permitted,
member fns may be called).
- For `epoch tick` / `epoch duration(d)` / `epoch dissolve`
/ `epoch birth` / `epoch explicit`, the runtime evaluates
the closure at the appropriate event boundary.
A closure failure at evaluation produces a typed
`ClosureViolation` event (per F.9), not a generic error.
## Fallible typing (v1.x-FORM-1)
A function declared `-> T fallible(E)` produces a value of
type `T fallible(E)` at every call site. This type cannot be
used where a plain `T` is expected — the caller MUST address
the error before the value is consumable. See
`spec/design-rationale.md` for the design rationale.
**Declaration sites are restricted by the two-channel rule
(see `spec/semantics.md` § "Fallible call semantics"
§ "Where each channel lives").** `fallible(E)` may be
declared on:
- Free fns.
- Stdlib-synthesized methods over `@form(...)` containers
(`@form(vec).get` / `.pop`, `@form(hashmap).get` / `.remove` /
`.key_at` / `.entry_at`, `@form(ring_buffer).pop`).
- **User-declared `fn` member fns on a locus** (open-question
#24, shipped 2026-05-25). Heap-bearing success and err
payload types are supported via the same TLS caller-arena
snapshot non-fallible heap-returning locus methods use.
`fallible(E)` is **rejected** on substrate-facing surfaces
that have no caller frame to address the error channel:
- **Lifecycle methods** (`birth` / `run` / `accept` / `drain` /
`dissolve` / `on_failure`). Physically rejected at the AST
level — `LifecycleDecl` doesn't carry a `fallible` field.
- **Mode methods** (`bulk` / `harmonic` / `resolution`).
Same shape as lifecycle: AST doesn't carry a `fallible`
field.
- **Closure assertions.** Substrate evaluates the assertion at
the epoch boundary; no caller frame exists in the expression.
- **Bus-subscribed handlers.** A fn that's declared
`fallible(E)` may not also be referenced by a `bus
subscribe ... as ` declaration. Bus dispatch has no
return path. Rejected at the subscribe site, not the fn
decl (one fn may be referenced by zero subscriptions; the
subscription is what fails to typecheck).
The narrowing from "no fallible on locus methods" to
"substrate-facing surfaces only" preserves the two-channel
separation (structural failures still flow vertically via
closure violations + `on_failure`) while removing the
friction that made devs extract free fns just to get a value-
error channel back. The typechecker emits the diagnostic at
the offending site (locus member decl for lifecycle / mode;
subscribe site for bus-handler-fallible conflict).
### `Ty::Fallible { success, payload }`
The checker represents fallible returns as a wrapper around
the underlying success type:
| Source | Inferred type |
|------------------------------|-------------------------------------|
| `fn f() -> T fallible(E)` | `f()` has type `Ty::Fallible { success: T, payload: E }` |
| `match` / `or` on fallible | unwraps to `T` |
A `Ty::Fallible` is **not assignable** to its success type. It
must be unwrapped at the immediate call site. The checker
emits `error: error not addressed` at:
- `let v = f();` with `f` fallible
- `let v: T = f();` with `f` fallible (typed binding)
- `f();` as an expression statement
- `g(f())` — fallible passed as a non-fallible-typed arg
- assignment, return, condition positions
### Disposition operators (`or`)
` or raise`
: Propagate the error one frame up the static call stack.
Evaluates the inner; on `FallibleErr` payload, re-enters
the fallible-return shape of the enclosing `fallible(E)`
fn (the error climbs the call stack until a frame
addresses it). The value-error channel is value-level
and **orthogonal** to the closure-violation channel; the
`bubble` / `on_failure` machinery is not entered by
default. (An application may later promote a value error
to a closure violation explicitly; no such syntax exists
in v1.) On success, passes the inner value through. The
resulting expression's type is the success type T.
Past every enclosing `fallible(E)` frame — at the implicit
main locus's root boundary — the runtime panics via
`lotus_root_panic`. See `spec/semantics.md` § "Process
exit" for the boundary semantics.
` or `
: Substitute a fallback value of type T. On failure, evaluates
the fallback expression with `err` implicitly bound to the
payload (typed as E). On success, passes through. Fallback
type must be assignable to T. The fallback may itself be
a call (`or handler(err)`), making `err` a regular
expression-position binding inside the fallback.
**Fallible handlers:** the handler may itself be
`fallible(E2)`. Its success value substitutes; its FAILURE
propagates through the ENCLOSING fn's error path — implicit
`or raise`, sugar for the already-legal nested spelling
`call() or (handler(err) or raise)`. E2 must be assignable to
the enclosing fn's declared payload ("handler's failure has
nowhere to go" / "propagated payload must match" otherwise).
User free fns, imported-path fns, and locus member fns are
classified; `@form`-synthesized methods and stdlib path-calls
still need the explicit nested spelling. In statement position
the substituted value is discarded, so the handler's success
type needn't match the call's.
The fallback may be a **`{ block }`** — `or { … }`, with `err`
in scope — for multi-statement recovery. Two cases:
- A block that **always diverges** (`return` / `fail` on every
path) produces no substitute value, so it imposes no
constraint on T and is accepted for **any** success type —
`let s = read_file(p) or { return "fallback"; };` where the
fallible's success type is `String`, `Bytes`, a struct, etc.
(It disposes like `or raise`: the err branch is closed and
only the success value reaches the continuation.)
- A non-diverging block substitutes its **tail expression** as
the fallback value, whose type must be assignable to T:
`let s = read_file(p) or { log(err); "default" };`.
On a **Unit-success** fallible (`() fallible(E)`, e.g.
`std::io::fs::write_file`), `or { block }` runs the block for
effect — including in **statement position**:
`write_file(p, s) or { println("failed"); };` — the same as
`or raise` / `or discard` there.
The `or` operator is right-associative: `a() or b() or raise`
parses as `a() or (b() or raise)`, so each level disposes one
fallible in turn until a non-fallible value remains.
### `fail` statement
`fail ;` is only valid inside a fallible fn body. It
evaluates ``, requires the result type to match the fn's
declared payload type E, and exits via the error path (the
caller sees a `FallibleErr` value).
### Custom payload types
The payload E is an ordinary type expression — usually a small
user-defined record (`type ParseError { ... }`) or a stdlib-
synthesized type. The runtime / typechecker does NOT impose a
common base — there is no `Error` trait, no `impl Error for
ParseError`. Failure is a single anonymous fact; the payload is
just a value tagged onto the failure for diagnostic purposes.
### Synthesized stdlib payload types
The resolver injects four fallible-payload types into the top
scope so user code can name them in `fallible(...)` markers and
`or` substitute clauses. All are idempotent — a user-declared
type with the same name wins.
| Trigger | Type | Fields |
|---|---|---|
| `@form(vec)` | `IndexError` | `kind: String`, `index: Int`, `len: Int` |
| `@form(hashmap)` | `KeyError` | `kind: String` (also surfaces `IndexError` on `key_at` / `entry_at` — those are index-based, added 2026-05-16) |
| `@form(ring_buffer)` | `EmptyError` | `kind: String` |
| `std::io::fs::*` / `std::io::tcp::*` | `IoError` | `kind: String`, `errno: Int`, `path: String` |
The `IoError` payload is the unified shape for the
fallible I/O surface — see `spec/stdlib.md` § "IoError" for the
errno → kind tag taxonomy.
## Recovery-primitive typing
Recovery primitives (`restart`, `restart_in_place`,
`quarantine`, `reorganize`, `bubble`, `dissolve`, `drain`)
are statement-level keywords (per `precedence.md`); they
don't have types in the value sense. They take a locus handle
or error value as argument:
```
restart(child);
quarantine(child) for 30s;
bubble(err);
```
The compiler verifies the argument is a valid handle / error
in the current scope.
## Working-set estimator (F.32-2)
F.32-2 ships a compile-time working-set estimator that
projects each user-declared locus's approximate byte cost
and compares against a cache-tier budget. The estimator runs
post-typecheck, pre-codegen; it doesn't change codegen
output and emits diagnostics rather than changing program
behavior.
**Estimator formula** (per-locus, in bytes):
```
working_set(L) =
sizeof(L's struct) [arena + user fields with alignment padding]
+ sum(slot in L's capacity slots) cap × cell_stride
+ sum(child in L's params if locus-typed) working_set(child)
```
Cache-tier budgets are read from
`/sys/devices/system/cpu/cpu0/cache/index{0,2,3}/size` on
Linux at first probe (cached for the build's lifetime);
static fallbacks 32 KB / 512 KB / 8 MB apply on non-Linux or
when sysfs is unavailable. See `hale_types::working_set` for
the engine.
**Per-locus annotation** (F.32-2 v0.2):
`@locality(L1)` / `@locality(L2)` / `@locality(L3)` declare
a per-locus cache-tier expectation. `@locality(any)`
explicitly opts the locus out of any global gate. The
annotation stacks with `@form(...)` in either order:
```hale
@form(hashmap, sync = lockfree, cap = 64)
@locality(L2)
locus Registry {
capacity { pool entries of Entry indexed_by k; }
}
```
The grammar surface is in `grammar.ebnf`
§ `locality_annotation`.
**Build-flag surface** (CLI, on `hale build`):
| Flag | Effect |
|---|---|
| `--locality-report` | Emit a per-locus stderr report listing each locus's estimated bytes, smallest-fitting tier, and a struct / capacity / children byte decomposition. Build proceeds. |
| `--target-cache l1\|l2\|l3` | Evaluate each locus against the named tier's budget. Over-budget loci surface as a stderr warning by default. |
| `--strict` | Convert the warning into a build error (exit 1 before codegen). Only meaningful in combination with `--target-cache` or a program that carries `@locality(...)` annotations. |
**Effective budget precedence** (per locus):
1. `@locality(L1|L2|L3)` annotation → that tier (hard
contract; evaluated regardless of CLI flag).
2. `@locality(any)` annotation → no budget (opts out even
under `--target-cache`).
3. No annotation → falls through to `--target-cache` global
tier (or no budget when the flag isn't set).
`--strict` controls warnings vs errors uniformly across both
sources. The diagnostic names which source applied (per the
`BudgetSource::label()` "@locality" / "--target-cache"
attribution).
**Estimator approximations** (deliberately imprecise; see
the working_set module doc for the full list):
- `params { }` field layout uses alignment-correct
accumulation (each field rounded up to its natural
alignment; struct rounded up to max field alignment). Was
packed-layout in v0.1; v0.2 adds the padding.
- Arena overhead modeled as a flat 64-byte budget for
synthetic headers.
- Heap-managed primitives (`String` / `Bytes` / `*View`)
count as 16 bytes (ptr + len); the heap buffer's contents
are not counted (lives in the locus's arena).
- Per-method scratch high-water mark — not modeled. The
scratch arena is destroyed at method exit, so contents
are transient, not resident.
- Unbounded-cap capacity slots (no `cap = N` on the form)
surface in `WorkingSetEstimate::unbounded_slots` rather
than contributing zero silently.
> Forward-looking / deferred items for this area now live in the
> decision log — see [`decisions.md` § Deferred & future
> work](./decisions.md#deferred--future-work).
## Verification responsibilities
Where each typing rule lives in the compiler pipeline:
- **Parse + AST construction**: grammar.ebnf rules.
- **Name resolution**: identifier scopes, qualified-name lookup.
- **Type inference**: let-binding types, fn return types when
explicit `-> T` omitted.
- **Type checking**: assignment compatibility, function call
signature, generic instantiation, projection-class
constraints.
- **Locus-shape checking**: contract compatibility (F.8 / F.14),
k_max bounds (F.1 / runtime-checked when dynamic), mode
signature consistency, lifecycle method signatures.
- **Region-lifetime checking** (compile-time): no escape from
shorter-lived to longer-lived scope; no sibling references.
- **Closure-cycle existence check**: closure assertions
reference defined values in the closure's scope.
The Phase 1 compiler in Rust implements these checks; the
Phase 6 self-hosted compiler ports them.
================================================================================
## Memory model
## source: spec/memory.md
================================================================================
# Memory model
This document specifies Hale's memory model: how regions are
allocated, organized, and freed; how locus structure constrains
memory layout; how access between loci is mediated.
> **Naming note:** The language is **Hale**; the runtime
> substrate is called **lotus** ("lotus structure provides the
> memory hierarchy" below refers to the substrate concept, not
> the language). C-runtime symbols stay `lotus_*`.
## Foundational rule
**A locus owns a region. The region's lifetime is the locus's
lifetime. When the locus dissolves, the region is freed
wholesale.**
There is no garbage collector. There is no borrow checker. The
lotus structure provides the memory hierarchy for free; the
allocator just respects it.
## Hierarchy
Regions form a tree. The runtime root locus has a top-level
region. Each child locus's region is a *sub-region* of its
parent's region.
```
root region
├── main's implicit-locus region
│ ├── coordinator region
│ │ ├── leaf-1 region
│ │ ├── leaf-2 region
│ │ └── leaf-3 region
│ └── attribution-service region
│ └── ...
└── runtime services regions
```
A region's allocations are bounded above by its own size; sub-
regions allocated within count against the parent's budget.
Regions cannot escape their hierarchy: a value allocated in
leaf-1's region cannot be referenced by leaf-2 (lateral access
is blocked at the type system level; physical layout makes it
moot anyway since wholesale dissolution would invalidate cross-
region references).
## Per-projection-class allocation
A locus's projection class (rich / chunked / recognition) is
a **perspective-resolution commitment** — a declaration of
what observation granularity the locus serves to perspectives
one tower up. Rich serves named-child observation (fine
resolution); chunked serves chunk-level observation (mid
resolution); recognition serves population-level observation
(aggregate resolution — "represent as a curve," "as a
histogram," "as a count"). The allocator strategy below is a
*consequence* of the resolution commitment, not the
commitment itself — same syntactic source, three generated
implementations chosen to make observation at the declared
resolution cheap:
| Projection class | Allocator | Coordinatee model |
|---|---|---|
| **Rich** (proj_rich, N≈4–10) | Per-locus bump arena. | Coordinatees attached as small array; full state per coordinatee; freed wholesale on locus dissolution. Low churn expected. |
| **Chunked** (proj_chunked, N≈10–30) | Per-locus arena with **per-coordinatee sub-regions**. Each accept allocates a sub-region; each dissolution frees one wholesale. Bookkeeping slots reclaimed via free-list. | Typed-message-header per coordinatee; moderate churn supported. |
| **Recognition** (proj_recognition, N≈100–500) | Sub-mode-typed recpool selected at the locus declaration site (`fixed_cell`, `shared_slab`, `spillover`, or `summary_only`). v1 ships `fixed_cell` (bitmap-tracked fixed cells; child's arena lives inline in the cell) and `shared_slab` (one bump arena shared across all children; per-child release is a no-op). See **Recognition sub-modes** below. | Summary-only per coordinatee; many supported; minimal per-coordinatee state. |
The `N≈…` figures are the *expected* coordinatee counts each
class is tuned for, not hard caps. The `for child in
self.children` iteration handle list — the codegen-level array
of accepted child `self_ptr`s, distinct from where coordinatee
state lives above — is a heap buffer that grows geometrically
(`lotus_children_push`), so a parent may accept arbitrarily many
children regardless of class. (Before 2026-05-29 this was a
fixed 16-slot inline array with no bounds check; accepting a
17th child while iterating `self.children` corrupted adjacent
struct memory.) Recognition's `fixed_cell` cap is the one real
hard limit, and it is an explicit, declared budget enforced as a
runtime error — not an implementation ceiling.
The compiler picks based on the locus's declared projection
class. A locus with no explicit `projection` annotation defaults
to `chunked` if it accepts coordinatees, `rich` if it doesn't.
Recognition is **explicit-only** — there is no implicit
recognition fallback, and v1.x-3 made the sub-mode commitment
required at the declaration site (bare `: projection recognition`
is a parse error). Same forcing-function discipline as the
2026-05-12 two-channel rule: the substrate doesn't pick a default
for you.
### Recognition sub-modes (v1.x-3)
A locus annotated `: projection recognition(cap=N, )`
commits to a storage discipline for its accepted children at
the declaration site. The cell stride (`K` in the table below)
is *not* a user knob — it is derived at codegen time from the
union of accept-method param types on the parent locus, taken
as the max. The contract the author writes is "how many
children, what discipline"; the layout is the compiler's job.
v1 ships two sub-modes; the other two parse + typecheck but
reject at the resolver with a "v1.x pending" diagnostic.
| Sub-mode | Commitment | Backing | Per-child release |
|---|---|---|---|
| `fixed_cell` | "Each child fits in a cell sized for the accept-type union. Cap of N children. Overflow is a hard runtime error." | `lotus_recpool_fixed_*`. One contiguous block of `N × stride` bytes; each cell holds an inline `lotus_arena_t` + chunk header + payload. Bitmap-tracked acquire/release. The cell IS the child's arena. | Clears the bitmap bit. Slot is reusable. |
| `shared_slab` | "All children share a single bump arena sized for the accept-type union × cap. The whole slab frees at parent dissolve." | `lotus_recpool_slab_*`. One `lotus_arena_t` with a single fixed-size chunk; `fixed_size=1` so it never grows. Every acquire returns the SAME arena pointer — sibling allocations interleave. | No-op. Slab freed wholesale at parent dissolve. |
| `spillover` *(v1.x pending)* | "Each child fits in a cell; overflow malloc-fallback with one-time warning. Graceful degradation under load." | Future: `lotus_recpool_spillover_*`. Per-cell `fixed_cell` plus a heap-allocated fallback. | TBD. |
| `summary_only` *(v1.x pending)* | "Children carry zero per-instance state; all allocations live in the parent's arena." | Future: type-system rule prohibiting child arena allocation; parent's `__arena` is the only storage. | No-op. |
The arena handle returned by `lotus_recpool_fixed_acquire` /
`lotus_recpool_slab_acquire` is a `lotus_arena_t*` so child
body code stays projection-class-agnostic per the F.22
architectural invariant. Overflow on the child's
`arena_alloc` returns NULL (the arena's `fixed_size=1` flag is
honored); v1.x-3 wires that to a hard NULL return — routing
through `lotus_root_panic` for value-error escalation is a
future polish.
The codegen dispatch at child dissolve picks the matching
`release` fn via a synthetic `__recpool_release_kind: i64`
discriminator on every locus struct (0 = regular
`lotus_arena_destroy`, 1 = `lotus_recpool_fixed_release`, 2 =
`lotus_recpool_slab_release`). Set at the parent's accept
step; consumed at child dissolve. Uniform layout so the
dissolve path doesn't branch on whether the locus opted into
the recognition surface.
## Capacity slots (F.22)
A locus's storage discipline is an **N-tuple of capacity slots**.
Slot 0 is the locus's own Arena (everything above this section).
Slots 1..N are user-declared in a `capacity { ... }` block:
```
locus Foo {
capacity {
pool entries of Int; // slot 1: cell-recycling of Int-sized
heap registry of Command; // slot 2: growable, individual free
}
}
```
Each non-Arena slot kind is a commitment the locus makes about
its own state, not a hidden implementation detail:
| Slot kind | Commitment | Backing | Lifetime |
|---|---|---|---|
| **Arena** (slot 0, implicit) | "I'm scratch — everything I touch dies with me." | Single bump arena per locus. | Wholesale-free at locus dissolve. |
| **Pool of T** (slots 1..N) | "I hold a bounded shape of recyclable state." | Chunked free-list (`lotus_pool_*`). Cells acquired / released; chunks grow geometrically (16 → 32 → 64, capped at 4096) when the free-list empties. | Wholesale-free at locus dissolve. |
| **Heap of T** (slots 1..N) | "I hold growable state bounded by my own lifetime." | Doubly-linked live list with intrusive header (`lotus_heap_*`). Cells alloc / free individually. | Wholesale-free at locus dissolve (live list walked, every still-live cell freed). |
Slot init runs in declaration order after slot 0; slot destroy
runs in reverse declaration order before slot 0. Cell alignment
is 8 bytes (the v0 substrate's universal scalar alignment); cell
size comes from T's LLVM struct layout. Restriction: a cell
type cannot be a `LocusRef` — locus membership goes through
`accept(c: Child)`, not slots. See `spec/semantics.md`
"Capacity slot lifecycle and dispatch (F.22)" for the user-
facing method-shaped surface and full restriction list.
**Slot 0 parent-override** is governed by projection class
(table above): Chunked / Recognition parents sub-region-allocate
their accepted children's slot 0; Rich parents do not. F.22
names this existing v0 behavior so future **slot 1..N parent-
override** (`pool entries of Int as_parent_for Child;`) sits
on consistent vocabulary. Slot 1..N parent-override is deferred
to v1.x — the first workload that demands a parent-owned Pool
shared across accepted children will unlock the syntax.
**Naming note.** The Recognition projection class's recpool
(see § "Recognition sub-modes (v1.x-3)" above —
`lotus_recpool_fixed_*` / `lotus_recpool_slab_*` per the chosen
sub-mode) is the *slot 0 storage strategy for recognition-
classed loci*. F.22's `pool` slot is a user-declared slot at
1..N with chunked-+-free-list backing and no projection-class
entanglement. The two systems may unify in v1.x once F.22
slots 1..N stabilize; until then they are structurally
distinct mechanisms that happen to share the word "pool."
## Lifetime rules
### Bound handles
```
let h = Locus { ... };
// h is bound. Locus lives until `h` goes out of scope.
// Then: drain() runs (cascades), dissolve() runs, region freed.
```
### Unbound expressions
Per design-rationale §A:
- **Ephemeral** (only `birth` + `params`, no ongoing-work surface):
dissolves at the enclosing statement boundary.
- **Long-lived** (has `run`, bus subscriptions, mode declarations
callable from outside, or any post-birth work surface):
becomes anonymous child of enclosing scope; lives until scope
dissolves.
### Lifecycle methods
Lifecycle methods (`birth`, `accept`, `run`, `drain`, `dissolve`,
`on_failure`) **do not have their own implicit locus** (per F.6).
They run *as the locus*. Locals in their bodies are bound to the
lifecycle method invocation; child loci instantiated in their
bodies attach to the enclosing locus, not to the lifecycle
method's scope.
### Free `fn` functions
`fn main()`, `fn helper()`, etc. — every free function has its
own implicit locus (per §D). Bound handles and anonymous
children are children of that implicit locus; the function
returns when:
- Body's last statement completes, AND
- All children of the implicit locus have dissolved.
### Value allocations vs. the free-fn implicit locus
The free-fn implicit locus governs the lifetime of **child loci
and bound handles** instantiated in the body — those dissolve at
the function's return. It does **not** give the function a
value-allocation arena of its own.
Ordinary **value allocations** (struct / record literals,
`String` concatenation, array and `Bytes` literals) made in a
free `fn` body bump into the nearest enclosing **locus's** region
and are reclaimed only when that locus dissolves — **not** when
the function returns. A value allocated inside a helper `fn`
called from a hot loop therefore accumulates in the caller's
region exactly as if it had been written inline; the function
boundary is not a reclamation boundary for values.
The practical consequence: a value allocation inside an
unbounded loop (a `run()` / bus-handler loop, a `while true`)
grows the locus region without bound for the locus's lifetime —
the recurring leak class behind several past OOMs. To bound it,
either bound the loop, route the value over the bus (the payload
arena reclaims per dispatch), or move the allocating work into a
child locus that dissolves per iteration. The compile-time
analysis that surfaces this is GitHub issue #18 item 1
(memory-bound proofs); see `spec/verification.md`.
## Bookkeeping reclamation (per-arena defrag)
Per F.3: within a parent's arena, dissolved-coordinatee
bookkeeping slots are reclaimed via a **per-arena free-list**
(chunked-class loci) or **periodic defrag** (high-churn).
Reclamation is:
- **Per-arena** — never crosses arena boundaries.
- **Bounded** — reclamation work is O(slots-being-reclaimed),
not O(heap).
- **Deterministic** — no stop-the-world, no opaque tracing.
Coordinatee sub-regions remain pristine arenas freed wholesale
on dissolution; only the parent's *bookkeeping* about
coordinatees (registry slot, dispatch entry) needs free-list
reclamation.
## Drain cascade
Per F.4: `drain()` always cascades depth-first.
1. Recursively call `drain()` on each child first.
2. Wait for all children to finish draining.
3. Drain self.
There is no separate `drain_cascade()` syntax — `drain()` is
*always* cascading. SIGINT triggers `drain()` on the runtime
root, cascading through the whole process tree.
This implies that during drain:
- New child accepts are refused.
- In-flight messages on bus subscriptions are delivered;
no new messages accepted.
- Any in-flight handler invocations complete.
- After in-flight work completes, dissolve runs; region freed.
## Mode-projections share the arena
Per F.5: a locus's three modes (bulk / harmonic / resolution)
operate on the same locus state via the same arena. Generated
code reads/writes one region across three implementations. No
duplicate allocation, no copy.
The compiler verifies that the modes don't write-conflict
(resolution-mode mutating state that bulk-mode also writes
during overlapping evaluation is a compile-time error if the
writes would race).
## Inter-locus access
### Vertical-only at the memory level
A locus L can read into a coordinatee C's region via the
contract surface (per F.7, F.8, F.11, F.14). C cannot read
into L's region except via the contract's `consume` declarations.
Siblings cannot read each other (vertical-only flow expressed
at the memory layer).
Practically:
- L → C: typed contract field access (`c.greeting`); routed
through C's translation function (per F.14); cost reflects
C's projection class.
- C → L: only what L declares in `consume`; goes back through
the contract; never direct address.
- C1 ↔ C2 (siblings): not permitted. Lateral coordination
flows through the parent (typed message via bus, or
contract-mediated).
### Cross-locus copies, not pointers
A typed message crossing a locus boundary is a **copy**, not a
pointer. The bus message arrives in the receiver's arena; the
sender's state is independent. This is required because:
- Sender and receiver may be in different schedulers (per the
cooperative scheduler model in `runtime.md`).
- Sender may dissolve before receiver finishes processing the
message; pointer-based access would dangle.
The framework's vertical-only-flow expresses itself at the
memory level as: pointers don't cross loci; values do.
## Region escape rules (forbidden patterns)
The compiler rejects these at compile time:
1. **Returning a sub-region pointer from a longer-lived scope.**
`let r = make_region_in_child(); use(r);` where `r` outlives
the child is a region-escape error.
2. **Storing a child reference in a parent or sibling that
outlives the child.** Triggers a region-lifetime check.
3. **Sibling-to-sibling reference.** No type rule permits this;
the compiler emits a clear "vertical-only flow" error.
## Edge cases
### Failure during birth
If a locus's `birth()` panics or otherwise fails, the region is
freed wholesale; no `dissolve` runs (since dissolve assumes
birth completed). The parent's `on_failure` receives the
failure event.
### Failure during accept
Per F.7, `accept()` runs before child region allocation. If
accept rejects (panics, returns error), the child region is
never allocated — no cleanup needed.
### Failure during run
Mid-`run()` panic triggers the parent's `on_failure(self,
StructuralFailure { ... })`. Parent decides recovery
(`restart`, `quarantine`, `bubble`, `dissolve`); region is
freed per the recovery primitive's rules.
### Closure violation at dissolve
Per F.9, a closure violation at the `dissolve` epoch is an
**audit failure** (explosion), not a structural failure. The
locus's region is freed regardless; the parent receives
`on_failure(self, ClosureViolation { ... })` with typed event
data.
### Quarantine
A quarantined coordinatee retains its region (the parent has
chosen to preserve it for inspection). Region is freed only
when explicitly `dissolve(child)`d or `restart`ed. Quarantine
is the one case where a "dissolved-from-the-system" coordinatee
keeps its region alive.
## Allocator implementation outline
(Informative; specifies expected behavior, not the literal
implementation.)
### Rich
```
struct RichArena {
bump_ptr: *mut u8,
end_ptr: *mut u8,
coordinatees: [Option; MAX_RICH_N],
}
```
Single bump arena. Allocations are pointer-bumps. Dissolution
resets bump_ptr to start.
### Chunked
```
struct ChunkedArena {
parent_bump: BumpAllocator,
coordinatee_subregions: Vec,
bookkeeping_freelist: FreeList,
}
struct SubRegion {
bump_ptr: *mut u8,
end_ptr: *mut u8,
coordinatee_id: u32,
}
```
Each accepted coordinatee gets a sub-region (pristine bump
arena). Bookkeeping (`coordinatee_id`, dispatch slot) lives in
the parent's bump area; reclaimed via free-list when
coordinatee dissolves.
### Recognition
Sub-mode-typed at the locus declaration site
(`recognition(cap=N, )`); v1 ships two sub-modes:
```c
/* fixed_cell — bitmap-tracked cells; the cell IS the child's
* arena. Per-child release clears the bit. The compiler
* computes cell_bytes from the parent's accept-method type
* union; user code only spells cap. */
struct lotus_recpool_fixed {
size_t cap_count, cell_bytes, cell_stride, bitmap_words;
uint64_t *bitmap;
char *cells; /* cap_count * cell_stride bytes;
* each cell holds an inline arena
* header + chunk header + payload */
};
/* shared_slab — one shared arena; every acquire returns the
* same pointer; per-child release is no-op. Slab size also
* derived from the accept-type union × cap. */
struct lotus_recpool_slab {
size_t cap_count, slab_bytes;
lotus_arena_t *slab_arena; /* fixed_size=1 */
};
```
Allocation is a bitmap search (fixed_cell) or a regular arena
bump (shared_slab); no dynamic memory in steady state for
either. See `crates/hale-codegen/runtime/lotus_arena.c` for
the canonical implementation and § "Recognition sub-modes
(v1.x-3)" above for the per-sub-mode contract.
## What the compiler emits
For each locus, the compiler generates:
1. A region-allocation function (per projection class) called
at birth.
2. A drain handler that walks children depth-first.
3. A dissolution handler that releases the region wholesale.
4. Per-mode implementations that read/write the locus's region
in-place.
5. Translation function entries (per F.14) accessible through
the arena.
The runtime provides the underlying bump allocators, free-list
machinery, scheduler integration, and lifecycle dispatcher.
**Arena alignment contract.** `lotus_arena_alloc(a,
size, align)` returns a pointer whose address (not the within-
chunk offset) is aligned to `align`. The chunk header
(`{next, used, cap}` = 24 bytes on x86_64 LP64) sits before the
data region, so a naive "align the offset" approach yields
8-byte-aligned pointers even when callers ask for 16. The
correct shape: align the cursor address `(c+1) + c->used` to
`align`, then convert back to a within-chunk offset. The codegen
side passes `align = 16` from `arena_alloc` to cover the widest
scalar type (i128 / Decimal) — earlier the codegen passed 8 and
the C side only aligned the offset, leading to `movaps` segfaults
on Decimal stores into struct fields (i128-alignment segfault,
root cause for two downstream repros). Both layers are necessary: the codegen
must ask for the natural alignment of the widest scalar it can
emit, and the C arena must honor that alignment at the pointer
level, not the offset level.
The same discipline governs the **bus payload path**, which uses
distinct allocation sites. (a) The mailbox cell
`lotus_bus_cell_t.payload_inline` is forced to 16-byte alignment
via a struct attribute: a *pinned* subscriber is handed
`&cell.payload_inline` directly (unlike a cooperative drain, which
copies into a 16-aligned scratch), so an 8-aligned cell — its
widest natural member is a pointer — traps a whole-`Decimal`-payload
copy (an aligned `vmovaps`) even though at `-O3` LLVM scalarizes
individual i128 *field* ops into misalignment-tolerant paired
64-bit moves. (b) The wire *deserialize* allocations for nested
`TypeRef` (struct) fields request `align = 16`, matching the
fixed-size-array element path, since a nested struct may carry a
`Decimal`. A payload/cell allocation must request the natural
alignment of the widest scalar the struct can hold — never a
hardcoded 8.
## Codegen ABI (v0)
The native-codegen path (`hale build`) lowers each locus to an
LLVM struct one field per declared param, and each lifecycle
method to an LLVM function whose first parameter is a pointer to
that struct. Field reads / writes via `self.X` lower to
`getelementptr` + `load` / `store` against the `self_ptr`. This
is the substrate the region allocator + scheduler will sit on top
of when they land — the ABI is the load-bearing contract; the
allocator and dispatcher refine *where* the struct is allocated
and *how* methods get scheduled, not the struct's shape.
```
locus Greeter {
params { greeting: String = "hi"; }
contract { expose greeting: String; }
}
locus Coord {
params { factor: Int = 1; }
contract { consume greeting: String; }
accept(g: Greeter) { ... }
run() { ... }
}
```
lowers to:
```
%locus.Greeter = type { ptr } ; greeting
%locus.Coord = type { i64 } ; factor
declare void @Coord.accept(ptr %self, ptr %child)
declare void @Coord.run(ptr %self)
```
Statement-level instantiation `T { ... };` lowers to: `alloca`
on the caller's stack, store each field (call-site override or
declared default), then dispatch lifecycle methods in the F.7
order:
1. **If we're inside a parent locus's lifecycle method AND that
parent has an `accept(child: T)` method matching the locus
being instantiated** → call `parent.accept(parent_self, child_ptr)`.
2. Call `T.birth(child_ptr)` if declared.
3. Call `T.run(child_ptr)` if declared.
4. Call `T.drain(child_ptr)` if declared.
5. Call `T.dissolve(child_ptr)` if declared.
`accept` runs *before* the child's own `birth`, per F.7. This
is how `02-parent-child`'s `Coord.accept(g: Greeter)` fires for
each `Greeter { ... }` instantiated in the coordinator's `run()`
body. Inside `accept`, `self.X` GEPs through the parent's struct
and `g.X` GEPs through the child's struct — different `getelementptr`
chains, same lowering machinery.
`drain` / `dissolve` run last, in that order, before the alloca
dies. The F.4 depth-first cascade is implicit in v0's
synchronous-instantiation model: any descendants instantiated
inside this locus's `run()` body have already gone through their
own full birth → run → drain → dissolve sequence (each via this
same lowering, recursively) before `run()` returns. So when this
locus's `drain()` fires, all descendants are already gone — no
explicit cascade walk is needed at the substrate level. When the
cooperative scheduler lands and loci can be long-lived, the
cascade becomes explicit; the lifecycle-method ABI doesn't
change.
v0 codegen is **ephemeral-only**: every alloca is on the caller's
stack and freed when the enclosing fn returns. Long-lived loci
and the parent-child region hierarchy described above (each
child's region nested in the parent's) wait on the cooperative
scheduler + region allocator work.
Constraints v0 codegen enforces (will relax as more lands):
- Lifecycle methods supported: `birth`, `accept`, `run`,
`drain`, `dissolve`.
- `birth`, `run`, `drain`, `dissolve` take no user-declared
params (only implicit `self`); `accept` takes exactly one
param, the typed child reference. All lifecycle methods
return `void`.
- Locus param defaults must be literals (Int / Float / Bool /
String / Duration). Non-literal defaults are rejected.
- Contracts are typecheck-only at this layer — they're accepted
in the AST and skipped by codegen. The expose / consume
surfaces are still type-checked across coordinator / coordinatee
per F.8 by the typechecker pass.
- Locus members beyond `params`, `contract`, the five-method
lifecycle set, `bus { subscribe / publish }` declarations, and
`fn` members (used as bus handlers) are rejected at declare
time. Modes, closures, failure handlers, and nested
consts/types wait on later milestones.
The struct ABI + accept + drain/dissolve dispatch is what makes
`01-locus-with-run`, `02-parent-child`, `10-stateful-locus`, and
`11-drain-dissolve` compile to native ELF.
### Bus router (m12)
The bus router lowers as **one global subscription table per
program**, sized at compile time from the total `bus subscribe`
declaration count. Layout:
```
%lotus.bus_entry = type { ptr, ptr, ptr } ; subject, self, handler
@bus.entries = internal global [N x %lotus.bus_entry] zeroinitializer
@bus.count = internal global i64 0
```
Each `bus subscribe "S" as h ...` declaration on a locus
contributes one slot; registration happens when the locus is
instantiated, BEFORE its `birth()` runs:
```
@bus.entries[bus.count] = { @.str.S, %self_ptr, @.h }
bus.count += 1
```
`<-` lowers to a call into the generated dispatch fn:
```
define void @lotus.bus_dispatch(ptr %subject, ptr %payload) {
; for i in 0..bus.count:
; if strcmp(bus.entries[i].subject, %subject) == 0:
; bus.entries[i].handler(bus.entries[i].self, %payload)
}
```
Subject equality uses libc `strcmp` (subjects are NUL-terminated
global strings). Handler functions are called through a
type-erased function pointer — every bus handler has the same
LLVM signature `void (ptr self, ptr payload)`, with payload
typing enforced by the typechecker upstream.
#### Long-lived locus deferral
A locus with any `bus subscribe` declaration is **long-lived**:
its drain/dissolve must NOT fire at the end of its instantiation
expression (which would dangle its `self_ptr` in the bus table
before later publishes can reach it). Instead, each fn body /
lifecycle method body opens a deferred-dissolve frame; long-lived
loci instantiated inside push their `(self_ptr, locus_name)` onto
the frame; at body exit (just before `ret`) the frame is flushed
in reverse instantiation order, calling drain → dissolve on each.
Ephemeral loci (no subscribe) at *statement-position* keep the
original semantics: drain → dissolve fires at end of
`lower_locus_instantiation`, inside the same lifecycle body
that instantiated them. m82 changed the *let-bound* case:
`let h = LocusName { ... }` now defers dissolve to the
enclosing fn's scope-exit flush instead of the struct-literal
boundary, so the user-visible binding stays valid for
subsequent method calls. Long-lived loci (with `bus subscribe`)
continue to defer regardless of binding shape. See
`spec/semantics.md` "Dissolve timing rules" for the full rule.
The F.4 cascade still falls out structurally — children
dissolve before their parent, regardless of which mechanism
handles each.
### Region allocator substrate (m19)
The codegen path links a small C arena runtime
(`crates/hale-codegen/runtime/lotus_arena.c`, bundled into
the compiler via `include_str!`) into every emitted binary.
Public ABI:
```
ptr lotus_arena_create(void) // new arena
ptr lotus_arena_alloc(ptr arena, i64 size, i64 align) // bump
void lotus_arena_destroy(ptr arena) // wholesale free
```
An arena is a linked list of bump chunks (default 64 KiB each;
oversized requests get a fresh chunk sized to fit). Allocation is
a pointer-bump in the head chunk; on overflow, a new chunk is
malloc'd and pushed to the front. Destruction walks the list and
frees every chunk wholesale — no per-object free, ever.
### Locus-owned arenas + bus copy semantics (m20)
Every locus struct carries a synthetic `__arena: ptr` field at
**struct slot 0**. Initialized at instantiation time (right after
the `alloca`) via `lotus_arena_create()` and torn down via
`lotus_arena_destroy(self.__arena)` after the user's `dissolve`
method runs (in both the ephemeral path and the deferred
long-lived-locus flush at body exit). Per spec: "A locus owns a
region. The region's lifetime is the locus's lifetime."
The arena field's fixed-offset placement is load-bearing for the
bus path: `lotus.bus_dispatch` is type-erased — it sees only
`ptr self` from the subscription table — so its only way to find
the subscriber's arena at runtime is a fixed-offset load. Slot 0
makes that a constant GEP.
Allocation routing inside codegen has three tiers, in order:
1. **`current_arena_override`** — set during locus-instantiation
field init so composite-literal defaults / overrides land in
the new locus's arena (rather than the parent's arena where
the default expression literally executes).
2. **`current_self`'s arena field** — when we're inside a
lifecycle method body (or any fn with a `current_self`
binding), allocations go to the locus's own arena.
3. **`@lotus.arena.global`** — fallback for `main` and free fns,
which have no enclosing locus. Initialized in main's prelude;
destroyed at every `ret` from main.
The three tiers above route *value* allocations (composite
literals, strings, payload copies). The **locus-struct**
allocation (where a freshly-instantiated locus's own struct
lives) is a separate dispatch in `lower_locus_instantiation`,
ordered: returned/payload-routed → lazy global payload arena;
`parent_accepts_us` → the accepting parent's arena;
**`parent_owns_via_field`** → the owning locus's arena (see
below); otherwise → entry-block stack alloca.
#### Owned param-field child allocation
An F.29 owned param-field child (`parent_owns_via_field` — e.g.
`locus PerConn { params { reader: ConnReader = ConnReader { }; } }`)
allocates its struct in the **owner locus's arena**, taken from
`current_arena_override` (which the owner's instantiation set to
its own arena before running params-init, where the field
default is constructed). The alignment passed is **16** (the
widest scalar — i128 / `Decimal`), per the Arena alignment
contract above.
Before this, such children fell through to a stack alloca in the
instantiating method's frame. That was a latent dangle on any
cross-lifecycle read of `self.children[i].` (owner
birthed in one method, the field read in another) and became a
hard crash once an owner's `run()` is posted to a cooperative
pool (`spec/runtime.md` § "Runtime pool inheritance"): the
posted `run()` executes after the instantiating frame returns,
so the field-child's stack slot is gone — garbage field reads,
or a segfault on the first parking `recv` in a field-child
method one locus-boundary deep. Allocating in the owner's arena
makes the whole owned subtree share the owner's lifetime
(wholesale-freed with the owner's arena), fixing the dangle
without leaking into a longer-lived parent arena.
(Per-child reclamation, 2026-05-30. An `accept`'d child whose
type is a **flow** — some parent declares `release(c: Child)` —
is reclaimed when its `run()` completes (or it calls
`terminate;`): drain → `release(owner, child)` → dissolve →
`__arena` reclaim, on the child's own pool worker while the
parent lives. So a daemon that accepts one flow child per
connection reclaims each as its connection ends — bounded, no
per-instance accumulation. A **resident** child — a type no
parent `release`s — still lives until the parent dissolves (it
is meant to: e.g. a fixed cohort of subscribers). The earlier
"a daemon accepting many children leaks until process exit" held
only when neither reclamation trigger applied; declaring the
flow's `release` closes it. See spec/semantics.md § "release(c)
and flow children".)
#### Accept'd-child struct recycling
The child's **locus struct** (as opposed to its `__arena`
contents) lives in the OWNER's arena — that's what keeps
`owner.__children` reads valid cross-lifecycle. Arena
allocations are never individually freed, so per-child
reclamation alone left ~sizeof(child struct) pinned in the
owner's arena per child ever accepted: a churn daemon grew
O(total children), violating F.3's O(peak-alive) intent
(measured ~110 B/child; 443 MB at 4M children).
The substrate closes this with an intrusive per-owner free-list
of dead child structs, threaded through the owner's arena
struct (`child_struct_free`, guarded by `subregion_lock` under
multithreading):
- `lotus_child_struct_release(owner_self, child, size)` — called
from the teardown chokepoint (`emit_locus_arena_destroy`)
after the `__arena` NULL-latch, exactly once per reclaimed
child (the latch gates it). Node layout inside the dead
struct: offset 0 is never written (it is the child's own
NULL'd latch), offset 8 holds `next`, offset 16 the block
size. Covers subregion-owning children AND arena-elidable
(empty-lifecycle) children, which share the parent's arena
pointer and latch on their (otherwise unused) `__arena` field.
- `lotus_child_struct_alloc(owner_arena, size, align)` — the
instantiation front: pops a size-matched block (bounded
first-fit, ≤8 probes) before falling back to the bump
allocator. Alignment is **16** per the Arena alignment
contract (was 8 — a latent `Decimal`-param `movaps` trap).
Steady-state churn (accept → run → reclaim per event) therefore
reuses one struct slot per concurrent child, per type. Resident
children don't route through mid-life reclaim and are unaffected.
Bus dispatch implements the spec's copy-not-pointer semantic:
```
void lotus.bus_dispatch(ptr subject, ptr payload, i64 size):
for i in 0..bus.count:
if strcmp(bus.entries[i].subject, subject) == 0:
sub_self = bus.entries[i].self
sub_arena = load (sub_self + 0)
copy = lotus_arena_alloc(sub_arena, size, 16)
memcpy(copy, payload, size)
bus.entries[i].handler(sub_self, copy)
```
Each `<-` call site passes the payload's compile-time-known
struct size as a third arg. The subscriber's handler receives a
pointer into the subscriber's own arena, valid until that
subscriber dissolves — independently of when the publisher's
locus dissolves. This unblocks `self.current_kernel = msg`
patterns where the subscriber stores a payload reference across
multiple bus events (fitter-applier-demo's central pattern).
m20 deliberately keeps free fns + main on the program-wide arena
(no per-call arena yet) and doesn't yet specialize per projection
class — chunked-class per-coordinatee sub-regions land in m22,
the recognition-class fixed pool in m23.
**Phase-3 hard byte-cap on `g_bus_payload_arena` (2026-05-19;
safety net).** The arena now refuses to grow past
`LOTUS_BUS_PAYLOAD_ARENA_CAP` (default 64 MiB, env-overridable
for capacity-planning experiments). When the cap fires
`lotus_arena_alloc` returns NULL; one diagnostic line goes to
stderr identifying the cap event and the arena's name; subsequent
allocations against the capped arena keep returning NULL.
Existing callers — BytesBuilder `snapshot()` / `finish()` via the
alloc-fail sentinel + violate routing, recv_bytes returning
empty Bytes, `lotus_bytes_create` returning NULL through
`empty_global` — already surface NULL as degraded service, so the
cap converts a slow OOM into structural failure that surfaces
through the F.27 channel. This is the floor for a long-running
program leaking into the payload arena, not the fix; the fix is
per-subscriber arena routing for m70 + `__caller_arena` threading
for the stdlib primitives that land here.
**Phase-3 Task 11 intra-process bus per-subscriber routing
(2026-05-20).** Extends Task 9's per-sub arena pattern to the
intra-process `<-` path. Previously `lotus_bus_dispatch` enqueued
the publisher's struct bytes verbatim into each subscriber's
queue cell — payload String / Bytes pointers stayed aliased to
the publisher's locus arena. For long-running publishers (a high-rate
normalizer class) that meant an unbounded leak in the publisher's
locus arena (the per-arena cap from Task 10 doesn't apply to
locus-owned arenas), bounded only by the publisher's eventual
dissolve — which for a daemon's root locus never happens.
The fix: when the codegen has synthesized a wire codec for the
payload type (the common case — every `<-` payload type gets
one), the dispatcher serializes the publisher's struct to wire
bytes once, then routes through `lotus_bus_dispatch_wire`. The
wire path's per-subscriber TLS routing rebuilds the struct in
each subscriber's own `__arena`; payload pointers end up bounded
by the subscriber's lifecycle. Cost: one serialize + N
deserializes per publish (N = matching subscribers). For
cooperative-only programs with no remote subs, the
previously-skipped serialize work is now paid on every publish.
For programs with both local and remote subs, the serialize cost
is amortized (same wire_buf feeds both).
A payload-typeless subject (no codegen-synthesized wire codec —
the `serialize_fn` arg is NULL) falls back to the legacy
verbatim enqueue, preserving the pre-Task-11 v1 behavior. This
escape hatch is intentional: it lets a hot-path subject opt out
of the round-trip cost when the publisher controls all
subscribers and can guarantee the payload's pointer-aliasing
discipline.
**Phase-3 Task 9 m70 per-subscriber arena routing.**
`lotus_bus_dispatch_wire` no longer parks deserialized String /
Bytes pointers in the program-lifetime g_bus_payload_arena.
Instead it iterates the matching subscribers, sets the TLS
caller-arena (Task 8 indirection) to each subscriber's own
`__arena` (via the m20 fixed-offset slot-0 GEP), and deserializes
the wire bytes per-subscriber into that arena. The payload
pointers in the enqueued struct_buf now alias the subscriber's
own arena, bounded by the subscriber's lifecycle — no
program-lifetime deposit, no eventual OOM.
Cost: deserialize is invoked once per matching subscriber rather
than once total. Acceptable for typical fan-out (1–3 subs per
subject); high-fan-out subjects pay a real bill that could be
optimized via deserialize-once-then-clone-per-sub if a workload
demands it.
Closes the original Phase-2 (4) investigation's finding ("not
reclaimable under current semantics"): the answer was never to
reclaim the global arena but to skip it entirely — the m20 spec
("each subscriber's arena outlives the payload pointer") now
holds by construction because the deserialize-time allocator IS
the subscriber's arena.
**Cross-thread wire cell per-delivery reclaim (2026-07-15;
refines Task 9 for the owner-routed path).** Task 9 deserializes
into the subscriber's arena on the *publisher's* thread. When the
target subscriber lives on a *different* thread (a `pinned`
publisher fanning out to a `where async_io` pool subscriber),
that write races the subscriber's own unlocked arena mutations —
so the owner-routed dispatch instead posts a *wire cell* carrying
the payload's wire bytes plus its deserializer, and the
subscriber's own thread deserializes it just before invoking the
handler (`lotus_bus_cell_materialize`). The first cut of that
path deserialized directly into the subscriber's locus arena —
correct for lifetime, but a per-delivery **leak**: on a
long-lived subscriber whose `run()` is parked (the canonical
server loop — an accept / recv that never returns), the locus
arena never dissolves, so every delivery's payload fields
(String / Bytes) piled up unboundedly (~one payload's heap per
message; measured ~320 MiB over 20k 16-KiB deliveries). The fix
deserializes each wire cell into a fresh **per-delivery
subregion** of the subscriber's arena, destroyed the instant the
handler returns (alongside the existing per-cell payload free).
This gives the deserialized payload exactly the lifetime the
same-thread (inline) delivery path already gives it — alive for
the handler, reclaimed after — and is safe for the
`self.current_kernel = msg` retention pattern because a
locus-field store deep-copies nested String / Bytes fields into
the *locus* arena (the only way user code can retain payload
data: address-taking is disallowed, so the payload pointer itself
never escapes the handler). A subscriber on the same thread as
its publisher, or a payload with no owned fields, is unaffected
(no wire cell, no subregion).
**Phase-2 (4) `g_bus_payload_arena` reclaim investigation
(2026-05-19; superseded by Phase-3 Task 9).**
The handoff posed: "should `lotus_bus_dispatch_wire`'s
`g_bus_payload_arena` deposit reclaim per dispatch since m20
memcpy's into subscriber arena anyway?" The answer is no, and
the reason exposes a load-bearing constraint.
m20's `memcpy(copy, payload, size)` is a flat struct copy: the
publisher's payload bytes (size = compile-time-known struct size)
land in the subscriber's arena. The struct's String / Bytes /
TypeRef fields are POINTERS inside that struct; the memcpy copies
the pointers, not the pointed-to bytes. For cross-process wire
dispatch (`lotus_bus_dispatch_wire` → deserialize → struct_buf →
`lotus_bus_local_dispatch`), the deserialized String / Bytes data
lives in `g_bus_payload_arena`. After m20's struct memcpy, the
subscriber's copy still aliases that arena.
Handler-side assignment (`self.foo = payload.string_field`) is a
pointer store — `lotus_str_clone` is invoked only at *free-fn
return* boundaries, not at struct-field assignment. So if the
handler retains payload fields on its own struct, the retention
extends the `g_bus_payload_arena` deposit's lifetime to the
subscriber's entire lifetime. Reclaiming per dispatch would dangle
the subscriber's retained pointers.
Enabling per-dispatch reclaim requires changing handler-side
String / Bytes assignment to clone-on-store from payload, OR
introducing a per-dispatch arena that's reset only after every
subscriber's handler has run — neither is a small change. For
v1 the arena grows unbounded for high-message-rate cross-process
subscribers; bounding it is forward work, not a follow-up to F.28
/ F.29 / F.27. Documented here so the next surface that asks
"can we reclaim per dispatch?" finds the previously-investigated
answer.
**Phase-4 per-method scratch reclaim.** Locus
method bodies (lifecycle `birth` / `run` / `accept` / `drain` /
`dissolve`, user-fn members, mode bodies) open a per-call
scratch subregion of `self.__arena` on entry, route transient
allocations through it via `current_arena_ptr()`, and destroy
the subregion at every return point — except a body proven to
allocate nothing that returns a by-value scalar (or Unit), where
the scratch is elided entirely. Eliding is sound
because there is nothing to reclaim and no return value to
deep-copy: it removes a `malloc`/`free` per call with no
observable change to lifetimes or bounds. Before this, every
allocation made by a long-running `run()` loop (JSON parse
strings, format-string concats, metric-label entries, every
stdlib primitive that lands on `lotus_caller_arena_or_global`)
landed in `self.__arena` directly — bounded only by the
locus's lifetime, which for a daemon's root locus or any
event-loop service is the entire process. A real workload
measured multi-MB/sec growth on a hot message-dispatch path
before the fix, OOM-killed at a typical container cap within
minutes. Post-fix the scratch resets each method call so
transient allocations have a bounded lifetime matching the
call's frame.
Two correctness invariants make this safe:
1. Heap-typed `self.X = expr` stores deep-copy `expr` into
`self.__arena` BEFORE the store, so the persisted
pointer outlives the scratch destroy. `String`, `Bytes`,
`TypeRef`, `Tuple`, `Array`, `Interface`, and
payload-bearing `Enum` are heap-typed; `BytesView` /
`StringView` / `LocusRef` / scalars / cells pass through.
The copier reuses `emit_return_value_deep_copy`. Bytes
now uses `lotus_bytes_clone` (was a pass-through under
the previous program-lifetime assumption — broken once
payloads can live in scratch).
2. Heap-typed return values from a method are deep-copied
into the caller's arena before the scratch destroy. The
caller publishes its `current_arena_ptr()` via
`lotus_set_caller_arena` immediately before each method
call (mirroring the stdlib primitive contract). The
callee snapshots `lotus_caller_arena_or_global()` at the
method's entry block into a fn-local alloca and uses
THAT snapshot — NOT a fresh TLS read at exit — as the
deep-copy destination. The snapshot dance avoids a
subtle bug where any nested method call inside the body
would clobber TLS, leaving the epilogue to deep-copy
into the wrong arena (whichever nested callee was
called last).
Cost: two mallocs (subregion arena struct + initial 64 KiB
chunk) and two frees per method call. For typical short
methods this is roughly 100–400 ns of overhead per call; at a
typical hot-path rate of dozens of methods per dispatched
frame at ~70 frames/sec, that's ~7 µs per second of aggregate
overhead — invisible next to the JSON parse / decimal
arithmetic the methods do.
An `lotus_arena_reset` primitive (subregion reuse: keep
chunks, set `used=0`) could amortize this to ~5–10 ns per
call but is deferred — the leak's the load-bearing bug; the
fast-path optimization can follow.
Locus instantiation routing is unchanged. Child locus structs
allocate via their own routes (parent arena if parent accepts
them, lazy-global if the fn returns the child type, otherwise
stack alloca) — `lower_locus_instantiation` doesn't read
`current_arena_ptr()`. So a method body that does `let _w =
ChildLocus { };` still gets the child instantiation routed to
the parent's arena, not scratch, and the deferred-dissolve
mechanism continues to govern child teardown.
The free-fn path (`lotus_arena_create_subregion(__caller_arena)`
at entry, destroyed at exit, with allocations routed to
`__caller_arena` directly post-cross-seed-segv-fix) is
unchanged. Free fns called from inside a method body get
`__caller_arena = method's scratch` — anything they alloc
lives in the same scratch and gets reclaimed at the outer
method's exit. The cross-seed-segv test pattern (foreign
vec push from inside a free fn) continues to work because
those tests call from `main`, not from a method body. A
method body that pushes a heap value into a foreign locus's
vec without the wrapping locus owning it would still
dangle — same boundary the cross-seed-segv fix originally
documented; the fix here doesn't widen or narrow it.
**Phase-4 perf follow-ons.** Three substrate
tunings that fell out of profiling the per-method scratch
reclaim on a real-world long-running workload:
1. **Lazy initial chunk.** `lotus_arena_alloc_struct`
leaves `head = NULL`; `lotus_arena_alloc` falls through
to its existing fresh-chunk path on the first call.
The dominant scratch shape on a hot dispatch path is
"open scratch → do non-allocating work → close
scratch" (e.g. a method body that only reads / does
arithmetic). Eagerly mallocing a 64 KiB chunk for
those scratches was pure waste — the lazy variant pays
zero mallocs end-to-end for non-allocating bodies.
2. **Thread-local chunk pool, 256 slots.** Bumped from 16
after observing ~99.6% miss rate at hot-path scratch-
churn rates. Each scheduler thread holds up to 256 ×
64 KiB = 16 MiB of recycled chunks ready for the next
scratch open. Pool predicate: only default-sized
chunks (`c->cap == LOTUS_ARENA_CHUNK_BYTES`) recycle;
larger one-off chunks bypass the pool and free via
libc directly. `LOTUS_CHUNK_POOL_STATS=1` dumps the
per-thread counters at exit.
3. **`lotus_str_clone` / `lotus_bytes_clone` skip
optimizations.** Two cases pass through without
allocating:
* Static-literal skip — `src` is in the binary's
`[__executable_start, _edata)` range (text + rodata
+ initialized data). String literals (`"foo"`)
lower to globals in .rodata; cloning them into an
arena is pointless since the original pointer is
already program-lifetime. Glibc-only via
`__GLIBC__` ifdef; other platforms pay the clone.
* Same-arena skip — `src` is already inside one of
the destination arena's chunks. Catches the
dominant `Counter.inc` / `Gauge.set` pattern where
the caller reads `e = store.get(key)` and writes
back `store.set(MetricEntry { key: e.key, ... })`
— the fields are already in the store's arena, so
re-cloning them is wasted memcpy. O(chunks); typical
arenas have 1–10 chunks so the walk is single-digit
ns. A sortable chunk index would tighten the bound
for very-long-running arenas; deferred until a
workload surfaces the cost.
4. **Outer-struct same-arena skip at pointer-storage
containers.** @form(vec).push / @form(vec).set /
@form(ring_buffer).push slots hold an 8-byte pointer to
a struct allocated separately. The deep-copy path runs a
runtime `lotus_arena_contains_ptr(dest_arena, src)` check
before allocating a fresh outer struct: if `src` is
already in dest, the existing pointer is stored and the
allocation is skipped. Catches read-modify-write shapes
where the value flows through dest's arena.
5. **Anchor-in-place at inline-storage containers.**
@form(hashmap) cells live inline in the slots buffer
(`value_size = sizeof(Cell)`) — the runtime memcpys the
value's bytes directly into the slot, so the deep-copy's
outer struct allocation is wasted work. Instead, walk
the source struct's fields and rewrite each heap-typed
field to a `dest_arena`-anchored version; leave scalars
untouched. The runtime's memcpy reads from the now-
anchored source struct. Net for the canonical
`Counter.inc()` / `Gauge.set()` RMW pattern (same-pointer
heap fields carried through from `e = store.get(key)`):
zero new allocations per call.
The per-field deep-copy routes through the same
`lotus_arena_contains_ptr` gate as #4 — implemented by
`emit_cross_arena_store_deep_copy_ptr`. For String/Bytes
fields the gate degenerates into `lotus_str_clone` /
`lotus_bytes_clone`'s own same-arena skip; for pointer-
shaped compound fields (`Array`, nested `TypeRef`, `Tuple`,
`Interface`, `Enum`) the gate is a runtime arena-membership
check on the loaded field pointer — identity-store when
the field already lives in `dest_arena`. Until 2026-05-25
the compound-field path went straight through the
unconditional deep-copy: cells with a fixed-size array
field (e.g. `BookSignalState`'s two `[BookLevel; 100]`)
allocated a fresh element buffer on every `set` of the
same key. A downstream market-data app's long-burn surfaced
this as ~3 KB / set × ~100 sets/sec → OOM at the
container cap in ~20 min; the field-level gate folded the
RMW back to zero allocations once the previously-stored
buffer is already in the hashmap's arena.
Cell single-owner (2026-07-18): the String/Bytes leaves of
the walk no longer use the plain clones' same-arena SKIP —
they route through `lotus_str_clone_cell_owned` /
`lotus_bytes_clone_cell_owned`, which force-copy a
same-arena input (statics still pass through; cross-arena
values clone exactly as before). And the walk operates on a
stack *snapshot* of the value struct rather than the source:
anchoring in place used to rewrite a self-storage source
(`m.set(self.rec)`) so the locus field and the cell ended up
pointing at the same blob — an in-place field overwrite then
mutated the cell silently, and anchor retirement could free
a blob the other owner still held. With snapshot +
owned-clone, a cell's top-level String/Bytes fields are
always exclusively owned. The compound-field gate (`Array`,
nested `TypeRef`, …) keeps its identity-skip — compounds
don't retire, so sharing there is a mutation-visibility
caveat, not a use-after-free (tracked in
notes/anchor-retirement.md). Note the RMW zero-allocation
claim above narrows accordingly: same-pointer String/Bytes
fields carried through a get-then-set now cost one owned
copy per set, served by the retire freelist (the replaced
generation recycles into the next), so steady-state memory
stays flat while compound fields keep the zero-copy skip.
6. **In-place mutation at `self.X = Struct{...}` and
`self.X[i] = Struct{...}` assigns.** Locus self-fields
(and elements of self-array-fields) are pre-allocated at
instantiation time — every field has either a default or
an instantiation-supplied value, so by the time any
method body runs the slot's pointer is non-null and points
to a struct in `self.__arena`. Assignment to such a slot
anchors the rhs's heap fields in `self.__arena` (same
mechanism as #5), then memcpys the rhs's bytes over the
existing struct's bytes via the slot's pointer. The slot's
pointer doesn't change.
Since 2026-07-17 the store also runs a per-String-field
replace fixup (`lotus_str_field_replace_fixup`): each old
field pointer the memcpy overwrites is *retired* (pending
until the user-method activation-boundary flush, then
reusable — the same anchor-retirement machinery as
@form(hashmap) set/remove), and a same-arena rhs pointer
that the anchor walk passed through unchanged is force-
copied so two self-storage slots never share a blob
(single-owner; see #7's aliasing note). RMW round-trips
(`self.X = self.X`, a scratch copy with untouched fields)
keep their pointers and retire nothing. Net: repeated
whole-struct replaces with fresh String contents hold
`self.__arena` flat — previously each replace orphaned the
old clones for the locus lifetime. String fields only at
v0.1 (Bytes and pointer-shaped compound fields of the
replaced struct keep the pre-fix behavior); fields of
@form vec / bounded cells and stores from boundary-less
contexts (a `run()` loop) do not retire.
7. **In-place String / Bytes reassignment at `self.X =
heap_value`.** `lotus_str_assign_in_place(arena, old, new)`
reuses the existing slot's buffer when `strlen(new) <=
strlen(old)`: memcpy new bytes + NUL into `old`'s buffer,
return `old` unchanged. `lotus_bytes_assign_in_place` is
the Bytes companion — same shape against the `[int64 len]
[payload]` Bytes header; when `new_len <= old_cap` it
updates the prefix and memcpys the payload in place.
Static-literal `old` falls through to the clone path
(`.rodata` isn't writable). When new is longer than old's
buffer, the abandoned buffer is *retired* (String since
2026-07-17, Bytes since 2026-07-18) — reusable after the
activation-boundary flush instead of leaking. The retire
freelist mixes align-1 String blocks and align-8 Bytes
blocks; pops are alignment-aware (a String request reuses
either kind, a Bytes request only 8-aligned blocks). Note
the capacity-collapse caveat below bounds what Bytes-grow
retire can reclaim: a single oscillating field's grow
requests are always larger than its own shrink-collapsed
retire records, so the recycled blocks pay off through
other same-arena allocations of matching sizes.
Single-owner rule (2026-07-17): on every path that stores
a replacement pointer, an incoming pointer that is already
inside `a` is another self-storage slot's blob (fresh
values live in method scratch; only self-storage reads
produce same-arena pointers here) and is force-copied
rather than shared. Pre-fix, `self.g = self.f` on the
non-fitting path stored `f`'s own pointer into `g`'s slot,
and `f`'s next in-place overwrite silently mutated `g` —
a value-semantics violation, and unsound to combine with
retirement. The codegen emits the right helper at every
`self.X = String|Bytes` site inside a method-with-scratch.
Closes the per-update heap-field-reassignment leak class
— measured against a per-frame `self.last_ts = ts` pattern
in a long-running daemon: the receiver locus's arena
dropped from ~+1-3 chunks per instance per 4 min to flat.
Caveat (both helpers): the
length-tracking field doubles as the capacity field, so a
reduce-then-grow oscillation gradually loses available
capacity and degrades toward "always clone"; bounded-
variance fields keep the in-place path indefinitely.
8. **m49 sret-pattern return-arena routing.** When a method-
with-scratch is about to return a heap-typed value, the
codegen sets `current_arena_override = caller_arena` for
the duration of the return-expression lowering. Fresh
allocations during that lowering (struct literals, nested
call results) route through `current_arena_ptr` which
honors the override, landing directly in caller storage
instead of the method's scratch subregion. The boundary
deep-copy in `emit_method_return_deep_copy` then contains-
checks the value via `emit_cross_arena_store_deep_copy_ptr`
and passes it through unchanged — no second memcpy.
`populate_user_type_fields` conditionally deep-copies each
heap field when the override is active (the
`BookSignalSnapshot { buys: let_bound_buy_res, ... }` alias-
field case anchors `buy_res` in caller_arena at field-init);
the populate gate flips on `current_arena_override.is_some
()` so ordinary struct construction (`hashmap.set`'s Cell
argument, locus param defaults) stays on the pre-rework
path and downstream anchor same-arena skips remain intact.
Closes the SweepResult / BookSignalSnapshot return-value
leak class.
**m49 closes the free-fn gap.** Every non-main free fn takes
an implicit `__caller_arena: ptr` first param at the LLVM ABI.
`main` keeps the program-wide `arena.global` it always had —
it's the single fn without a caller. Heap-typed returns of
Array, TypeRef-struct, or has-payload-Enum are rejected at
v0.1 — none currently appear as free-fn returns; ship as a
follow-up when a workload demands.
**Allocation routing (post-2026-05-18 cross-seed-segv fix,
commit 907837a).** Free-fn-body allocations now route to
`__caller_arena` directly. The earlier m49 design routed them
through a per-call subregion (`lotus_arena_create_subregion(
__caller_arena)`); that proved unsound because the codegen has
no escape analysis, so any value alloc'd in the subregion and
stored on a longer-lived structure (canonically: pushed onto a
`@form(vec)` on a foreign locus arg) dangled at fn-exit. The
fix routes allocations directly to `__caller_arena`. The
subregion is still created / destroyed at entry / exit so the
cleanup hooks for `fail E { ... }` payloads still have a
short-lived arena to anchor in, but the per-call performance
tier the subregion was meant to provide is deferred — it
needs escape analysis to ship safely. The fn-exit deep-copy
into `__caller_arena` is now a same-arena memcpy in the common
case (correct, marginally wasteful; can be elided in a
follow-up).
**Subregion elision for non-allocating bodies (FORM-3).** Codegen classifies each user fn at declare time
via a conservative syntactic walk
(`fn_body_definitely_non_allocating`). A body is non-
allocating iff every expression in it lowers to a known-non-
arena-touching shape: literals (incl. String — global static),
identifier reads, KwSelf, field/index reads (excluding
range-index slices), numeric/bool/bitwise Binary (Add excluded
since it could be String concat without type info threaded
in), Unary, If with non-allocating arms. For fns that pass the
classifier, the subregion `create` + `destroy` are skipped
entirely and the return-value memcpy epilogue is skipped — the
return value is either a primitive or a pointer to a region
stable across the fn frame (String-literal global, caller-passed
pointer, field read of one of those). Closes the bench's
per-call cost for leaf fns (`fn_call` went 188 ms → 37.1 ms =
5×, ratio vs Go 0.04× → 0.21×; `form_vec_push` reached 1.00× =
Go parity). The `__caller_arena` LLVM param is still passed
even to non-allocating fns (kept uniform per-fn ABI); the
optimization is purely on the body side. Fallible fns always
pay the full subregion lifecycle because `fail E { ... }`
allocates the payload struct into the subregion. Post-907837a
the elision benefit is narrower than its m49-era framing: with
allocating-body allocations now routed to caller-arena directly
(see "Allocation routing" above), the deep-copy epilogue is a
same-arena memcpy and the subregion lifecycle is mostly
overhead for cleanup hooks — the optimization still skips
both, just with a smaller per-call cost being avoided.
This delivers the spec's "every free function has its own
implicit locus" memory boundary at the codegen substrate.
Bound handles in free fn bodies still attach to the enclosing
deferred-dissolve frame (lifecycle parity with main and
lifecycle methods); the implicit-locus *handle-rooting*
semantic — fn return waits for in-fn-bound children to
dissolve as if the fn were itself a locus — remains a
future-work item, not exercised by any current example.
### Per-projection-class arena strategies (m22 + m23)
Each locus's projection class is resolved at codegen-declare
time: from an explicit `: projection rich|chunked|recognition`
annotation when present, otherwise per the spec/memory.md
default rule (chunked if the locus declares accept; rich
otherwise — recognition is explicit-only).
The class drives the *child arena allocation* strategy when this
locus accepts coordinatees:
- **Rich** parents: each child gets a fresh top-level arena via
`lotus_arena_create()`. Independent allocation lifetime;
parent does no bookkeeping. v0 default for non-coordinator
loci.
- **Chunked** parents: each accepted child gets a sub-region via
`lotus_arena_create_subregion(parent_arena)`. The parent
tracks a slot index per child; on child dissolve, the slot
returns to a per-arena free-list so peak slot space stays
O(concurrent children alive), not O(total children ever
accepted). Per F.3. The free-list + `next_slot` counter are
protected by a per-arena `pthread_mutex_t subregion_lock`
— without it, two threads concurrently
creating or destroying children of the same parent (common
under cross-pool cooperative placement where a worker's
handler-scratch sub-region sits under the App arena) would
race on the slot tracker: concurrent `realloc` of the
freelist buffer, double-pop yielding duplicate slot
indices, lost pushes. The lock window is O(1) per
create/destroy (a counter increment or a freelist push);
steady-state allocations within a sub-region remain
lock-free. **Single-thread fast-path:** the
lock/unlock and `pthread_mutex_destroy` are skipped — and
init is a const `PTHREAD_MUTEX_INITIALIZER` copy rather than
a `pthread_mutex_init` call — until the program spawns a
second thread. A monotonic `g_runtime_multithreaded` latch
(`lotus_mark_multithreaded`, set before every `pthread_create`:
coop-pool workers, pinned-locus spawns via `lotus_bus_mark_pinned`,
and unix/udp/shm transport reader threads) flips the lock sites
on. The first transition happens while only the main thread
exists and is inside spawn code (not an arena op), so no
in-flight op observes it mid-op; thereafter all threads lock as
before. Single-threaded programs pay zero mutex cost on the
arena hot path (recovers the bulk of the +91% `locus_instantiation`
regression the mutex introduced); the residual is the const-copy
init, removable only by a lock-free freelist (deferred).
- **Recognition** parents (v1.x-3): the sub-mode commitment
spelled at the declaration site picks the allocator family.
`fixed_cell` routes children through
`lotus_recpool_fixed_acquire` (bitmap-tracked cells, inline
arena per cell); `shared_slab` routes children through
`lotus_recpool_slab_acquire` (every child shares one bump
arena). Cell stride for either sub-mode is derived at
codegen time from the parent's accept-method param type
union — not a user-supplied byte budget. At parent instantiation the recpool is
allocated via the matching `_create` fn and stashed on the
synthetic `__recpool: ptr` struct field; at parent dissolve
it's torn down via `_destroy`. The child's arena teardown
is dispatched at the C ABI level: a discriminator on the
child struct picks `lotus_arena_destroy` (kind=0, regular),
`lotus_recpool_fixed_release` (kind=1), or
`lotus_recpool_slab_release` (kind=2). The surface contract
("parent owns a pool, no dynamic allocation in steady
state for `fixed_cell`; one bump for `shared_slab`") is
exercised by `examples/14-projection-classes`.
C runtime ABI as of v1.x-3:
```
ptr lotus_arena_create(void)
ptr lotus_arena_create_subregion(ptr parent) // m22
ptr lotus_arena_alloc(ptr arena, i64 size, i64 align)
void lotus_arena_destroy(ptr arena) // auto-detects sub-region
// and returns slot to
// parent's free-list
ptr lotus_recpool_fixed_create(i64 cap, i64 bytes) // v1.x-3
ptr lotus_recpool_fixed_acquire(ptr pool) // v1.x-3
void lotus_recpool_fixed_release(ptr pool, ptr arena) // v1.x-3
void lotus_recpool_fixed_destroy(ptr pool) // v1.x-3
ptr lotus_recpool_slab_create(i64 cap, i64 bytes) // v1.x-3
ptr lotus_recpool_slab_acquire(ptr pool) // v1.x-3
void lotus_recpool_slab_release(ptr pool, ptr arena) // v1.x-3
void lotus_recpool_slab_destroy(ptr pool) // v1.x-3
```
`lotus_arena_destroy` is unified across the regular + subregion
shapes — it inspects the arena's optional `parent` pointer and
slot, and returns the slot to the parent's free-list when
present. The recpool variants are NOT unified with
`lotus_arena_destroy` — their backing storage layout is
distinct (inline-in-cell for fixed_cell; shared-bump for
shared_slab) and routing through `lotus_arena_destroy` would
corrupt the recpool's bookkeeping. The codegen dispatch
discriminator (`__recpool_release_kind`) is what keeps the
right release function reachable at child dissolve.
> Forward-looking / deferred items for this area now live in the
> decision log — see [`decisions.md` § Deferred & future
> work](./decisions.md#deferred--future-work).
================================================================================
## Runtime
## source: spec/runtime.md
================================================================================
# Runtime
What every compiled Hale binary always ships with. Always-
loaded; not optional; no `import` needed; the substrate every
Hale program depends on.
This document distinguishes the **runtime** (always there) from
the **standard library** (`stdlib.md`, importable but bundled).
Go's distinction between `runtime` and other stdlib packages is
the model: runtime is automatic; stdlib is explicit.
> **Naming note:** The language is **Hale**; the runtime/
> substrate concept is called **lotus**, and the C-runtime
> symbols stay `lotus_*` (per project memory). When this doc
> says "lotus" it means the substrate; "Hale" means the
> language proper.
## What's in the runtime
### Memory
- **Region allocator.** Per-locus arenas, hierarchical, freed
on dissolution. Bump allocation within a region; no per-object
metadata; no GC. The framework's lotus structure provides the
scope; the allocator just respects it.
- **Per-method scratch.** A locus method body
(lifecycle / user-fn / mode) opens a per-call subregion of
`self.__arena` at entry and destroys it at every return —
*unless* the body provably allocates nothing and returns a
by-value scalar (or Unit), in which case the scratch is elided
(2026-06-28; an optimization with no observable effect — there's
nothing to reclaim, so skipping the subregion just removes a
`malloc`/`free` per call). Transient allocations made inside the
body — `to_string`,
`String` concat, `std::str::*` / `std::json::*` / `std::bytes::*`
results, format-string composition — route through the
scratch via `current_arena_ptr()` and get reclaimed at method
exit. Heap-typed `self.X = expr` stores deep-copy into
`self.__arena` before the store so persisted state outlives
the scratch destroy. Heap return values are deep-copied into
the caller's arena via a fn-local snapshot of
`lotus_caller_arena_or_global()` taken at the method's entry
block. Callers publish their `current_arena_ptr()` via
`lotus_set_caller_arena` immediately before each method call
(same TLS contract as stdlib primitives). Without this,
long-running `run()` loops accumulated every transient
allocation into the locus's lifetime arena — a real
workload measured multiple MB/sec of growth on a hot
message-dispatch path, OOM within minutes under a typical
container cap. See
`spec/memory.md` "Phase-4 per-method scratch reclaim" for
the full design (invariants, cost model, interaction with
the cross-seed-segv routing).
- **Per-projection-class allocation strategy.** Rich → simple
arena; chunked → arena with per-coordinatee sub-regions;
recognition → recpool, sub-mode-typed at the declaration
site (see "Recognition pool allocators" below). Selected
at compile time per locus.
- **Free-list within parent for bookkeeping reclamation.** When
a coordinatee dissolves, its bookkeeping slot in the parent's
arena is reclaimed via a per-arena free-list (chunked-class
loci) or periodic defrag (high-churn loci). Reclamation is
**per-arena**, **bounded**, **deterministic** — never stop-
the-world. Coordinatee sub-regions remain pristine arenas
freed wholesale on dissolution.
- **F.22 capacity-slot allocators.** Each `pool X of T;` /
`heap Y of T;` declaration on a locus adds a per-instance
allocator. The C runtime ships two symbol families:
| Family | Surface | Backing |
|---|---|---|
| `lotus_pool_*` | `create(cell_size, cell_align) -> pool*`, `acquire(pool) -> cell*`, `release(pool, cell)`, `destroy(pool)` | Linked list of chunks; each chunk is one malloc holding N contiguous cells. Free-list threads through the cells themselves (each free cell stores the next-free pointer at its base). Chunks grow geometrically (initial sized so one chunk fits in a host page when stride permits, else 16 cells; doubling; capped at 4096). Cell stride = max(cell_size, sizeof(void*)) aligned to cell_align. v1.x-17: initial chunk cell count is `max(16, page_size / cell_stride)` capped at 4096 — sysconf(_SC_PAGESIZE) queried once and cached; falls back to 4 KiB on systems where sysconf returns implausible values. |
| `lotus_heap_*` | `create(cell_size, cell_align) -> heap*`, `alloc(heap) -> cell*`, `free(heap, cell)`, `destroy(heap)` | Doubly-linked live list with intrusive header (prev/next pointers) sitting just before each cell. `free()` unlinks in O(1); `destroy()` walks the list and frees every still-live cell wholesale. |
Both allocator families are type-erased at the C ABI (sizes
+ aligns are i64 args). Cell alignment is 8 bytes uniformly
in v0; loci with cells requiring >8-byte alignment (e.g.
AVX-aligned types) are not supported. Per F.22 §"Slot
lifetime", slot init runs after slot 0 / arena and destroy
runs in reverse before slot 0 / arena. See
`spec/semantics.md` "Capacity slot lifecycle and dispatch
(F.22)" for the language-level surface and `spec/memory.md`
"Capacity slots (F.22)" for the lotus-substrate framing.
- **Recognition pool allocators (v1.x-3).** A locus with
`: projection recognition(cap=N, )` allocates one
recpool at instantiation; child loci accepted by that parent
draw their arena from the pool instead of `lotus_arena_create
_subregion`. Two symbol families ship in v1:
| Family | Surface | Backing |
|---|---|---|
| `lotus_recpool_fixed_*` | `create(cap, cell_bytes) -> recpool*`, `acquire(recpool) -> arena*`, `release(recpool, arena)`, `destroy(recpool)` | One contiguous block of `cap × cell_stride` bytes. Each cell carries an INLINE `lotus_arena_t` + `lotus_arena_chunk_t` header at its front followed by `cell_bytes` of payload — the cell IS the child's arena. Bitmap (`uint64_t[ceil(cap/64)]`) tracks occupancy; acquire scans the lowest unset bit via `__builtin_ctzll`. Release clears the bit so the slot is reusable. The returned arena has `fixed_size=1`, so `lotus_arena_alloc` returns NULL on overflow (caller routes to the closure-violation channel). |
| `lotus_recpool_slab_*` | `create(cap, slab_bytes) -> recpool*`, `acquire(recpool) -> arena*`, `release(recpool, arena)`, `destroy(recpool)` | One `lotus_arena_t` with an initial chunk of `slab_bytes` and `fixed_size=1` so it never grows. Every `acquire` returns the SAME arena pointer — children share the bump space and per-child release is a no-op. The whole slab frees at parent dissolve via `lotus_arena_destroy(slab_arena)`. `cap` is recorded but not enforced at the C layer (codegen's birth-cap check bounds concurrent children; the slab is a memory budget). |
Both families return `lotus_arena_t*` from `acquire` so child
body code stays projection-class-agnostic per the F.22
architectural invariant — the same `arena_alloc` path handles
fresh, subregion, fixed-cell, and shared-slab children. The
codegen dispatch at child dissolve picks the matching
`release` fn via the synthetic `__recpool_release_kind`
discriminator (0 = regular `arena_destroy`, 1 = fixed_cell
release, 2 = shared_slab release). v1 ships `fixed_cell` and
`shared_slab`; `spillover` and `summary_only` parse + AST
through but reject at typecheck with a `v1.x pending`
diagnostic (the spillover malloc-fallback machinery and the
`summary_only` "no child arena allocation" type-system rule
are separate work).
### Lifecycle
- **Lifecycle dispatcher.** Invokes `birth → run → drain →
dissolve` per locus; invokes `accept` on coordinatee
attachment; invokes `on_failure` on child failure with the
parent's policy.
- **Interest-based ownership (accept bubbling)**.
`accept(c: I)` collects not only a *direct* child but the
nearest such acceptor for an `I{}` instantiated anywhere in the
subtree: when a locus instantiates `I{}` and its direct
enclosing locus does not `accept(I)`, ownership *bubbles* to the
nearest enclosing ancestor that does (innermost-wins).
Resolution is entirely static — there is no polymorphic locus
instantiation, so the closed-world instantiation graph fixes
every owner edge at compile time; no runtime ancestor walk.
Backward-compatible by construction: innermost-wins selects the
direct parent whenever it accepts, so no existing parent↔child
edge changes; bubbling only *adds* an owner where a child would
otherwise be a transient throwaway. An `I{}` with no accepting
ancestor stays transient — ownership is opt-in via `accept`, and
the absence of an owner is never an error. Same-tower bubbling
costs nothing beyond the direct-parent case (the owner pointer is
a constant for a singleton owner, or threaded down the birth
chain for multiple owner instances — giving each owner instance
its own isolated collection — then the ordinary accept path). A
cross-pool owner (e.g. a `main locus` registry collecting
entities spawned on a worker pool) is served by an async handoff
over the bus queue: the child is born on the owner's thread and
reclaimed by the owner's same-thread cascade, so a cross-pool
`I{}` is **fire-and-forget** — it may only be a bare statement;
using the instance as a value is rejected at compile time.
- **State machine enforcement.** A locus can't accept after
drain has begun, can't run before birth completed, etc. The
runtime tracks state; transitions are rejected if they
violate ordering.
- **`drain()` cascades depth-first.** Calling `drain()` on a
locus first recursively drains all its children (depth-first),
waits for them, then drains itself. SIGINT triggers `drain()`
on the runtime root, cascading through the whole process
tree. No separate cascade syntax — `drain()` is always
cascading. For locus-typed param fields specifically
(F.29), the codegen walks `LocusRef` fields in declaration
order at the cascade-teardown sites (ephemeral scope-exit
and deferred-flush) and calls each child's drain BEFORE the
outer locus's own drain. The subsequent dissolve cascade
runs the outer's `closures → dissolve` body next, then per
child `closures → dissolve → arena_destroy`, then outer's
arena_destroy. Pinned-thread tail still skips the cascade
per the v1 trade-off. An `accept`'d child is reclaimed on its
OWN run-completion / `terminate` when it is a flow (see
"Per-child reclamation" below) rather than waiting for the
parent's cascade.
- **Per-child reclamation**. An `accept`'d child's
`run()` is posted to its pool as a coro, run through a
synthesized `__coop_pool_run_` wrapper. When that run()
completes, the wrapper reclaims the child — drain → (for a
flow) the parent's `release(owner, self)` → dissolve →
arena/recpool release — iff the child is a **flow** (some
declared locus has `release(c: ThisType)`) OR it set the
`__drain_requested` latch via `terminate;`. A non-flow
("resident") child whose run() merely returns is NOT reclaimed
(it lives to parent dissolve). The reclaim runs on the child's
own pool worker while its arena is valid; `emit_locus_arena_
destroy` is idempotent (NULLs `__arena`), so a later
parent-dissolve of the same locus no-ops. At pool shutdown a
coro may still be PARKED (a listener in `accept()`); the
**wakeable park** handles it: a per-pool wake `eventfd` in the
pool's epoll lets `shutdown_all` unblock a worker sitting in
`epoll_wait(-1)` (the condvar broadcast can't), so the worker
returns from the drain and the join completes instead of
hanging. The parked coros are then *abandoned* — their stacks
freed without resuming them — because a forever-loop `run()`
(`while true { accept }`) cannot be cooperatively unwound
without the loop checking `self.draining` (a future
refinement), and the process is exiting anyway. Per-child
reclamation proper (terminate / flow run-completion) never
needs this: there the coro returns from `run()` on its own.
- **Classic-pool blocking-accept shutdown.** A
*classic* (non-`async_io`) pool worker blocked in a blocking
`accept(2)` inside a locus's `run()` (e.g. `std::http::Server`
or `std::io::tcp::Listener` placed on a plain
`cooperative(pool = X)`) can't be woken by the wake `eventfd`
(there is no epoll on a classic pool) — so two rules keep its
teardown clean: (a) the classic `accept` polls the listen fd
with a short timeout and checks its pool's shutdown flag, so it
returns a sentinel `-1` once `shutdown_all` is signalled and the
stdlib accept loops (`Server`/`Listener`) break out of their
forever loop; and (b) the **main locus joins all pool workers
before dissolving its `params` fields**, so a worker still
executing a pool-placed field's `run()` can never touch that
field's arena after it's freed (the alternative — freeing first
— is a use-after-free; the alternative join-without-(a) is a
hang). Together these let a program whose `main` run() returns
while a classic-pool server child is live shut down cleanly
rather than hanging or segfaulting.
- **Recovery primitives.** `restart`, `restart_in_place`,
`quarantine`, `reorganize`, `bubble`, `dissolve`, `drain` —
all language keywords; runtime implements the actual
effects.
- **Recovery primitives.** `restart`, `restart_in_place`,
`quarantine`, `reorganize`, `bubble`, `dissolve`, `drain` —
all language keywords; runtime implements the actual
effects.
### Scheduler — multi-scheduler cooperative
Lotus uses a **multi-scheduler cooperative** model (closest
existing analog: Erlang BEAM, *not* Go's M:N). The reasons are
framework-discipline:
- **Lateral-access prohibition is physical, not just typed.**
Within a single cooperative scheduler, sibling loci cannot
run concurrently — only one locus is executing at a time per
scheduler. There is no thread of execution that could attempt
a lateral memory reference. The compile-time type rule
("vertical-only flow") is reinforced by the substrate.
- **Substrate-cell atomicity is naturally aligned.** Cooperative
yield points — between message-handler invocations, between
lifecycle phases, on bus dispatch — are exactly where the
substrate-cell boundary lives. No preemption inside a
substrate-cell because the runtime can't preempt at all;
it only switches at yield points.
- **Per-scheduler region allocators.** Each scheduler is
single-threaded, so its allocator state is naturally
per-scheduler with no synchronization. Lock-free by
construction.
- **Failure-traversal is a call-stack walk on one scheduler.**
No cross-thread synchronization for parent-catches-child
failure when both are on the same scheduler.
Concurrency comes from running **multiple cooperative schedulers
in parallel** (one per CPU core, by default). Loci belong to a
specific scheduler; cross-scheduler communication uses the bus
just like cross-process communication. Loci may be migrated
between schedulers transparently for load balancing because all
their communication is bus-mediated already.
Specifically:
- **One scheduler per CPU core** at startup, configurable.
- **Cooperative yield points**: between handler invocations,
between lifecycle transitions, on bus message dispatch, on
explicit `yield` (rare, for long-running computations).
Plain fn exit is NOT a yield point — and since 2026-07-02 a
proven-non-allocating fn's exit provably skips the queue drain
(a non-allocating body cannot have published; payload copies
allocate). A cooperative compute-only loop that leaned on
helper-call returns for delivery never had that guarantee and
must use an explicit `yield;`.
- **No preemption within a scheduler.** A locus's handler runs
to completion or an explicit yield.
- **Cross-scheduler is bus.** No shared memory; no locks.
- **Failure-traversal**: if parent and child are on the same
scheduler, failure-traversal is a stack walk. If different
schedulers, the failure is delivered as a typed bus message
to the parent's scheduler, which dispatches to `on_failure`.
### Placement classes (per-locus execution strategy)
Just as **projection class** governs a locus's memory strategy,
**placement class** governs its execution strategy. Placement
is a *deployment seam*, not an intrinsic property of the locus
(see `spec/decisions.md` § F.31). Placement entries
live in a `placement { }` block on `main locus` only, parallel
to `bindings { }` for bus topology:
```hale
main locus App {
params {
gateway_a: Gateway = Gateway { venue: "venue-a" };
gateway_b: Gateway = Gateway { venue: "venue-b" };
metrics: MetricsServer = MetricsServer { port: 9100 };
ui: Renderer = Renderer { };
}
placement {
gateway_a: pinned(core = 1);
gateway_b: pinned(core = 2);
metrics: cooperative(pool = io);
ui: cooperative(pool = render);
// unspecified main-locus params → cooperative(pool = main)
}
}
```
Placement remains honestly **bimodal**: either a locus shares
a cooperative pool (an OS thread running a cooperative drain
loop) or it owns its own OS thread. There is no third position.
| Class | Yield discipline | Resource |
|---|---|---|
| **`cooperative(pool = X)`** (default for unspecified main-locus params, with `X = main`) | Yields between substrate cells (handler exit, lifecycle transition, bus dispatch, `time::sleep`, explicit `yield`). `time::sleep` slices into ≤100ms intervals and folds in `lotus_bus_queue_drain` for the locus's pool after each slice, so cells posted by other threads deliver mid-loop even during a long keep-alive sleep. Handler bodies are atomic. | Shares pool `X`'s OS thread with other cooperative loci placed on the same pool. |
| **`pinned`** / **`pinned(core = N)`** / **`pinned(cores = A..B \| A..=B \| {a, b, c})`** / **`pinned(node = N)`** / **`pinned(l3 = name)`** / **`pinned(..., replicas = K)`** | No yield to siblings; owns its OS thread. Bus events to/from cross-thread boundaries via formal mailbox post. | Dedicated OS thread. `core = N` pins it to one CPU; `cores = ...` (Phase 1a) sets its mask to a core *set*; `node = N` / `l3 = name` (Phase 1b) set the mask to a NUMA node / cache domain from `topology { }` (and bind the arena there); `replicas = K` (Phase 1c) fans into K single-threaded instances, one per core. The OS schedules freely within the mask. Linux-only; best-effort no-op elsewhere. |
**Pool inference rule.** The cooperative pool set is inferred
from `cooperative(pool = X)` references in the `placement { }`
block. The runtime spawns one OS worker thread per inferred
pool name beyond `main` (which is always the program's main
thread). No separate `threads { }` declaration block at v1 —
when per-pool attributes (priority, affinity, realtime hint)
become useful, the block lands then as a typed extension. Pool
`main` exists in every program regardless of whether
`placement { }` references it.
**Nested-instantiation inheritance.** Placement entries apply
only to top-level `main locus` `params` fields. Loci
instantiated nested in another locus's body (in `birth` /
`run` / lifecycle methods, or as let-bound children) inherit
the parent's pool. There is no way to spell "this nested child
runs on a different pool than its parent" — that would require
the nested-instantiation expression to carry placement, which
would re-mix the deployment and intrinsic layers F.31 separates.
**Topology block (Phase 1b).** A `main locus` may
also declare a `topology { }` block — a **declare-only**
description of the host's core partition, a sibling deployment
seam to `placement { }` / `bindings { }`:
```hale
topology {
reserve cores 0..2; // held back for the OS / main
node 0 {
l3 fast { cores 4..8; } // a CCD / shared-L3 group
l3 slow { cores 8..12; }
}
node 1 {
l3 heavy { cores 12..16; }
}
}
```
A `placement { }` entry then targets a domain: `pinned(node =
0)` sets the thread's affinity mask to node 0's core set (the
union of its L3 domains — here `{4..12}`), and `pinned(l3 =
fast)` sets it to the named domain's cores (`{4..8}`). The
compiler resolves the domain to a concrete core set at compile
time (closed-world: node ids and domain cores are literals),
reusing the same cpuset affinity mechanism as `pinned(cores =
...)`. Validation (unique node ids, globally-unique L3 names,
non-overlapping domains, no domain/reserved overlap, and every
`pinned(node/l3)` referencing a declared domain) is static. L3
domain names go through the identifier rule, so a hard keyword
(e.g. `bulk`) can't name a domain.
**Replicas (Phase 1c).** `pinned(..., replicas = K)` is the
parallelism sugar: it fans the field into **K single-threaded
instances**, replica `i` pinned to one core of the affinity set
(round-robin — `pinned(cores = 4..12, replicas = 8)` puts replica
`i` on core `4 + i`; more replicas than cores wraps; with no
affinity the K instances are OS-scheduled). This is deliberately
*not* a multi-worker pool — a cooperative pool is one consumer
thread, and the lock-free rings, bus devirtualization, and
single-threaded-method guarantee all rest on that invariant.
Parallelism instead comes from more single-threaded units, each
its own single consumer, so every invariant survives. `replicas`
is **pinned-only** (K cooperative loci on one pool would share a
thread, which isn't parallel) and composes with the topology
targets (`pinned(node = 0, replicas = 4)` fans across node 0's
cores with each replica's arena bound to node 0). Codegen emits K
instantiations at the field's init site; all K register their bus
subscriptions (a subscribed topic fans out to every replica) and
all K are joined + dissolved at parent teardown via the deferred-
dissolve frame. The replicas are non-addressable — there is no
`field[i]` surface; they are workers that pull from the bus or run
their own loop.
**Thread + memory co-location.** A `pinned(node = N)` /
`pinned(l3 = name)` locus binds not just its thread but its
*memory*: its arena is created via
`lotus_arena_create_labeled_on_node`, which flags the arena's
NUMA node, and every chunk that arena grows is `mmap`'d
(page-aligned) and bound to the node with the `mbind` syscall
(`MPOL_BIND`) before first touch — so pages fault in on the
node regardless of which thread touches them first (the locus
struct is instantiated on `main` but runs on its own pinned
thread). Sub-regions inherit the node, so a node-pinned locus's
**method scratch** — the dominant per-invocation allocation —
lands on its node too. `mbind` is invoked as a raw syscall, so
this adds **no libnuma dependency**; the whole feature is
zero-cost for programs that don't opt in (an unbound arena, the
default, takes the ordinary malloc / chunk-pool path
byte-for-byte). Best-effort and Linux-only, exactly like
`pinned(core = N)`: an `mbind` the box can't honor (node absent,
capability denied) falls back to first-touch, and on non-Linux
hosts the arena allocates normally. (Huge-page-backed chunks
and node binding don't currently combine — a node-bound arena
uses regular pages; a follow-up.)
**Single-threaded-method invariant.** A locus's methods may
be invoked only on the OS thread that owns its placement's
pool. Cross-pool method calls and lateral field accesses go
through the bus's existing copy-and-condvar dispatch
machinery, which already crosses thread boundaries safely.
The typechecker walks the static call graph from each
top-level placement entry, propagates pool ownership through
method receivers, and rejects calls that cross pools without
going through the bus. This is the substrate enforcement that
makes M:N safe — without it, multi-pool deployments would
silently race on locus arenas (which are unsynchronized bump
allocators).
The single shared **bus payload arena** is the deliberate
exception to "one owning thread per arena": it is reachable
concurrently from any pool, because some stdlib primitives
allocate their result there directly rather than into a
per-locus arena (e.g. `std::io::tls::recv_bytes`, which always
targets it regardless of the caller's per-thread scratch). That
arena therefore carries an internal lock on its bump — two
pinned loci calling such a primitive at once do not corrupt each
other's allocations. Per-locus arenas keep the lock-free bump;
only this one shared arena pays for the lock.
#### Why no "greedy" class
A natural temptation is to want a third option: "shares a
pool's thread but doesn't yield." That would be a bimodality
violation. Cooperative already guarantees handler-level
atomicity — no preemption within a substrate cell — so the only
thing such a class could add over cooperative is "don't yield
*between* cells either." But that means leaving the shared
pool entirely. The place you go when you leave is your own
thread. That's pinned.
Latency-critical work, or anything that genuinely shouldn't
share with neighbors on its pool, is signaling that it belongs
on its own pool (placed `cooperative(pool = some_quiet_pool)`)
or on its own thread (placed `pinned`). The first option is
new in F.31: pools partition the cooperative substrate so
"shouldn't share with siblings" no longer forces pinned — but
the underlying bimodality (cooperative-pool vs pinned-thread)
holds.
#### `time::sleep` drain semantics
The codegen lowering of `std::time::sleep(d)` slices the request
into intervals of at most **100ms** and ends each slice's
EINTR-retry loop with an inline call to `lotus_bus_queue_drain`
against the program-wide cooperative queue (plus a pinned-mailbox
drain). The total wall-clock sleep is preserved (the slices sum to
`d`); sleeps of 100ms or less take exactly one slice, so the common
case is unchanged. A cooperative subscriber looping
```hale
run() {
while !self.bail {
std::time::sleep(100ms);
// ... loop body sees handlers fired during the sleep
}
}
```
receives cells posted by other threads — unix-bound reader
threads, pinned publishers via `lotus_bus_local_dispatch`, etc.
— right when each sleep returns, without an explicit `yield;`.
The drain is idempotent, so existing code with `sleep; yield;`
stays correct.
**Which path delivers to whom.** The queue drained *here* is the
program-wide *in-process cooperative* queue, drained on the `main`
thread — so this sleep-loop drain delivers in-process cells to a
cooperative subscriber on the `main` pool. It is **not** the only
delivery path: cross-process topics (`udp://` / `unix://` bound via
`LOTUS_BUS_CONFIG`) are delivered by the transport reader thread,
which dispatches directly into the subscribed handler set and
reaches a cooperative locus on *any* pool — **provided that pool's
thread is free to run the dispatch.** So a non-`main` cooperative
subscriber receives reliably as long as it doesn't monopolize its
pool thread with a blocking call. (Pinned loci are a third path: a
per-locus mailbox drained at each `sleep`/`yield`.) The failure mode
is a cooperative subscriber whose `run()` *blocks* — the dispatch
can't run and its handlers never fire; that blocking-and-subscribing
combination is what the dead-bus-receiver rule rejects (see
"Type-check rules" in `spec/semantics.md`), **not** non-`main`
placement on its own.
**Long sleeps no longer starve main-pool handlers.**
Before the slicing, the drain happened only *after the whole sleep
returned*, so the natural keep-alive idiom
```hale
run() { while true { std::time::sleep(60s); } } // on the main thread
```
starved every `pool = main` bus handler for 60s at a time: a
main-pool subscriber registers with `coop_pool == NULL`, so the
wire/reader-thread dispatch path lands its cells on the global
cooperative queue (`g_bus_queue_for_remote`, the same object only
`main` drains). A 60s blocking sleep on `main` left those cells
unserviced for the full 60s — indistinguishable from "the handler
never fires." Slicing keeps the queue serviced ~10×/s during any
sleep, so a main-pool handler that republishes onto a topic an
async-pool subscriber listens to (e.g. a udp-reader-fed producer
forwarding to per-connection writers) now flows promptly regardless
of how long `main` is asleep.
The pinned-mailbox path is unchanged: pinned subscribers wake
on `lotus_mailbox_post`'s condvar broadcast regardless of what
the cooperative scheduler is doing.
#### Owner-executed handlers (2026-07-15, downstream handoff)
Two runtime rules restore the single-threaded-locus invariant
(F.31) *dynamically* — the compiler already enforces it for
direct calls, but two bus paths used to violate it under load
(reproduced as a SIGSEGV in a 10k msg/s ingest bench):
1. **The global cooperative queue is drained only by its owner
thread** (`main`, recorded at queue creation). The scope-exit
flush emitted at the end of every fn/method body — and the
sleep-slice / `yield` drains — call `lotus_bus_queue_drain` on
whatever thread ran the body; on any thread but the owner the
call is now a no-op. Previously a pinned publisher's flush
would execute a main-pool subscriber's handler on the
publisher's thread, concurrently with `main`'s own drains (the
locked drain releases the queue mutex before each handler
invocation) — two threads inside one locus.
2. **Payload deserialization happens on the subscriber's owner
thread.** The non-flat dispatch paths deserialize each
published payload into the subscriber's arena (Task-11 arena
routing); for a target owned by a different thread this write
used to happen on the *publisher's* thread — an unlocked
cross-thread write into a foreign arena. Now a cross-thread
publish enqueues the *wire bytes* plus the deserialize fn in
the cell, and the owner materializes (deserializes into its
own arena) at drain, just before invoking the handler.
Same-thread targets keep the deserialize-at-dispatch fast
path, so single-pool programs are unchanged.
Consequences: a main-pool subscriber's handlers run **only on
`main`** (at sleep-slice / yield / scope-exit drains — worst-case
~100ms after a cross-thread publish, per the slicing above);
pool subscribers run only on their pool's worker; pinned
subscribers only on their own thread. Flat (pointer-free POD)
payloads are exempt from rule 2 — a verbatim byte copy writes no
arena. Delivery and FIFO order per subscriber are unchanged.
**Item B from the 2026-05-21 friction log** (a cooperative
publisher's `<-` to a pinned subscriber) is **resolved as of
2026-06-01** — the earlier "drains only at dissolve in some
configurations" was a *sequencing* effect, not a lost wakeup.
The mailbox condvar path is correct: a pinned subscriber whose
`run()` **returns** proceeds into the blocking
`lotus_mailbox_drain_one`, and a cooperative publisher's `<-`
wakes it via the `not_empty` broadcast — confirmed by
`coop_to_pinned_mid_program::pinned_returning_run_drains_mailbox`.
The residual constraint is inherent to a single pinned thread:
a pinned subscriber with a **long-running `run()`** that never
returns or yields cannot drain its mailbox *during* `run()` —
the one thread is busy in the loop. Such a `run()` must reach a
cooperative yield point (`time::sleep` / `yield`) for the
TLS-cached `lotus_mailbox_drain_pending` to service the mailbox
(the `coop_to_pinned_mid_program` sleep-loop test), or let
`run()` return so the post-run blocking drain takes over. This
is a property of the bimodal model — "pinned owns its thread,
no yielding between cells" — not a bug; a pinned subscriber that
must receive bus traffic while busy should yield in its loop.
Cross-binary flows (unix / shm_ring with their own reader
threads) land in `g_bus_queue` and benefit from the sleep-folded
drain above.
#### Long-running cooperative children: placement closes Item D
Pre-F.31, declaring a long-running cooperative child as a
`params` field (`metrics_server: std::http::Server = ...`)
serialized parent and child onto the main thread: the child's
`run()` body never returned, so the parent's `run()` never
started. The workaround was the **sibling-in-main pattern** —
hoisting the child to a top-level `main locus` param so it
ran "alongside" the parent rather than "inside" it.
Under F.31 the sibling-in-main pattern IS the canonical
shape, and it composes cleanly because `placement { }` lets
each sibling pick its own pool:
```hale
main locus App {
params {
gateway: Gateway = Gateway { };
metrics: std::http::Server = std::http::Server { port: 9100 };
}
placement {
gateway: pinned(core = 1);
metrics: cooperative(pool = io);
}
}
```
Parent's `run()` no longer serializes against `metrics`'s
accept loop — they sit on different OS threads. To shut down
gracefully when one finishes (e.g. a duration-bounded gateway
exits), call `metrics.shutdown()` from the finishing locus's
thread; the C-iii interruptible-accept work makes this the
supported pattern.
The nested-as-child shape (long-running cooperative locus as
a `params` field of a non-`main` locus) remains structurally
serialized — that's a consequence of the nested-instantiation
inheritance rule (children share their parent's pool by
construction). Nested long-running children are an antipattern
under F.31: hoist to main-locus siblings.
**Typecheck enforcement.** The compiler rejects
the antipattern at typecheck. A non-main locus with a non-trivial
`run()` body holding a `params` field of a locus type whose own
`run()` is also non-trivial — including `std::http::Server` and
the other entries on the known-long-running stdlib allowlist —
gets a hard error pointing at the canonical sibling-in-main +
placement fix. The runtime starvation that motivated this rule
is silent (the parent's `run()` simply never executes), so the
type-side rejection is load-bearing: it converts a class of
hard-to-diagnose runtime bugs into a clear compile-time signal.
#### `where async_io` — green-I/O cooperative pools (F.35)
The sibling-in-main fix puts each long-running child on its own
OS thread, which caps concurrent connections at one-per-pool. To
scale beyond that without spawning a thread per connection, a
placement entry may declare `where async_io`:
```hale
main locus App {
params {
listener: std::websocket::Server = std::websocket::Server { ... };
worker: WsWorker = WsWorker { ... };
}
placement {
listener: cooperative(pool = ws_accept) where async_io;
worker: cooperative(pool = ws_workers) where async_io;
}
}
```
`where async_io` opts the pool into green-I/O scheduling: the
pool's worker drain loop integrates an epoll instance, and
blocking I/O syscalls inside locus methods on this pool park-
and-resume instead of blocking the OS thread. The user code
inside the locus is unchanged — `recv_bytes(stream)` reads the
same line of source whether the pool is `async_io` or not; the
substrate picks the right lowering at the syscall boundary.
Because parking yields the shared worker, N reader loci that each
park on their own fd — the F.35 one-reader-per-signal shape —
are serviced concurrently by a single pool. Two invariants make
that multiplexing correct: (1) the drain loop starts a queued
`run()` the moment the running coro *parks*, not only when it
completes, so a long-lived reader never starves the readers
queued behind it; and (2) each coro's caller-arena — the
thread-local that decides where its stdlib allocations (recv
result blobs, string builders) land — is snapshotted across the
park and restored on resume, so a coro that resumes after a
sibling ran (and perhaps dissolved) never allocates through an
arena the sibling has since torn down. Every blocking-recv
primitive on the pool honors invariant (1) by parking rather than
blocking `recvfrom`/`read` (the `std::io::udp` recv family joined
the `tcp`/`tls` siblings here in the 2026-07-15 downstream
handoff).
Each bus delivery to a subscriber on an `async_io` pool runs its
handler on a coroutine (a struct + a 64 KiB stack). Rather than
allocate and free that pair per delivery, the pool keeps a bounded
per-worker free-list (cap 64) of completed coro slots and reuses
them — a warm fan-out skips the per-dispatch stack allocation
entirely (2026-07-16). The free-list is worker-thread-local (no
lock) and drained at pool teardown, so a busy async pool retains up
to 64 × 64 KiB (~4 MiB) of coro stacks at steady state. Transparent
to user code — a pure allocation optimization, no behavior change.
Typecheck rules:
- All placement entries on the same named cooperative pool must
agree on `where async_io`. The pool's drain loop is one-or-the-
other.
- `where async_io` is rejected on `pinned` entries. Pinned loci
own their own OS thread and have no shared drain loop to park
on.
- `where async_io` is rejected on pool `main`. The main pool
runs inline on the binary's primary thread, with no dedicated
worker to integrate epoll into.
See `spec/decisions.md § F.35` (forthcoming) for the
green-I/O substrate design + perf-axis trade-offs.
(Compare: rich / chunked / recognition projection classes are
genuinely three-way because N≈10, N≈30, and N≈300 are
different cost regimes at scale — memory has more genuine
intermediate ground than time does. Placement, even with M:N
pool partitioning, stays bimodal.)
#### Cross-class bus semantics
- **Cooperative → cooperative, same pool**: handler enqueues
on the pool's queue; runs at the next substrate cell on
that pool's drain loop. Sender never blocks.
- **Cooperative → cooperative, different pools**: cross-thread
post via the destination pool's queue; the destination pool's
drain thread wakes on the condvar broadcast (same machinery
as the cooperative→pinned path). Sender never blocks.
- **Any → pinned**: cross-thread post via the pinned locus's
lock-protected mailbox. Sender never blocks.
- **Pinned → any**: cross-thread post; pinned publisher doesn't
block waiting for delivery acknowledgement.
#### Implementation status (m26 + m27 + m28a + m28b + m28c; F.31 pending)
m25 wired the annotation through parse / typecheck / codegen.
**m26 ships cooperative semantics; m27 ships pinned threads
(run-only); m28a lifts pinned to full lifecycle; m28b lights up
cross-thread bus mailboxes — pinned loci can subscribe and
publish, with cells routed across threads via per-locus
mailboxes; m28c adds optional CPU-core affinity via
`pthread_setaffinity_np`.**
**F.31 (2026-05-23, Phase 1-5 + Phase 4a shipped):** the
placement-at-main surface and M:N cooperative pools. The
per-locus `: schedule` annotation is removed; the placement
choice moves to a `placement { }` block on `main locus`.
Cooperative subscribers can be partitioned across N pools
(N OS threads, each running its own
`lotus_coop_pool_worker` drain loop against its own per-pool
ring buffer); cross-pool bus dispatch reuses the m28b
condvar+memcpy machinery. The single-threaded-method invariant
— a locus's methods may be invoked only from its pool's thread
or via the bus — ships as a typecheck rule (Phase 5).
**Phase 4 v1 limit (handler-only cross-pool delivery).** The
runtime ships pool-aware **bus dispatch** for now: a subscriber
whose enclosing locus is placed on a non-`main` cooperative
pool gets its handler invoked on that pool's worker thread.
Lifecycle methods (`birth` / `run` / `dissolve` / `accept`)
still run on the main thread for cooperative-pool loci —
the codegen does NOT yet relocate them to the pool worker.
For state mutated only inside bus handlers this is enough to
honor the single-threaded-method invariant (the handler is
the only writer of locus state on the pool thread). State
touched by both lifecycle bodies and handlers on a non-main
pool is a Phase 4b concern; the typechecker doesn't flag it
today because lifecycle methods are not user-callable in the
`recv.method()` shape that Phase 5 checks. Plan: Phase 4b
moves lifecycle dispatch onto the pool worker via the same
queue mechanism (post "run_init" / "drain_exit" cells at
instantiation / scope-exit boundaries).
**Runtime pool inheritance for in-method-body instantiation
(2026-05-29).** A locus instantiated *inside a method or
bus-handler body that is itself executing on a pool worker*
inherits that pool **at runtime**. Codegen has no static
placement name for such a locus (placement keys on main-locus
`params` fields only), so the run-posting site and the
subscription-registration site resolve the pool as: the
compile-time-known pool when the locus IS a placed main-locus
field, else the pool whose worker is currently on-CPU
(`lotus_coop_pool_current()`, which reads the per-thread
`g_current_pool_tls`; NULL on the main thread). When the
resolved pool is non-NULL the child's `run()` is posted to it
(its own cell — and, on an `async_io` pool, its own parkable
coro, so a blocking `recv` in the child's `run()` parks that
child's coro rather than the spawning handler's) and its bus
subscriptions are tagged with that pool so dispatch routes to
the right worker. This is **gated on the child being owned
beyond the spawning scope** — `accept`'d, an owned param
field, or returned. A handler-local `let`-bound long-lived
locus is *not* owned (its deferred dissolve fires at the
handler's scope exit), so posting its `run()` would execute
after it's dissolved; those keep the prior behavior
(synchronous `run()`, global-queue subscription). The
canonical N-dynamic-children shape (per-connection handlers,
per-tenant workers) is therefore an `accept`'d child whose
`run()` holds the recv loop — it multiplexes on an `async_io`
pool by construction. See `spec/memory.md` § "Owned
param-field child allocation" for the companion arena rule
that makes the owned child's full subtree outlive the
spawning frame.
**Adapter loci instantiated inline in `bindings { Topic:
AdapterLocus { ... }; }` are NOT main-locus `params` fields**
and so receive no `placement { }` entry. Their `run()` recv-
loops need a dedicated thread by construction; the substrate
places them pinned-equivalent implicitly (same m90 routing +
pthread spawn that pre-F.31 fired via the adapter's
`: schedule pinned` annotation). The annotation goes away but
the behavior is preserved automatically — the bindings-inline
shape unambiguously signals "transport adapter with a recv
loop."
The implementation notes below describe pre-F.31 shapes (m25
through m28c). They remain accurate for the cooperative-only-
on-main-pool case, which is the v1-compatible default when
no `placement { }` block is declared.
**m26 (cooperative):** Each `<-` enqueues `(handler, self,
payload_copy)` cells onto a program-wide FIFO queue
(`@lotus.bus_queue.global`) instead of running handlers
inline. The scheduler drain loop pops cells one at a time
and invokes the handler — handler-atomic per substrate cell,
with cooperative yields BETWEEN cells rather than nested call
frames. Handlers may publish more events; drain continues
until empty.
Drain runs at the start of every `flush_dissolve_frame` —
before any long-lived locus dissolves — so subscribers process
pending cells while still alive. Plus an explicit `yield;`
statement (m26b) drains at user-placed points inside long
internal loops. v0 limitation: cells enqueued DURING a
dissolve are leaked.
The C runtime gained the queue surface:
```
ptr lotus_bus_queue_create(void)
void lotus_bus_queue_enqueue(ptr q, ptr handler, ptr self, ptr payload)
void lotus_bus_queue_drain(ptr q)
void lotus_bus_queue_destroy(ptr q)
```
m20's "memcpy payload into subscriber's arena" step happens at
ENQUEUE time (publisher's frame).
**m27 + m28a (pinned threads + full lifecycle):** Pinned-class
loci spawn a pthread at instantiation; the locus's full
declared lifecycle (birth → run → drain → dissolve, each only
if declared) executes on that thread, in order. Main thread
continues immediately after spawn. At scope exit (deferred-
dissolve flush), `pthread_join` blocks until the pinned
thread has finished its lifecycle and returned; the main
thread's only remaining work for a pinned entry is the join
plus the locus's arena destroy wholesale (drain / dissolve are
SKIPPED on the main side — they ran on the pinned thread).
m28a synthesizes a per-locus `__pinned_main_`
function whose signature matches pthread's start-routine
contract directly (`ptr (ptr)`); pthread_create gets that
function pointer with `self_ptr` as its argument. No C-side
adapter, no thread_args struct. The synthesized body simply
calls each declared lifecycle method in sequence, then
returns null.
**m28b stage 1 (inline-payload queue):** Bus queue cells now
carry an inline `[u8; 512]` payload buffer (with `pthread_mutex_t`
guarding the cell array) instead of a pointer to subscriber-arena
memory. The publisher memcpy's into the cell at enqueue; the
drain (running on the subscriber's thread) memcpy's from the
cell into the subscriber's arena before invoking the handler.
This makes the queue the single point of cross-thread
synchronization: each per-locus arena stays single-threaded
territory, the boundary between layers is where the lock lives.
Per spec/memory.md, "every locus boundary copies the payload"
still holds — just with two memcpy's per cell instead of one.
**m28b stage 2 (cross-thread mailboxes):** Each pinned locus
that declares `bus subscribe` allocates its own
`lotus_mailbox_t` at instantiation: a bounded ring buffer with
`pthread_mutex_t` + `pthread_cond_t` + a shutdown flag, sharing
the same inline-payload cell shape as the global queue. The
locus's struct grows a `__mailbox: ptr` field to hold it.
The bus entry table grows from `{subject, self, handler}` to
`{subject, self, handler, mailbox}`. Cooperative subscribers
register with `mailbox = NULL`; pinned subscribers register
with their mailbox pointer. At dispatch time, the `bus_dispatch`
fn loads `entry.mailbox` and branches: null → enqueue on the
global cooperative queue (handler runs on the cooperative
thread); non-null → `lotus_mailbox_post` on the pinned
subscriber's mailbox (handler runs on the pinned thread).
The synthesized `__pinned_main_` body grows a mailbox
loop between `run()` and `drain()`: it calls
`lotus_mailbox_drain_one`, which blocks on the condvar until
either a cell arrives (returns 1, after dispatching the
handler) or shutdown is signaled with empty queue (returns 0,
breaking the loop). Pending cells flush before the loop
returns 0 even after shutdown — the order check is "queue
empty AND shutdown."
Coordinated shutdown: at the deferred-dissolve flush, the main
thread calls `lotus_mailbox_shutdown` on the pinned locus's
mailbox (sets the flag + broadcasts the condvar), then
`pthread_join`. The pinned thread observes the empty+shutdown
condition, breaks its loop, runs `drain()` and `dissolve()`,
and exits — main joins, then destroys the mailbox and the
arena.
Per The Design / lotus, this is the canonical "any → pinned"
bus path: publisher and subscriber sit in different layers of
the lotus, the substrate cost lives at the layer boundary
(the mailbox lock + the inline payload's two memcpy's), and
each arena stays single-threaded territory. Bimodality holds.
Still gated: pinned loci cannot declare `accept()` (children
of pinned would need cross-thread cascade-dissolve
coordination, which is meaningful new infrastructure beyond
m28b's mailbox post-and-continue) or closures (cross-thread
violation routing). Codegen errors clearly if those are
present.
**m28c (CPU-core affinity):** When a pinned locus declares
`: schedule pinned(core = N)`, codegen emits a call to
`lotus_set_core_affinity(tid, N)` immediately after
`pthread_create` succeeds. The C-side helper wraps
`pthread_setaffinity_np` (with a `cpu_set_t` zeroed and bit N
set) so codegen doesn't have to know the cpu_set_t layout
(opaque + size-variable across glibc versions). Best-effort
semantics: if the requested core is unavailable (e.g., CI box
with fewer cores than the source declares) or the syscall is
denied, the runtime silently falls back to ordinary OS
scheduling rather than refusing to run the binary. The
underlying bimodality is unchanged — `pinned(core = N)` is a
refinement WITHIN the pinned mode, not a third position.
**Topology Phase 1a (cpuset affinity):**
`pinned(cores = A..B)` / `pinned(cores = A..=B)` /
`pinned(cores = {a, b, c})` generalize the single core to a
core **set**: the thread's affinity mask is the whole set and
the OS schedules it freely within it — a range carves out an
isolation domain rather than picking one CPU. Bounds are
integer literals (placement is a closed-world deployment
seam), so the compiler expands the spec statically — sorted,
deduplicated — into a constant array and emits one call to
`lotus_set_core_affinity_set(tid, cores, count)` after
`pthread_create`. Range inclusivity follows expression
ranges: `..` excludes the upper bound, `..=` includes it. The
typechecker rejects a spec that selects no cores (`4..4`,
`8..=4`) and a duplicated set element; whether the cores
exist on the deploy box stays best-effort at runtime — the
C helper skips out-of-range indices and applies the mask only
if at least one valid core remains. CPU affinity is
Linux-only: on other hosts (macOS) both helpers are compiled
as no-ops and the loci run unpinned. `pinned(core = N)`
continues to route through the single-core helper unchanged.
Linker dependency: clang invocation now passes `-lpthread`
unconditionally; small fixed cost in the resulting binary
(libpthread is on every modern Linux).
### Bus message router
The runtime's bus is **transport-agnostic**. From the
framework's perspective, a transport is the bus kernel projected
through a parameter regime: NATS and UDP multicast and TCP and
Unix sockets are the same primitive (typed pub-sub) at different
(B, c, σ, φ) values. The runtime knows about subjects, channels,
and modes; specific transports come from stdlib (`std::bus::*`).
- **Subject → handler dispatch.** Declared `bus subscribe
"..." as fn` declarations are wired by the runtime at
startup; inbound messages on declared subjects route to the
declared handler.
- **Outbound publish.** Declared `bus publish "..."` allows
emit from any handler return; the runtime routes to the
configured transport.
- **Multi-transport dispatch.** A single binary may bind
different channels to different transports (a real-time event
channel to UDP multicast; a control channel to NATS; a
test channel to in-memory). The router maintains per-channel
transport bindings established at deployment time from
config.
- **Transport adaptation interface.** Cross-host transports
(NATS, MQTT, TCP-with-framing, custom) plug in via
`interface std::bus::Adapter` — a contract for user-supplied
loci that ship messages on whatever protocol they choose. The
contract definition lives in `runtime/stdlib/bus.hl`; concrete
adapter implementations live in user code or downstream
packages, NOT in std. The substrate-provided `unix(...)`
transport is in the runtime itself (substrate-guaranteed
atomic delivery via SOCK_SEQPACKET) and doesn't go through
the Adapter interface.
**v1.x source surface.** Subjects are now declared as typed
top-level `topic Foo { payload: T; subject: "..."; }` decls
(with optional `: Parent` for hierarchical wire subjects);
deployment-time bindings live in the `main` locus's
`bindings { Topic: ; }` block. Two transport shapes
ship: substrate-provided `unix("/path", role: ...)` and
user-supplied adapter loci named directly on the right-hand
side (any locus satisfying `__StdBusAdapter` —
`fn send(subject: String, bytes: Bytes)` — qualifies).
In-memory delivery is absence-of-entry. Adapter bindings let
protocol-layer transports (NATS, MQTT, TCP-with-framing,
custom JSON-over-WebSocket) live in user code without the
language having to enumerate protocol variants. See
`spec/semantics.md` "Topic declarations → Phase 2" for the
full surface, including the closed-world topology
optimization that elides bus dispatch for unambiguous
intra-locus and single-hop parent→child tower patterns when no
binding is declared.
**Adapter dispatch.** At codegen, an adapter binding
instantiates the adapter locus into the program-lifetime
payload arena (same m90 routing the `-> LocusRef(L)` return
path uses), resolves the locus's `send` method's fn pointer,
and registers the (self, send_fn) pair with the runtime via
`lotus_bus_register_remote_adapter`. The runtime stores both
in `lotus_bus_remote_entry_t`'s adapter slot. Outbound fanout
packages the wire bytes as an Hale-level `Bytes` value
(built via `lotus_bytes_from_buf` against the lazy global
payload arena) and indirect-calls
`send_fn(self, subject, bytes)`. No vtable lookup is needed
at the runtime layer — codegen resolved the method at
binding-emit time.
**Adapter inbound (m105).** Adapters receiving wire-bytes
from their protocol layer call `std::bus::__local_dispatch(
subject, bytes)`; the primitive backs onto
`lotus_bus_dispatch_wire`, which looks up the subject's
registered deserialize fn in `g_bus_entries` (same table the
publish-side fanout consults), reconstructs the struct-layout
bytes, and fans into local subscribers via
`lotus_bus_local_dispatch`. Symmetric to the unix reader-
thread path; out-of-band recv loops (any code holding wire
bytes for a bound subject) can use this too.
**SHM ring substrate (Form K5).** POSIX shared-
memory ring backing the zero-copy bus route. Six C primitives in
`runtime/lotus_shm_ring.c`, linked unconditionally so user
programs that bind a topic to a zero_copy route resolve cleanly:
- `lotus_shm_ring_open(name, slot_size, slot_count)` — open or
attach (creates if it doesn't exist; validates header on
attach). Returns a per-process handle.
- `lotus_shm_ring_claim(ring)` — publisher gets a pointer to
the next slot. v1 is single-producer; never fails.
- `lotus_shm_ring_commit(ring)` — release-orders the slot
writes before atomic-incrementing the published seqno.
- `lotus_shm_ring_published(ring)` — acquire-load the seqno
for subscriber-side polling.
- `lotus_shm_ring_read_slot(ring, seqno)` — subscriber gets a
pointer to the slot for `seqno`. Returns NULL if not yet
committed OR wrapped past (slow consumer).
- `lotus_shm_ring_close(ring)` — unmap + close fd; unlink the
SHM object if this handle created it.
Layout in SHM: a 64-byte cache-aligned header
(`lotus_shm_ring_header_t` — magic, slot_size, slot_count,
atomic seqno) followed by N slots of `slot_size` bytes each.
Header magic + sizes are validated on attach to catch ABI
mismatches across binaries pinned to the same ring name.
**Foreign-layout consumer (Proposal B).** Two more
primitives read an *externally*-defined ring described by a
`ring_layout` (see semantics.md § "Foreign rings"), rather than
the native LRSRNG1 shape:
- `lotus_shm_ring_open_layout(name, desc)` — attach an existing
foreign segment READ-ONLY (never creates), `fstat` for the map
length, validate `magic`/`version`, read `buffer_size` for the
data-region capacity. `desc` is a `lotus_shm_layout_t` built
from a flat 16-entry uint64 descriptor codegen emits.
- `lotus_bus_register_subscriber_shm_ring_layout(subject, name,
desc_words, self, handler)` — open via the above and spawn a
`byte_records` reader thread that walks `[len_prefix][payload]`
records (modular over `capacity`, skipping `pad_sentinel`
tail-pads, advancing by `align_up`). Shares the native
subscriber registry + `atexit` teardown; a layout subscriber is
marked `is_layout` and torn down via `lotus_shm_ring_close_layout`.
The producer side (Proposal B M3a) mirrors these:
- `lotus_shm_ring_create_layout(name, desc, capacity)` — CREATE +
own the segment (size `data_at + capacity`, write the
magic/`version`/`buffer_size` header, zero the cursor). Attaches
read-write without re-init if it already exists.
- `lotus_bus_register_shm_ring_layout(subject, name, desc_words,
capacity)` — create via the above + register a producer (one per
subject). `lotus_bus_publish_shm_ring_layout(subject, value,
size)` frames one `byte_records` record (the inverse of the
reader: reserve `align_up`, `pad_sentinel` at the wrap, write the
length prefix + payload, release-store the cursor). The producer
rings are closed + `shm_unlink`'d at `atexit`.
Field reads/writes are host-native endianness (the foreign producer and Hale are
both little-endian x86-64). v1 is `byte_records` only; the `slots`
framing kind and a zero-copy writable producer view are post-v1.
v1 scope: single-producer, multi-consumer; in-memory delivery is
in scope (POSIX shm_open works intra-machine cross-process).
Multi-producer (CAS-based claim), back-pressure / timeout
modes, and named-ring registry are post-v1. The Hale-side
`fallible(ClaimError)` signature is reserved for those; v1's
`claim()` never actually fails.
**Lifecycle / cleanup.** Both
`lotus_bus_register_shm_ring` (publisher) and
`lotus_bus_register_subscriber_shm_ring` (subscriber) register
a single `atexit` hook on first call. The hook:
1. Signals every subscriber reader thread to stop via an
atomic `should_stop` flag.
2. `pthread_join`s each reader thread, ensuring no in-flight
handler is interrupted.
3. Frees the subscriber state allocated by the registration
call.
4. `lotus_shm_ring_close`s every ring opened in this process
— which `shm_unlink`s the ones this process created
(`owns_unlink=1`), keeping `/dev/shm/` clean across
restarts.
The atexit hook runs on a clean process exit (return from
main, `exit(3)`). Signal-driven termination (SIGTERM, SIGKILL,
`_exit`) bypasses atexit and leaves the SHM namespace entry
behind until reboot or manual `shm_unlink`. A future v1.x
SIGINT/SIGTERM handler can fold into this same teardown.
**Constraint: subscriber handlers must not call `exit()`.** The
handler runs on the reader thread; calling `exit()` from inside
the handler invokes atexit on the reader thread, which then
attempts to `pthread_join` itself (undefined behavior). Use
`_exit()` if a handler needs to terminate the process
immediately.
Codegen surface for K5 lands in K4 (route-selection +
slot-locus synthesis) and K6 (subscriber view + epoch guard).
K5 ships the substrate; user code can't reach these symbols
directly without going through the slot-locus surface a
zero_copy binding produces.
### Closure-test infrastructure
- **Default epoch is `dissolve`.** Closures with no `epoch`
clause evaluate at the locus's dissolution. Other epochs:
`epoch tick`, `epoch duration(...)`, `epoch birth`,
`epoch explicit` — runtime-managed per declaration.
- **Accumulator engine.** For each `closure name { ... }`, the
runtime maintains accumulators for the left and right sides
of `~~`, scoped per epoch (when accumulation is needed; not
needed for one-shot self-referential closures like
`self.x ~~ self.y within 0`).
- **Band checking + reporting.** At each epoch boundary, the
runtime evaluates left and right expressions, checks the
band, and emits a typed `ClosureReport` event the application
can subscribe to via bus.
- **Collapse vs. explosion.** A closure-pass at any epoch is
silent. A closure-fail flips an "exploded" flag on the locus.
At dissolve, if exploded, the parent's
`on_failure(self, ClosureViolation { ... })` is invoked with
a typed event carrying closure name, epoch, left/right
values, tolerance, diff. Distinct from hard substrate
failures (OOM, divide-by-zero, null-deref from
miscompilation) — those terminate the process directly
without the ClosureViolation routing path. See
decisions §F.9.
- **Recovery-event interaction.** `persists_through(...)` and
`resets_on(...)` clauses are honored at recovery time; the
accumulator is preserved or zeroed per declaration. The
exploded flag itself persists across `restart_in_place` and
`quarantine` (per default; future `clear_violation_on(...)`
clause may override).
### Perspective infrastructure
- **The global slot.** Each `perspective P` has one program-global
`{ data, vtable }` slot (`__persp.`). Every holder of
`perspective(P)` dispatches through it — a load plus a predicted
indirect call, near-direct cost. A program that declares no
perspectives pays nothing.
- **Live swap (`reperspective`).** Re-points the slot at a new
`serves P` impl with a single atomic store, redirecting every
call site at once. State-preserving across impls of one footprint:
the `{ data, vtable }` split means `data` — the live, arena-backed
state — is untouched and only the vtable changes. When the
perspective declares a bus surface, the swap also re-points its
subscriptions on that same `data`. (See `spec/semantics.md`
§ Perspectives.)
- **Wire hot-load (aspirational).** Transport-driven redeploy —
decode a serialized perspective against the compiled-in schema,
gate on `stable_when`, atomically install with no torn read — is
specified but not yet shipped. See `spec/semantics.md`
§ "Perspective hot-load".
### Failure handling
- **Failure = `ClosureViolation` propagation.** Any `closure`
assertion that fails in a locus body produces a
`ClosureViolation` record routed to the parent's
`on_failure(child, err)` handler per **F.9**. The parent
picks one of `restart` / `restart_in_place` / `quarantine` /
`reorganize` / `bubble`, or absorbs (returns without calling
any). A violation that bubbles past the root exits the
process non-zero with the violation report on stderr.
- **No source-level panic / exceptions.** Hale has no
`panic(msg)`, `assert(cond)`, `throw` / `catch`, or
implicitly-propagating exception machinery. Failure is
either structural (closure violation; parent-policy
recovery, Erlang let-it-crash with the parent locus as the
supervisor) or value-level via `fallible(E)` (v1.x-FORM-1
addressing protocol; every fallible call MUST be addressed
by an `or` clause at the immediate caller). The two
channels are orthogonal at every frame except the implicit
main locus's root, where a value error escaping past every
enclosing `fallible` frame triggers `lotus_root_panic` —
the runtime's only value-error escape valve. See
`spec/semantics.md` § "Process exit".
### Time
- **Monotonic + wall-clock.** `time::now()` and
`time::monotonic()` are runtime-provided. `time::monotonic()`
returns a `Duration` (i64 nanoseconds since an unspecified
reference); only meaningful for elapsed-time differences.
Backed by `clock_gettime(CLOCK_MONOTONIC)`. `time::now()` (C7,
pond follow-up) returns
wall-clock seconds since the Unix epoch as `Int` via
`clock_gettime(CLOCK_REALTIME)`; observation only — NTP
slewing and leap seconds can warp the value, so
`time::monotonic` stays the basis for scheduling. Richer
`Time`-typed wall-clock (with calendar arithmetic) is
deferred until a consumer surfaces a concrete date-shape
need. Mocking is available for tests via
`time::mock_clock(...)` (stdlib).
- **Monotonic-only scheduling.** Every scheduling primitive in
Hale — `time::sleep`, `time::tick`, the cooperative
scheduler's deadline queue — is grounded on the monotonic
clock. NTP slewing, leap seconds, and wall-clock jumps cannot
warp scheduling decisions. `time::sleep` retries on EINTR
using the kernel's reported remaining time, so a delivered
signal does not shorten the total sleep. `CLOCK_REALTIME` is
used by `time::now()` for wall-clock observation only and
has no scheduling role.
- **Implementation invariant.** `time::sleep(d)` lowers to
`clock_nanosleep(CLOCK_MONOTONIC, 0, &req, &rem)` with EINTR
retry — important for a system targeting high-precision clock
semantics.
### I/O — minimal
- **stdout / stderr** for `print` / `println`. That's it for
runtime-level I/O. Files, networking, etc. live in stdlib.
- **Errno surface helpers** (2026-05-16, used by the fallible
`std::io::fs::*` / `std::io::tcp::*` wrappers):
- `lotus_get_errno() -> i32` — surfaces the current platform
`errno` to LLVM. Each fallible wrapper calls this
immediately after the failing primitive (POSIX errno is
sticky until the next syscall sets it).
- `lotus_io_error_kind(errno_val: i32) -> *const char` —
maps errno to a stable kind-tag string (`"not_found"`,
`"permission_denied"`, `"is_dir"`, `"already_exists"`,
`"would_block"`, `"connection_refused"`, `"timeout"`,
`"host_unreachable"`, `"broken_pipe"`, `"interrupted"`,
..., catch-all `"io"`). Returns a static-table pointer;
caller must not free.
### Text + string primitives (v1.x adds)
- `lotus_str_parse_float(s) -> double` / `lotus_str_can_parse_float(s) -> int`
— v1.x-16. Strict trailing-NUL parse; 0.0 on failure paired
with a bool predicate. Mirrors the parse_int contract.
- `lotus_text_base64_decode(s) -> Bytes*` — v1.x-16. Standard
alphabet, whitespace tolerated, non-alphabet / wrong padding
returns empty Bytes. Inverse of `lotus_text_base64_encode`.
- `lotus_str_builder_new()` / `_append(b, s)` / `_len(b) -> i64` /
`_finish(b) -> char*` — v1.x-15. Doubling-realloc malloc
buffer. N appends are amortized O(N). `finish()` copies into
the bus payload arena (program-lifetime) and frees the
builder.
- `lotus_bytes_builder_new(i64 initial_cap) -> ptr` /
`_append(ptr handle, ptr chunk) -> i64 status` /
`_len(ptr handle) -> i64` /
`_finish(ptr handle) -> Bytes*` /
`_shift_front(ptr handle, i64 n)` /
`_clear(ptr handle)` /
`_snapshot(ptr handle) -> Bytes*` /
`_view(ptr handle) -> Bytes*` /
`_free(ptr handle)` — C10 / Phase 0 / Phase-2 (1)
(2026-05-19, pond/websocket follow-up). Binary-safe sibling
of the str-builder family. Append reads the chunk's
`[i64 len]` prefix instead of `strlen`; finish emits a
length-prefixed Bytes blob with no trailing NUL.
In-place ops: `shift_front` memmoves the tail to the head
and drops n bytes (capacity preserved). `clear` sets len=0
(capacity preserved). `snapshot` copies the current
`[0..len)` into a fresh Bytes blob in the bus payload
arena, builder unchanged. `view` returns a non-owning Bytes
pointer aliasing the builder's inline `[i64 len][u8 data]`
region — zero allocation, zero copy; lifetime valid until
the next mutation on the source builder. `free` disposes
the malloc-backed buffer.
**F.30 type promotion.** The Hale-visible
method surface returns `BytesView` / `StringView`
(typecheck-distinct from `Bytes` / `String`). The view-to-
owned upgrade paths (`std::bytes::clone`, `std::str::clone`)
are backed by `lotus_bytes_clone(arena, src)` (new) and
`lotus_str_clone(arena, src)` (existing m49).
**F.30b view layout + epoch guard (2026-05-22 PM compaction).**
The `_view` / `_text_view` C primitives return a 16-byte
by-value struct — no arena allocation in the hot path. Pre-
compaction was a 24-byte struct heap-allocated per call,
and that allocation was the dominant residual chunk-
allocation trigger in long-running recv loops:
```c
#define LOTUS_VIEW_EPOCH_STATIC ((int64_t)-1)
typedef struct lotus_view {
void *src; // builder ptr (epoch >= 0, real view)
// OR static data ptr (epoch == -1,
// static-lifetime view from
// lotus_view_from_static_data or
// the null-handle path of
// builder_view / builder_text_view).
int64_t epoch; // stamped mutation_epoch, or the
// static sentinel.
} lotus_view_t;
```
The `{void*, int64_t}` layout fits SysV AMD64's "two
INTEGER eightbytes ≤ 16 bytes" return-by-value rule —
both registers (`rax`, `rdx`) carry the view; arg-by-value
passes in two integer arg registers. The underlying data
pointer is *recomputed* at unpack time from
`((lotus_bytes_builder_t*)v.src)->buf` (Bytes-shape:
`buf - 8`; C-string shape: `buf`), so the view itself
doesn't need to store it.
`lotus_bytes_builder_t` gains an `int64_t mutation_epoch`
field bumped by every mutating op (`append`, `append_slice`,
`shift_front`, `clear`, `advance`). Codegen at view-coerce
sites emits a call to `lotus_bytes_view_data` /
`lotus_str_view_data`, which compares the stamped epoch
against the live epoch and calls `lotus_view_stale_panic`
(noreturn — stderr + `_exit(1)`) on mismatch. The 5b
literal-default coercion calls `lotus_view_from_static_data`
to construct the view in-register with the static
sentinel; the helpers skip the epoch check on that branch
and return `v.src` directly as the underlying data pointer.
**Memory layout (Phase-2 (1)).** The builder header is
`{cap, buf, mutation_epoch}`; the data area is preceded
inline by an 8-byte length prefix matching the Bytes ABI:
```
malloc'd region: [int64_t len][u8 data[cap]][NUL]
^
buf
```
`view(b)` returns a 16-byte `lotus_view_t` whose `src`
field is the builder pointer; the read-site helper
recomputes the data pointer as `b->buf - 8` (Bytes-shape,
suitable for `lotus_bytes_len` / `lotus_bytes_at` /
`lotus_bytes_data`). Append / append_slice / shift_front /
clear / advance all update the inline prefix in sync with
the data mutation AND bump `mutation_epoch`. Cost: one
extra pointer dereference per len access vs the prior
`{cap, len, buf*}` shape, plus a one-load epoch check at
every view-coerce site. Zero arena allocation per view()
call (the dominant residual chunk-alloc trigger pre-
compaction). `lotus_str_builder_t` (for `std::str::*`)
keeps the prior layout — no view surface there yet.
These primitives are no longer the user-facing surface;
they're the C externs called by the
`std::bytes::BytesBuilder` stdlib locus
(`crates/hale-codegen/runtime/stdlib/bytes_builder.hl`).
See `spec/decisions.md` § F.28 for the rationale
and the locus's method shape. The locus-side calls reach
these via internal `std::bytes::builder::__*` path-call
dispatch.
**ABI notes.** `_new` takes `int64_t
initial_cap` (previously zero-arg, hardcoded 64) — values
`<= 0` are treated as the legacy default. `_append`
returns `int64_t status` (1=ok, 0=fail on realloc-NULL
or null-handle) — previously void; the status return is
what the locus's `append` method checks before routing
through `violate alloc_failed` per F.27.
**Builder handles are NOT layout-compatible with regular
Bytes blobs.** The struct shape is
`{ size_t cap; size_t len; char *buf; }` (24 bytes);
Bytes blobs are `[int64_t len][u8 data[]]`. So
`lotus_bytes_at(builder, i)` / `lotus_bytes_len(builder)`
read the wrong slots if a builder handle is passed as a
Bytes value. The Hale-level enforcement (`BytesBuilder`
as its own locus type) makes that mistake impossible to
express; this note is the C-side mirror — anyone calling
these primitives directly from C / Rust must keep the
distinction.
- `lotus_tcp_recv_into(fd, builder, max_bytes) -> i64` /
`lotus_tls_recv_into(handle, builder, max_bytes) -> i64` /
`lotus_udp_recv_into(fd, builder, max_bytes) -> i64` —
2026-05-19 (Phase 1, pond/websocket follow-up).
Caller-provided destination at the syscall layer. Reads
directly into the builder's tail; grows on insufficient
headroom; bumps the builder's len by the count read.
Return semantics mirror POSIX read(2): `> 0` bytes
appended, `= 0` peer closed cleanly (TCP) / zero-length
datagram (UDP), `< 0` error. EINTR retried internally.
**A `SO_RCVTIMEO` timeout is distinguished from a fatal
error: `-2` = "would-block / timed out, retryable"; `-1`
= fatal** (TCP: `EAGAIN`/`EWOULDBLOCK`; TLS: `SSL_read`
→ `SSL_ERROR_WANT_READ`/`WANT_WRITE`). The `-2` only
arises when the caller has set a recv timeout (opt-in via
`set_recv_timeout`), so it's backward-compatible — a caller
that treats all `< 0` as error keeps working; a liveness
loop checks for `-2` to run its ping/pong instead of
tearing the connection down. No allocation in
`g_bus_payload_arena`. No allocation in `g_bus_payload_arena` —
closes the residual ~80% of the pond/websocket recv-loop
leak that Phase 0's in-place builder ops surfaced (the
syscall layer's own `[i64 len][body]` blob per call).
Helpers `lotus_bytes_builder_reserve(handle, n)` +
`lotus_bytes_builder_advance(handle, n)` factor the
grow + offset-bump so `lotus_tls.c` (separate translation
unit) can implement its recv_into without seeing the
builder struct layout.
- `lotus_str_lower(s) -> char*` / `lotus_str_upper(s) -> char*`
— ASCII case folding. One-pass byte-level fold; non-ASCII
bytes pass through unchanged. Allocates in the bus payload
arena. Used by `http_request_header` for RFC 7230
case-insensitive lookup.
- `lotus_str_trim(s) -> char*` — strip ASCII whitespace
(space / tab / CR / LF) from both ends. Arena-anchored.
- `lotus_str_replace(s, needle, rep) -> char*` — greedy
non-overlapping substring replace. Two-pass (count, then
fill) so the output is right-sized in one arena alloc.
Empty needle is a no-op.
- `lotus_str_repeat(s, n) -> char*` — n copies of s
concatenated; n <= 0 returns empty. Single arena alloc.
- `lotus_str_pad_left(s, width, pad) -> char*` /
`lotus_str_pad_right(s, width, pad) -> char*` — width-aligned
output using the first byte of `pad` (default space) as
the fill char. No truncation: `len(s) >= width` returns
s unchanged.
### Process control
- **Exit codes.** `main()` returning `()` exits 0; returning
`int` exits with that code. Panics exit non-zero.
- **Signal handling.** SIGINT / SIGTERM trigger `drain` →
`dissolve` on the root locus. Stdlib provides finer-grained
control if needed.
- **SIGPIPE globally ignored** (added 2026-05-17, C2). The
prelude installs `signal(SIGPIPE, SIG_IGN)` once at
`lotus_io_init` so writes to a closed pipe (subprocess stdin,
closed TCP socket, etc.) surface as `EPIPE` through the
IoError channel instead of synchronously killing the parent.
Applies process-wide — no opt-out.
- **Subprocess lifecycle** (added 2026-05-17, C2 — see
[`spec/stdlib.md` § std::process](stdlib.md) for the API
surface). Every spawned child gets its own process group via
`setpgid(0, 0)` in the post-fork prelude. Chosen over
`prctl(PR_SET_PDEATHSIG, SIGKILL)` for POSIX portability
(macOS / BSD parity); a controlled `Child.dissolve()` covers
the orderly-shutdown path. `Child.dissolve()` closes the
three pipe fds and kill-escalates idempotently (SIGTERM →
100 ms grace → SIGKILL → waitpid; `ESRCH` / `ECHILD` count
as success) so an unwaited child doesn't leak zombies on
scope exit. The `std::process::run` synchronous form drains
stdout + stderr via interleaved `poll()` so the child can
write to either stream without deadlocking; 16 MiB cap per
stream.
### stdout buffering
stdout is **line-buffered** for the lifetime of the program,
regardless of whether it's attached to a TTY or a pipe. The
main prelude calls `setvbuf(stdout, NULL, _IOLBF, 0)` once
before any user code runs.
The default libc behavior (fully-buffered when stdout isn't a
TTY) silently dropped output for any program that printed then
blocked on a syscall — `println("READY"); accept_loop();` made
"READY\n" invisible to a piped consumer until the buffer
filled or the program exited. Test oracles, supervisors waiting
for a READY handshake, and log tailers all hung. Line-buffering
matches Python `python -u` discipline and Go's default; `\n`-
terminated `println` calls flush immediately under any stdout
target.
stderr is line-buffered by POSIX already; the runtime doesn't
touch it.
## What's NOT in the runtime (lives in stdlib instead)
- Specific bus transports (NATS, UDP, etc.)
- File I/O
- Networking (sockets, HTTP)
- JSON / protobuf / msgpack encoding
- Most collections beyond what the language has built-in
- Math beyond `sum` / `prod` (which are language-native)
- Statistics
- Linear algebra
- String manipulation beyond literal handling
- Time arithmetic beyond comparison and arithmetic
- Logging / metrics / tracing
These are bundled with the toolchain (no separate install) but
require explicit `import std::...`.
## Form-vec runtime (v1.x-FORM-1)
The `@form(vec)` form lowers to a contiguous growable buffer
implemented in C. See `spec/forms.md` for the form contract and
synthesized method set; this section documents the runtime
shape.
### C struct layout
Each `@form(vec)` locus's heap slot lowers to an inline
struct:
```c
typedef struct {
size_t cap; // allocated capacity (elements)
size_t len; // number of valid elements
char *buf; // contiguous element array
} lotus_vec__t;
```
The `` suffix is conceptual — codegen monomorphizes per
cell type T, but the runtime primitives operate on the
common prefix layout via `void *` casts. All `lotus_vec_*_t`
typedefs share the `{cap, len, buf}` prefix.
### Primitive functions
Defined in `crates/hale-codegen/runtime/lotus_arena.c`
(v1.x-FORM-1 PR4):
| Function | Behavior |
|-------------------------------------------------------|----------|
| `void lotus_vec_init(void *v)` | Zero-init: cap=0, len=0, buf=NULL |
| `void lotus_vec_push(void *v, size_t es, const void *x)` | Append; doubles cap on overflow |
| `int lotus_vec_get(void *v, size_t es, int64_t i, void *out)` | Bounds-checked read; returns 1=OK, 0=out-of-bounds |
| `int lotus_vec_set(void *v, size_t es, int64_t i, const void *x)` | Bounds-checked in-place write; returns 1=OK, 0=out-of-bounds (does not extend the vec) |
| `int lotus_vec_pop(void *v, size_t es, void *out)` | Returns 1=OK, 0=empty |
| `int64_t lotus_vec_len(void *v)` | Element count |
| `int lotus_vec_is_empty(void *v)` | 1=empty, 0=non-empty |
| `void lotus_vec_destroy(void *v)` | `free(buf)`; called at locus dissolve |
| `void lotus_vec_sort_int(void *v)` | In-place ascending sort of an `int64_t`-cell vec via `qsort` |
| `void lotus_vec_sort_float(void *v)` | In-place ascending sort of a `double`-cell vec; NaN treated as equal-to-anything |
| `void lotus_vec_sort_string(void *v)` | In-place ascending sort of a `char *`-cell vec under `strcmp` ordering |
| `void lotus_vec_sort_by(void *v, size_t es, int (*cmp)(const void *, const void *, void *), void *cookie)` | `qsort_r` wrapper; cmp is a codegen-synthesized per-(cell_type, direction) trampoline |
`es` (elem_size) is the cell type's size in bytes — codegen
passes `sizeof(T)` at each call site.
### Growth policy
- Initial: cap=0, no allocation at locus birth.
- First push: allocates a 4-element buffer.
- Each overflow: doubles cap; `realloc`s. Old contents copied
by realloc; previous buf freed.
- Shrink: not implemented in v1. Buf released at dissolve.
### Failure shapes
- `lotus_vec_get` / `lotus_vec_set` / `lotus_vec_pop` return 0
on contract break (out-of-bounds / empty). Codegen wraps this
into the `Ty::Fallible { success: T, payload: IndexError }`
surface (Unit-success for `set`) via a small adapter that
synthesizes the `IndexError` struct from the bool + the call
args (shipped v1.x-FORM-2 PR5/6; `set` added 2026-05-16).
- `lotus_vec_push` OOM routes through the substrate-trap →
closure-violation channel per the two-channel rule (shipped
v1.x-FORM-2 PR6).
- Sort family (`sort`, `sort_by`, `sort_desc_by`, added
2026-05-16) is infallible from the language surface; `sort_*`
C wrappers do not return a status code. If the user-supplied
comparator in `sort_by` faults (a fallible call inside the
comparator body raised through `or raise`), the fault
propagates and `qsort_r` stops mid-sort — the vec is left
with every element still present, ordering partially applied.
## Form-hashmap runtime (v1.x-FORM-4)
The `@form(hashmap)` form lowers to an intrusive open-addressing
hash table implemented in C. See `spec/forms.md` for the form
contract and synthesized method set; this section documents the
runtime shape.
### C struct layout
Each `@form(hashmap)` locus's pool slot lowers to an inline
struct:
```c
typedef struct {
size_t cap; // power-of-two slot count
size_t len; // live entry count
size_t key_size; // sizeof(K), set at init
size_t value_size; // sizeof(S), set at init
int key_type_tag; // 0 = Int, 1 = String
char *slots; // cap * (1 + key_size + value_size) bytes
} lotus_hashmap_t;
```
Each slot is `1 + key_size + value_size` bytes:
```
[occupied: u8] [key: key_size bytes] [value: value_size bytes]
```
`occupied = 0` means empty. Backward-shift deletion (no
tombstones) — probes terminate at the first empty slot.
The C ABI is type-erased: codegen passes `key_size` /
`value_size` at init time, and per-call sites pass raw `void *`
key/value pointers. Codegen GEPs the indexed-by field on the
caller's side to derive the key pointer before each `set`.
### Primitive functions
Defined in `crates/hale-codegen/runtime/lotus_arena.c`
(v1.x-FORM-4 PR4):
| Function | Behavior |
|----------|----------|
| `void lotus_hashmap_init(void *m, size_t key_size, size_t value_size, int key_type_tag)` | Allocate `cap=8` slots, zero them; freeze key/value sizes and key-type tag |
| `void lotus_hashmap_set(void *m, const void *key, const void *value)` | Insert or replace; grow at load factor 0.7 |
| `int lotus_hashmap_get(void *m, const void *key, void *out_value)` | Bounds-checked read; returns 1=OK, 0=missing_key |
| `int lotus_hashmap_has(void *m, const void *key)` | 1=present, 0=missing |
| `int lotus_hashmap_remove(void *m, const void *key)` | 1=removed, 0=missing |
| `int64_t lotus_hashmap_len(void *m)` | Live entry count |
| `int lotus_hashmap_is_empty(void *m)` | 1=empty, 0=non-empty |
| `void lotus_hashmap_destroy(void *m)` | `free(slots)`; called at locus dissolve |
### Key types and hashing
| `key_type_tag` | Type | Hash function | Equality |
|---|---|---|---|
| `0` (LOTUS_HASHMAP_KEY_INT) | `int64_t` | Knuth multiplicative (`k * 0x9E3779B97F4A7C15`) | `==` on i64 |
| `1` (LOTUS_HASHMAP_KEY_STRING) | `const char *` (NUL-terminated) | FNV-1a over the bytes | `strcmp == 0`, with pointer-identity fast path |
Other key types (Bytes, custom structs, enum tags) are not
supported at v1; codegen rejects `@form(hashmap)` with a
focused diagnostic when the indexed-by field's resolved type
doesn't map to one of these two tags.
### Growth policy
- Initial: `cap=8`, slots calloc'd at locus birth via
`lotus_hashmap_init`.
- Growth: when `(len + 1) > 0.7 * cap`, double cap and rehash
every live entry through the normal `set` path (the probe
sequence changes with the new mask, so we don't copy raw
bytes between tables).
- Shrink: not implemented in v1.
- Cap is always a power of two so hash-to-index folds to
`& mask`.
### Deletion policy
Backward-shift deletion (no tombstones). After clearing the
target slot, the runtime walks forward through the cluster and
shifts any entry whose natural position is "before" the freed
slot in the probe sequence. The cluster boundary is the first
empty slot encountered. This keeps probe chains tight and lets
`find_slot` terminate correctly without a separate tombstone
marker.
### Failure shapes
- `lotus_hashmap_get` / `lotus_hashmap_remove` return 0 on
contract break (missing key). Codegen wraps this into the
`Ty::Fallible { success: S, payload: KeyError }` surface via
the same machinery `@form(vec)` uses for `IndexError`,
synthesizing the `KeyError { kind: "missing_key" }` payload
at the call site (shipped v1.x-FORM-4 PR5).
- `lotus_hashmap_set` OOM during the slot calloc / realloc
routes through the substrate-trap → closure-violation
channel per the two-channel rule.
## Form-ring-buffer runtime (v1.x-FORM-5)
`@form(ring_buffer, cap = N)` lowers a pool capacity slot to an
inline `lotus_ring_buffer_t` and synthesizes a fixed-capacity
FIFO surface (push / pop / len / is_full). The cap is baked in
at `lotus_ring_buffer_init` from the form annotation arg; the
backing buffer is malloc'd once at locus birth and never grows.
### C struct layout
```c
typedef struct {
size_t cap; // fixed at init; never changes
size_t head; // index of oldest element (next pop)
size_t len; // current element count, 0..=cap
size_t elem_size; // bytes per element
char *buf; // cap * elem_size bytes
} lotus_ring_buffer_t;
```
Codegen emits the matching LLVM inline struct on the locus's
pool slot; the slot's struct field IS the ring buffer (no
indirection). Element-size is `sizeof(T)` from the cell type's
LLVM `size_of`.
### Primitive functions
Defined in `crates/hale-codegen/runtime/lotus_arena.c`
(v1.x-FORM-5):
| Function | Behavior |
|----------|----------|
| `void lotus_ring_buffer_init(void *rb, size_t cap, size_t elem_size)` | `malloc(cap * elem_size)`; head=len=0 |
| `int lotus_ring_buffer_push(void *rb, const void *src)` | 1=pushed, 0=full; wraps modulo cap |
| `int lotus_ring_buffer_pop(void *rb, void *out)` | 1=popped, 0=empty; advances head |
| `int64_t lotus_ring_buffer_len(void *rb)` | Current element count |
| `int lotus_ring_buffer_is_full(void *rb)` | 1=full (len==cap), 0=not |
| `void lotus_ring_buffer_destroy(void *rb)` | `free(buf)` at locus dissolve |
### Failure shapes
- `push` returns 0 when full → codegen converts to Bool false at
the language surface (`fn push(x: T) -> Bool`).
- `pop` returns 0 when empty → codegen lazily allocates an
`EmptyError { kind: "empty" }` payload on the err path,
surfaced to the caller's `or` clause.
- OOM during init (cap × elem_size too large to malloc) leaves
`buf == NULL`; subsequent push/pop see a 0-cap buffer and
refuse / fail. Routing OOM through the closure-violation
channel is deferred to a future hardening pass — the v1
contract is "fixed cap; if init can't allocate, the buffer
is permanently empty."
## Native codegen defaults
What the compiler emits for a native `hale build`:
- **Host-CPU tuning + O3 by default.** Native builds tune to the
host CPU (`target-cpu`/`target-features` from the build
machine) and run LLVM's aggressive (O3) pipeline — both the
module passes and the backend codegen level. This unlocks
autovectorization across all generated code (e.g. AVX-512 on a
capable host). **Consequence:** a native binary is **not
portable across microarchitectures** — it may use instructions
absent on an older CPU.
- **`--target-cpu native | baseline`.** `native` (default) is the
host-tuned build above. `baseline` pins a portable
**`x86-64-v3`** target (AVX2 + BMI2 + FMA) for **distributed
artifacts** that must run on any modern x86-64 CPU. The
emitted module self-describes its subtarget via per-function
`target-cpu`/`target-features` attributes, so the choice is
carried into bitcode (it survives LTO).
- **`LOTUS_LTO=1` — opt-in full-LTO.** Read at *build time*.
Emits the Hale module as LLVM bitcode and compiles the lotus C
runtime TUs with `-flto`, so the final `clang -flto -O3
-fuse-ld=lld` link inlines the runtime hot paths (arena
bump-allocator, string helpers, shm_ring framing) **across the
TU boundary** into the Hale-generated callers — a boundary
that's otherwise opaque. Worth a few percent on
allocation/coordination-heavy code; neutral on
already-vectorized loops (the host tuning is preserved under
LTO via the function attributes above). **Off by default:** the
LTO link is ~3–4× slower and requires `lld` on PATH. Native,
non-sanitizer builds only; `wasm32` and sanitizer builds keep
the ordinary non-LTO link. The `-Wl,--wrap` malloc/syscall
shims (and the `LOTUS_ARENA_LOG_BIG_CHUNKS` /
`std::diag::syscall_count` features that ride them) are
preserved under LTO — `lld` resolves `--wrap` before LTO
codegen.
- **`wasm32` is unaffected** — it stays `generic`/O2 (the browser
bundle is size/compat-sensitive).
## Diagnostic + tuning env vars
A small set of env vars toggle runtime instrumentation and
glibc tuning hooks. All are opt-in; unset (the default) keeps
the runtime quiet.
| Env var | Effect |
|---|---|
| `LOTUS_ARENA_LOG_BIG_CHUNKS=` | Logs every arena chunk + libc allocator (malloc / realloc / calloc / mmap) >= `` bytes to stderr with size, monotonic seqno, and 8-frame backtrace. Use `1` (= 1 MiB) as a shortcut; any positive decimal byte count works (e.g. `4096`). Each event labeled by source: `arena_big_chunk`, `malloc_big`, `realloc_big`, `calloc_big`, `mmap_big`. Note: only fires on the fresh-malloc path; chunks recycled from the per-thread pool bypass this hook — use `LOTUS_ARENA_LOG_CHUNK_ATTACH` for the full picture. |
| `LOTUS_ARENA_LOG_CHUNK_ATTACH=` | Logs every chunk attachment to ANY arena — both fresh-malloc (`chunk_attach_malloc`) AND per-thread-pool-recycled (`chunk_attach_pool`) paths — when `cap >= N`. Use `1` for "log every chunk attachment". Each event additionally prints `arena= kind= label=`: `root` means the chunk attached to a top-level locus-lifetime arena (a leak class if it grows); `sub` means a subregion (method scratch / free-fn body) that will recycle to the pool on destroy. `label` walks the subregion→root chain and looks up the root in the residency registry — requires `LOTUS_ARENA_RESIDENCY=1` to populate the label map. Filter `kind=root label=` to isolate the actual arena-growers. Shares the `LOTUS_ARENA_LOG_BIG_MAX_EVENTS` cap with the big-chunks logger. |
| `LOTUS_ARENA_LOG_BIG_MAX_EVENTS=` | Caps the log at `` events per process. Default 200. Set to 0 for unlimited (useful when watching low-rate sub-MiB allocation patterns over a long window). |
| `LOTUS_CHUNK_POOL_STATS=1` | Dumps per-thread chunk-pool hit / miss / store / overflow counters to stderr at process exit. Diagnostic for "pool isn't recycling" symptoms — pairs hits vs misses, stores vs overflows. The atexit handler runs on the main thread; counters are `__thread` so the dump is that thread's view. |
| `LOTUS_GLIBC_ARENA_MAX=` | Calls `mallopt(M_ARENA_MAX, )` at startup. Caps glibc's per-thread malloc arena count. `1` forces a single arena (max contention, min virtual-address fragmentation); higher `` trades contention for parallelism. Useful belt-and-suspenders against the per-thread arena heap-segment proliferation glibc default tuning can produce on long-running daemons. Unset keeps glibc's default. |
| `LOTUS_BUS_PAYLOAD_ARENA_CAP=` | Overrides the lazy-global bus payload arena's byte cap (default 64 MiB). When the cap fires, `lotus_arena_alloc` returns NULL and the existing alloc-fail paths (`empty_global` / `alloc_failed` violation) surface degraded service rather than OOM-killing the process. |
| `LOTUS_ARENA_RESIDENCY=1` | Registers every top-level arena (locus `__arena`s, `g_bus_payload_arena`, the program-wide global) into a side-table at creation time with a 24-frame construction backtrace. `std::process::dump_arena_residency()` walks the live set and emits one line per arena to stderr — bytes / chunks / parent / label, sorted by bytes desc — with the construction backtrace. Subregions (method scratch) are skipped; they destroy at method exit and don't accumulate residency. Atexit also dumps, but post-dissolve fires after all loci tear down — useful only for the global arena's final state. Long-running daemons should call `dump_arena_residency` from a heartbeat / checkpoint tick so locus arenas are sampled while still alive. |
| `LOTUS_CHUNK_POOL_PREFILL=` | Per-thread chunk-pool pre-fill on first touch. Default 32 (= 2 MiB resident per scheduler thread). Set 0 to disable. Bumps the pool's steady-state floor so brief bursts don't drain to zero and miss into malloc; the trade-off is per-thread resident memory. |
| `LOTUS_TSAN=1` | Read at *build time* (by the codegen's `build_executable`, not at runtime). When set, the emitted clang command passes `-fsanitize=thread` for both the C runtime compile and the binary link, and skips the `-Wl,--wrap=malloc/realloc/calloc/mmap` shim surface (TSAN intercepts malloc itself; the wrap'd `LOTUS_ARENA_LOG_BIG_CHUNKS` diagnostic is silently no-op under TSAN). The resulting binary runs ~5-15× slower; use only for race-hunting workloads. The C runtime embeds an empty `__tsan_default_suppressions` hook at link time so no external suppression file is needed; all originally-flagged substrate races (bus queue drain, arena destroy, coop pool worker, env-var lazy-init) have been fixed and the suppression list is empty. Opt-in tests live behind `#[ignore]` and the env var (see `crates/hale-codegen/tests/form_hashmap_lockfree_tsan.rs`). |
| `LOTUS_LTO=1` | Read at *build time*. Opt-in full-LTO native build: the Hale module is emitted as bitcode and the lotus runtime TUs compile with `-flto`, so the `clang -flto -O3 -fuse-ld=lld` link inlines the runtime hot paths (arena / string / shm_ring) across the TU boundary into the Hale callers. A few percent on allocation/coordination-heavy code, neutral on vectorized loops (host tuning preserved via per-function `target-features`). Off by default — ~3–4× slower link, requires `lld`. Native non-sanitizer only; `--wrap` shims survive (lld resolves them before LTO codegen). See *Native codegen defaults* above. |
| `LOTUS_BUS_LOG_UNMATCHED=1` | Surfaces silent no-key-match drops in `lotus_bus_local_dispatch_keyed` (Phase 3 routing keys). When set, each publish that matches no `where key == ...` subscriber for the topic emits a single stderr line citing subject, key, and the per-topic subscriber counts (specific vs unkeyed). Off by default — the silent-drop is correct for `on_unmatched: swallow` topics in steady state, but during bring-up the lack of any signal is load-bearing on debug cycles. Implied by `LOTUS_BUS_LOG_DROP=1`. |
| `LOTUS_BUS_LOG_DESERIALIZE_DROP=1` | Surfaces silent drops in the udp:// reader thread when (a) no deserializer is registered for the inbound subject, or (b) the deserializer returns `<= 0` (size mismatch, bounded-read failure). Emits one stderr line per drop naming the subject, the payload size, and (when applicable) the deserializer's return value. Off by default; the silent-skip on cross-routed multicast noise is the correct steady-state behavior. Same env-gated pattern as `LOTUS_BUS_LOG_UNMATCHED` for keyed-dispatch misses. Implied by `LOTUS_BUS_LOG_DROP=1`. |
| `LOTUS_BUS_QUEUE_CAP=` | Caps the cooperative bus dispatch queue, each per-pinned-locus mailbox, and each cooperative pool's queue at `N` cells (default 8192; floor 64; rounded up to a power of two; read once). **v0.9.0 footprint change:** the pinned mailbox and cooperative-pool queues are now lock-free MPSC rings (Vyukov bounded ring + signal-only-when-parked wake), and a fixed-size lock-free ring **pre-allocates its cap up front** rather than growing to it — so each pinned subscriber mailbox and each cooperative pool now costs ~4.3 MB resident at the default cap (vs the prior grow-as-needed). With the typical handful of pinned loci / pools this is a few-to-low-tens of MB; **lower `LOTUS_BUS_QUEUE_CAP` for pinned-/pool-heavy programs** to shrink it (the rings honor it identically). When a producer hits the cap it *back-pressures* instead of growing without bound (GH #125) — every message is still delivered. The mechanism: a **single-threaded** producer on the cooperative queue **inline-drains** it to free space; a **cross-thread** producer to a full ring **blocks** (a fenced producers-waiting handshake) until the single consumer drains a slot; a handler self-publishing to its own full ring spills to a consumer-thread-local overflow list (it can't block on itself). The cross-*cooperative*-pool *shared* queue path (multiple drainers, no single consumer) is the remaining non-lock-free path — a follow-on. Lower the cap to tighten the bound / footprint; raise it to reduce drain bursts at the cost of resident memory. |
| `LOTUS_BUS_LOG_DROP=1` | Broad superset for diagnosing "publish appears to succeed but handler doesn't fire" symptoms. Implies `LOTUS_BUS_LOG_UNMATCHED` + `LOTUS_BUS_LOG_DESERIALIZE_DROP` AND covers additional silent-drop sites the narrower vars miss: `lotus_bus_dispatch`'s serialize-fn-returns-<=0 case, the local-fanout (`lotus_bus_dispatch_wire` + `lotus_bus_local_dispatch`) zero-matching-subscribers case, per-entry deserialize-returns-<=0 on the local-fanout path, and the no-post-target case (mailbox / coop_pool / global queue all NULL on a matched entry). Each line names the call site, subject, and relevant size / index info so a bus-heavy repro can identify exactly which silent-skip is firing. Reach for this first when investigating bus-drop friction; the narrower vars stay supported for their specific bring-up scenarios. |
Every top-level arena is created via `lotus_arena_create_labeled(name)` and carries an immutable human-readable label string. The codegen passes the locus name (e.g. `WsClient`, `__lib_metrics_metrics_MetricMap`); the program-wide global is labeled `lotus.arena.global`; `g_bus_payload_arena` labels itself. The label is the load-bearing identifier in the residency dump; backtraces resolve via `-rdynamic` for cases where the label alone isn't enough.
The arena-chunk pool and -wrap=malloc family ship in every
binary unconditionally; the env vars are zero-cost when unset
(one int read + one branch per allocation). The `-rdynamic`
link flag is similarly unconditional so backtrace symbols
resolve without addr2line.
## Runtime size budget
The runtime should be small enough that a hello-world program
binary is < 1 MB statically linked, and < 100 KB if dynamic
linking against libc. This is a target, not a guarantee.
The framework's discipline enables this: no GC, no metadata
overhead per allocation, region-based MM compiles to bump
allocators. Comparable to C in size, with ergonomics closer to
Erlang.
## Open questions for runtime
- **Async / await integration.** Reserved keywords, no v0
semantics. The lifecycle state machine + cooperative yield
points subsume most of what async is for; explicit
async/await may not be necessary.
- **FFI to existing languages.** Generic FFI in stdlib;
team-specific bindings (e.g. domain-specific typed messages)
live as third-party packages. Marshalling helpers in stdlib.
- **Hot-reload of code (not just perspectives).** Erlang
supports module-level hot reload. Lotus's perspective
hot-reload is more granular and addresses most of the use
case; full code hot-reload may not be needed.
- **Determinism mode for tests.** Discussed in `testing.md`;
runtime needs to support deterministic scheduling when
requested. The cooperative scheduler makes this easier than
M:N would have — single-scheduler test mode is fully
deterministic by construction.
================================================================================
## Forms
## source: spec/forms.md
================================================================================
# Forms
A **form** is a compiler-recognized annotation on a locus
declaration that picks an efficient lowering for the locus's
storage and synthesizes a standard method set. Forms are the
mechanism Hale uses in place of parametric collection types
(`Map`, `Vec`, etc.). See
`spec/decisions.md` for The Design's grounding (F.0
form-before-parameter, F.22 capacity).
This document specifies the form annotation system in general
(syntax, contract, verification) and the `@form(vec)` contract
in detail. Subsequent forms (`@form(hashmap)`,
`@form(ring_buffer)`, `@form(lru_cache)`) get their own sections
as they're committed.
## Annotation syntax
```
form_annotation = "@form" "(" form_name [ "," form_arg { "," form_arg } ] ")"
form_name = LOWER_IDENT
form_arg = IDENT "=" expression
```
A form annotation sits on the line above a `locus` declaration,
like the existing `@projection` annotation:
```hale
@form(vec)
locus ItemList {
capacity { heap items of T; }
}
```
- **`form_name`** — the form identifier. Lowercase, single word.
The v1 form library is fixed (see "v1 form library" below);
user-defined forms are deferred to a future release.
- **`form_arg`** — keyword arguments specific to the form. Used
for tuning knobs that don't change storage discipline (e.g.
`cap = 100` for `@form(lru_cache)` / `@form(ring_buffer)`).
- **One form per locus.** Composition (`@form(vec) @form(ordered)`)
is rejected in v1.
Form-specific configuration that *does* change storage
discipline goes on the capacity slot, not in the annotation
arguments — see "`indexed_by` and slot clauses" below.
## Form contract
Each form specifies three things the compiler verifies and
implements:
1. **Required capacity shape.** What slots the locus must
declare, of what kinds, holding what cell types. Verified at
typecheck.
2. **Synthesized method set.** Names, parameter types, return
types of methods the form provides. Injected at typecheck so
call sites resolve normally.
3. **Lowering strategy.** What C-runtime substrate the compiler
emits in place of the literal F.22 pool / heap lowering.
If the locus's shape doesn't match the form's required capacity,
the compiler emits a focused diagnostic and rejects the program.
Example:
```
error[FORM-SHAPE]: @form(vec) requires exactly one `heap` slot;
found `pool entries of CmdEntry` instead.
--> registry.hl:3:1
|
3 | @form(vec)
| ^^^^^^^^^^
4 | locus Registry { capacity { pool entries of CmdEntry; } }
| ----------------------------
| expected `heap items of T`
```
## Synthesized methods
The form *synthesizes* its standard method set. The user does
not declare them; call sites resolve as if they were declared.
```hale
@form(vec)
locus ItemList {
capacity { heap items of T; }
// push, get, set, pop, len, is_empty come from @form(vec).
}
fn main() {
let l = ItemListL_Int { };
l.push(42);
let head = l.get(0) or raise;
println(head); // 42
}
```
**The user CAN add additional methods** on top of the
synthesized standard set. Naming a user method that collides
with a synthesized method (e.g. user writes their own `push`)
is rejected at v1 — override is deferred to v2.
## `indexed_by` and slot clauses
Form configuration splits between *slot clauses* and
*annotation arguments*. The dividing line:
- **Slot clause** — if the configuration changes how cells are
laid out or accessed. A storage-discipline concern.
- **Annotation argument** — if the configuration is a policy /
tuning knob the form's runtime consults; the underlying
storage shape is the same regardless.
```hale
// Storage discipline — slot clause.
@form(hashmap)
locus CmdRegistry {
capacity { pool entries of CmdEntry indexed_by name; }
// ^^^^^^^^^^^^^^^
// slot clause
}
// Policy / tuning — annotation argument.
@form(lru_cache, cap = 100)
locus SessionCache {
capacity { pool sessions of SessionEntry indexed_by id; }
}
```
`indexed_by` is a slot clause because indexing IS a storage
commitment — it changes the pool's layout and access path.
## Default lowering (no form annotation)
A locus without `@form(...)` gets the **literal F.22 default
lowering**: pool slots become `lotus_pool_t*` chunked free-list;
heap slots become a `lotus_heap_t*` doubly-linked live list
(individual O(1) alloc/free, wholesale-freed at dissolve — the
contiguous doubling buffer is the `@form(vec)` specialization,
not the default). The user's
own methods run as written; no synthesis, no shape verification
beyond the normal capacity-slot machinery.
The form annotation is the user's opt-in to a specific efficient
lowering. Without it, you get the predictable F.22 default.
## Form-annotated loci as application-layer storage substrate
A `@form(...)` locus occupies a different position in The
Design's taxonomy than a user-declared locus, and the
distinction is load-bearing for the two-channel failure rule
(`spec/semantics.md` § "Fallible call semantics" § "Where
each channel lives"):
- **User-declared loci** are substrate-facing — they
participate in the locus tower's lifecycle (bus
subscriptions, modes, contract reads, lifecycle methods).
Their methods communicate failure structurally via
closure assertions + `on_failure` routing.
- **`@form(...)` loci** are application-layer storage
substrate — they realize a substrate-honest *container*
shape that application code uses to hold data. Their
synthesized methods (`@form(vec).get`, `@form(vec).pop`,
future `@form(...)` accessors) operate per-access and
may be declared `fallible(E)`, addressing failure at the
immediate caller's `or` clause.
This is why the synthesized `@form(vec)` `get` / `pop`
methods carry `fallible(IndexError)` while user-declared
locus methods cannot. The `@form(...)` annotation is the
declaration-site marker that "this locus is application-
layer storage substrate, not a substrate-structural
participant." The synthesized method surface gets the
application-layer failure channel; the underlying form-vec
locus still respects every other substrate invariant
(arena ownership, dissolve cascade, capacity slot
discipline).
## Perspectives and forms
> **Perspectives reflect on structure, not on lowering.**
The form annotation changes how the compiler lays out memory
and synthesizes methods. It does not change:
- The locus's name or place in the tower.
- The set of fields declared in `params`.
- The capacity slot declarations.
- The `closure` / `on_failure` / `bus` blocks.
Perspectives that reflect on a form-lowered locus see the
*structural* view: the capacity slots, the params, the method
signatures (synthesized or user-written, treated uniformly).
They do not see the underlying C struct layout.
## Performance commitment
The form machinery commits to three distinct perf shapes,
distinguished because they measure different things and have
different bands:
> **(a) Tight-loop primitive cost.** A form-lowered primitive
> (e.g. `@form(vec).push`) must run within **10% of a
> hand-written equivalent in idiomatic C** on a microbench that
> exercises the primitive in isolation.
> **(b) Amortized workload cost.** A form-lowered workload that
> mixes the form's primitives with real per-call work (the
> shape real apps exhibit) must run within **2× of an equivalent
> idiomatic C program**.
> **(c) Per-op fallible-method cost.** A form-lowered fallible
> primitive (e.g. `@form(vec).get` / `.pop` /
> `@form(hashmap).get` / `.remove`) measured in isolation pays
> the C-function-call boundary to the `lotus_*` primitive plus
> the fallible-ABI plumbing. **No 10% commitment at v1**;
> isolated-microbench numbers may show 10–50× behind C. The
> contract is that fallible primitives are correct, predictable,
> and competitive when amortized (the (b) band).
>
> **Update: the isolated gap is largely closed.**
> `@form(vec)` `.get` / `.set` / `.pop` / `.push` are now inlined
> directly at codegen — bounds-check + typed GEP + load/store,
> no `lotus_*` C-call boundary. `.get` indexed by a counted-loop
> variable (`for i in 0..v.len()`, the vec unmutated in the body)
> additionally drops the bounds check entirely — it's provably
> in-bounds — so the read vectorizes. The remaining cross-boundary
> calls (arena allocation, etc.) inline under opt-in `LOTUS_LTO=1`
> (see `runtime.md`). The "IR-level inlining or LTO" this band
> once deferred to is shipped.
These bands track the same underlying performance reality at
different observer-perspectives (per The Design's
form/parameter cut): (a) measures primitive layout correctness,
(b) measures whether the substrate amortizes well at scale, (c)
measures the codegen-pattern overhead per primitive call.
`@form(vec)` is the canonical benchmark target (see "Bench
protocol" under the `@form(vec)` section below).
**Current standing (2026-06-28, vs Go at matched iteration counts):**
| Bench | Hale vs Go | Band | Status |
|---|---|---|---|
| `form_vec_push` (500k push) | 4.83× | (a) | beats Go ✓ |
| `vec_amortized` (push + fold, 200k) | 3.75× | (b) | beats Go ✓ |
| `form_vec_get` (200k get) | 2.60× | (c) | beats Go ✓ |
| `fn_scratch_work` (1k calls w/ work) | 7.05× | (b) | beats Go ✓ |
`Hale vs Go` = Go time ÷ Hale time (> 1 → Hale faster). The earlier
(2026-05-13) snapshot put `form_vec_get` at 0.026× and `vec_amortized`
at 0.42×; those were a benchmark **iteration-count mismatch** — the
Hale variants ran 20–25× more work than the `.go`/`.js`/`.py`
siblings (see the bench repo's N-audit) — compounded by
pre-optimization codegen. At matched N, with the `.get`/`.set`/`.pop`/
`.push` inlines + counted-loop bounds-check elimination + native-CPU/O3
defaults, Hale leads Go on all four. The formal within-10%/2×-of-**C**
verification for bands (a)/(b) still awaits the C twins noted in the
bench harness.
If a form fails its applicable band, the lowering is redesigned
before shipping more forms. The point of the form machinery is
not to
be clever — it's to be roughly as fast as the C the user would
have written by hand, with all the locus tower's structural
benefits on top.
---
# `@form(vec)`
A contiguous, growable buffer of `T`. The Hale analogue of
`Vec` / `std::vector` / Go slices. First form committed
for v1; canonical benchmark target for the 10% perf gate.
## Required capacity shape
The locus MUST declare exactly one `heap` slot. Its cell type
becomes the vec's element type `T`.
```hale
@form(vec)
locus ItemList {
capacity { heap items of T; }
}
```
Rules verified at typecheck:
- Exactly one slot. Zero slots, more than one slot, or any
`pool` slot is rejected.
- The slot MUST be a `heap` slot. (`pool` is the unordered free-
list shape; `vec` is the contiguous shape — they're different
storage disciplines, so a `pool` declaration with `@form(vec)`
is a contradiction.)
- The slot name is user-chosen and is not part of the contract.
The compiler finds the form's heap slot by *position*, not by
name. Idiomatic spellings: `items`, `entries`, `bytes`, `xs`.
The cell type `T` may be:
- A primitive (`Int`, `Float`, `Bool`, `Decimal`, `Time`,
`Duration`, `String`, `Bytes`).
- A user-defined `type` (struct or enum).
- A generic parameter (`heap items of T` inside a generic locus
`ItemList`); monomorphization (m63) produces a concrete
`@form(vec)` instance per binding.
The cell type MAY NOT be a locus reference — vecs hold values,
not loci. If you want a vec of child loci, use the F.22 `pool`
projection-class machinery instead; that's the structural shape
for parent-owns-children.
## Synthesized methods
```
fn push(x: T) -> () # infallible
fn get(i: Int) -> T fallible(IndexError)
fn set(i: Int, x: T) -> () fallible(IndexError)
fn pop() -> T fallible(IndexError)
fn len() -> Int # infallible
fn is_empty() -> Bool # infallible
fn sort() -> () # infallible; T in {Int, Float, String}
fn sort_by(cmp: fn(T, T) -> Bool) -> () # infallible
fn sort_desc_by(cmp: fn(T, T) -> Bool) -> () # infallible
```
The fallible methods return the locus-defined `IndexError`
payload type:
```hale
type IndexError {
kind: String; # "out_of_bounds" or "empty"
index: Int; # the requested index (0 for empty-pop)
len: Int; # the vec's len at fail time
}
```
`IndexError` is defined in the synthesized form preamble; the
user does not declare it. The same type is shared across all
`@form(vec)` instantiations (it's a flat record, not parametric
over T).
### `push`
```
fn push(x: T) -> ()
```
Appends `x` after the last element. Amortized O(1). The
synthesized lowering grows the underlying buffer by doubling
when capacity is exhausted; the realloc cost amortizes across
N appends to O(N) total.
`push` is **infallible**. OOM during the doubling realloc is a
substrate-level concern: the C runtime traps malloc failure and
re-raises as a closure violation, not as a `fallible(...)`
return on `push`. From the language surface, `push` never
errors. (See `spec/runtime.md` for the OOM trap convention.)
### `get`
```
fn get(i: Int) -> T fallible(IndexError)
```
Returns the element at index `i` (0-based). If `i < 0` or
`i >= len()`, fails with `IndexError { kind: "out_of_bounds",
index: i, len: self.len() }`.
Idiomatic call sites:
```hale
let head = vec.get(0) or raise; # bubble on empty
let first = vec.get(0) or default_value; # substitute
let nth = vec.get(i) or handle_oob(err); # custom handler
```
### `set`
```
fn set(i: Int, x: T) -> () fallible(IndexError)
```
Overwrites the element at index `i` (0-based) with `x`. If
`i < 0` or `i >= len()`, fails with `IndexError { kind:
"out_of_bounds", index: i, len: self.len() }`. `set` does not
extend the vec — index must be inside the current length;
appending new elements uses `push`.
```hale
vec.set(0, new_first) or raise;
vec.set(i, x) or noop(err); # swallow OOB
```
### `pop`
```
fn pop() -> T fallible(IndexError)
```
Removes and returns the last element. If `len() == 0`, fails
with `IndexError { kind: "empty", index: 0, len: 0 }`.
`pop` does not free the underlying buffer — capacity does not
shrink. Buffer release happens at locus dissolution.
### `len` and `is_empty`
```
fn len() -> Int
fn is_empty() -> Bool
```
`len()` returns the number of elements currently in the vec.
`is_empty()` is sugar for `len() == 0`. Both are infallible and
O(1).
### `sort`
```
fn sort() -> ()
```
Sorts the vec in place in ascending order. The cell type T MUST
be one of `Int`, `Float`, or `String`; any other T (struct, enum,
bytes, etc.) is a typecheck error suggesting `sort_by(cmp)`.
String comparison is lexicographic on the underlying byte
sequence (i.e., the C `strcmp` ordering); Float comparison
treats `NaN` as equal-to-anything to keep the ordering total
(no panic on NaN-bearing inputs).
`sort` is infallible. The substrate uses C `qsort` under the
hood — average O(N log N), worst-case O(N²) on pathological
inputs.
### `sort_by` and `sort_desc_by`
```
fn sort_by(cmp: fn(T, T) -> Bool) -> ()
fn sort_desc_by(cmp: fn(T, T) -> Bool) -> ()
```
Sort the vec in place under a user-supplied strict-less-than
comparator. The comparator's semantics: `cmp(a, b) == true`
means "a should come before b in the result." This is the
strict-`<` shape — `cmp(a, a)` SHOULD return `false` for any
`a`; a reflexive `<=` produces an unstable ordering but does
not panic.
`sort_desc_by(cmp)` is equivalent to `sort_by(|a, b| cmp(b, a))`
with the arg order swapped under the hood, so the same user
predicate produces the reverse ordering. Provided as a
convenience for the common "descending under the same key
extractor" pattern.
Both methods are infallible from the language surface. If `cmp`
itself faults (e.g., raises via `or raise` on a fallible call
inside the comparator body), the fault propagates through the
sort and the vec is left in an unspecified but valid state
(every element still present, ordering partially applied).
```hale
fn by_x(a: Point, b: Point) -> Bool { return a.x < b.x; }
points.sort_by(by_x); # ascending by x
points.sort_desc_by(by_x); # descending by x — same cmp
```
The cell type T may be any sortable shape, including structs and
enums. The substrate uses `qsort_r` with a per-(cell-type,
direction) trampoline synthesized at codegen time; the cookie
threads the caller's arena pointer through so the comparator's
body can use stdlib calls that allocate.
## Lowering strategy
`@form(vec)` lowers the heap slot to a three-field C struct:
```c
typedef struct {
size_t cap; // allocated capacity (elements)
size_t len; // number of valid elements
T* buf; // contiguous element array
} lotus_vec__t;
```
- Initial capacity: `0` (no allocation at birth). First `push`
allocates the initial buffer.
- Initial buffer size on first push: `4` elements. Chosen as a
small constant that avoids the malloc-per-element shape
without over-allocating for short-lived vecs.
- Growth policy: double `cap` on overflow. New buffer is
malloc'd; old elements are `memcpy`'d; old buffer is freed.
- Shrink policy: none. Capacity is monotonic in v1. (A
`shrink_to_fit` method may be added later if a workload
surfaces the need.)
Element storage is by-value: a `@form(vec)` of `Int` is a
contiguous `int64_t[]`; a `@form(vec)` of `type Pair { x: Int;
y: Int; }` is a contiguous array of `{int64_t, int64_t}`
records. No per-element heap allocation, no per-element header.
For elements of pointer-shaped types (`String`, `Bytes`), the
vec stores the pointer by value; the pointed-to bytes live in
whatever arena they were allocated from. Dissolution of the vec
frees the buffer but does not free the pointed-to bytes — those
follow their owning arena's lifetime per the standard F.22
contract.
## Arena ownership
`@form(vec)` is **not** a separate arena-allocated structure.
The three-field `lotus_vec_*` struct lives inline in the locus's
struct layout, the same way the literal `heap items of T`
declaration would. The growable buffer (the `buf` field) is
malloc'd from the *system allocator*, not from the locus's
arena — this is the existing F.22 heap-slot contract, unchanged.
Dissolution: when the locus arena is destroyed, the vec's
`buf` is freed via the F.22 dissolve cascade (the synthesized
destructor emits `free(self.items.buf)` for each formed heap
slot).
## Interaction with the locus tower
A `@form(vec)` locus is a locus in every other respect. It can:
- Have `params { ... }` with defaults.
- Have `birth`, `run`, `drain`, `dissolve` lifecycle bodies.
- Declare `closure { ... }` invariants.
- Route failures via `on_failure(child, err)`.
- Participate in a `bus`.
- Be projected by `perspective` declarations.
These are orthogonal to the form annotation. The form
*replaces* the literal F.22 heap-slot lowering; it does not
replace any other locus mechanic.
## Bench protocol (FORM-3 gate)
`@form(vec)` is the canonical benchmark target. The three perf
bands above map to three benches that ship under
`micro/` in the sibling `hale-lang/bench` repo:
1. **Tight-loop primitive (band (a), 10% gate).**
`form_vec_push` — 1M `push` on a `@form(vec)` of `Int`,
compared against an equivalent hand-written C program using
`malloc` + doubling realloc and raw `int64_t[]` indexing.
Wall-clock and peak RSS. **Status 2026-05-13:** 1.00× ratio
vs Go (effectively at C parity); gate met after the
`lotus_arena_create_subregion` elision for non-allocating
fn bodies (`notes/form-perf-checkpoint.md` documents the
path).
2. **Amortized workload (band (b), 2× gate).**
`vec_amortized` — 200k push + 200k fold over the result,
timed in one region. **Status 2026-05-13:** 0.42× ratio vs
Go (2.4× behind) — outside the 2× band; investigation
pending.
3. **Per-op fallible (band (c), advisory).** `form_vec_get` —
200k indexed reads via `vec.get(j) or raise`. **Status
2026-05-13:** 0.026× ratio vs Go (~38× behind isolated).
Documented residual; advisory only at v1 — the gap is the
C-function-call boundary to `lotus_vec_get` plus the
fallible-ABI plumbing. Closing it requires either inlining
the primitive's logic in IR at codegen time, or LTO. Both
deferred until a workload measures the cost.
4. **App bench.** A representative app rewritten to use
`@form(vec)` where it currently does explicit F.22 pool
walks. Wall-clock and RSS compared before / after, with the
form-lowered version targeted to be no worse than the F.22
baseline. Not yet wired.
The microbench harness lives in the sibling `hale-lang/bench`
repo; its `run.sh` resolves the `hale` binary via
`$HALE_BIN` → `hale` on PATH → `../hale/target/release/hale`.
Bench sources are sibling `.hl` / `.go` / `.js` / `.py` files
under `micro/` and `app/`.
If a bench fails its applicable band, the lowering is
redesigned before further forms are added to the library.
`@form(hashmap)` shipped via v1.x-FORM-4 without a parallel
bench under this protocol (the band (a) win on
`form_vec_push` was held to license the form-machinery
extension); a `micro/form_hashmap_*` family in `hale-lang/bench`
is the natural follow-up if perf becomes load-bearing for
hashmap consumers.
## Anti-patterns
### Hand-rolling the contract on a form-annotated locus
```hale
// WRONG — @form(vec) synthesizes push; user declaration
// collides with the synthesized name.
@form(vec)
locus ItemList {
capacity { heap items of T; }
fn push(x: T) -> () { /* ... */ } // rejected
}
```
The compiler rejects this at typecheck with `error[FORM-COLLIDE]:
@form(vec) synthesizes `push`; user declaration shadows the
synthesized method (override is deferred to v2).`
### Ignoring the fallible return
```hale
// WRONG — `get` returns fallible(IndexError); the bare let
// binding drops the error.
let v = vec.get(i); // compile error: error not addressed
```
```hale
// RIGHT — address the error with one of the three motions.
let v = vec.get(i) or raise;
```
### Treating the form annotation as syntactic sugar
```hale
// WRONG — assumes @form(vec) is "just like" hand-writing the
// methods over a literal F.22 heap. The lowering is different;
// the storage layout is different; the perf characteristics
// are different.
```
The form is a *contract*, not sugar. It commits to a specific
lowering and performance shape. Code that depends on
implementation details of the literal F.22 heap-slot lowering
(e.g. memory addresses of individual cells across pushes) will
not behave the same under `@form(vec)`.
## Open questions
Spec-level questions not blocking the current `@form(vec)`
contract; will be answered as workloads surface demand.
1. **Iteration surface — SHIPPED 2026-07-02.** `for x in
v.items { ... }` iterates a `@form(vec)` (fully inline buf
walk: len + buf loaded once, one GEP + load per element, no
C calls — vectorizes for scalar cells) and `for e in
m.entries { ... }` iterates a `@form(hashmap)` (cluster-aware
slot-cursor walk via `lotus_hashmap_iter_next`: O(cap) for a
full walk, where the index-based `key_at`/`entry_at` rescan
from slot 0 per call — O(cap×len)). The loop variable is a
per-iteration COPY for hashmap entries and a REFERENCE to the
vec-owned cell for vec struct cells (scalars are copies by
value). Mutating the form inside the body is unsupported (a
grow rehashes/reallocs under the cursor). `break`/`continue`
work. Ring-buffer iteration is still deferred (wrap-aware
oldest-first walk).
2. **Bulk operations.** `extend(other: @form(vec))`, `clear()`,
`truncate(n: Int)`. Useful but not foundational. Add after
the core methods land.
---
# `@form(hashmap)`
A keyed associative store: each entry is a struct value `S` that
carries its own key as one of its fields. The Hale analogue of
`Map` / `std::unordered_map` / Go `map[K]V` — but
*intrusive*: the value type S carries the key inside it rather
than the map storing separate (K, V) pairs. Shipped as the second
form in v1, following `@form(vec)`.
## Required capacity shape
The locus MUST declare exactly one `pool` slot, with an
`indexed_by ` clause naming a field of the cell type to
serve as the key:
```hale
type CmdEntry {
name: String;
handler: Int;
}
@form(hashmap)
locus CmdRegistry {
capacity { pool entries of CmdEntry indexed_by name; }
}
```
Rules verified at typecheck:
- Exactly one slot. Zero slots, more than one slot, or a `heap`
slot is rejected.
- The slot MUST be `pool`. (Hashmap recycles entry cells as
inserts / removes flow — the `pool` discipline. `heap` is the
growable-contiguous shape covered by `@form(vec)`.)
- The slot MUST declare `indexed_by `. The named field
must exist on the cell type.
- The cell type MUST be a user-declared `type` struct.
Primitives, enums, type aliases, and qualified paths are
rejected — the substrate needs the cell's field layout to
GEP the key out at insert time, which only resolves cleanly
for struct cells.
- The cell type MAY NOT be a locus reference. Cells are data;
loci are managed entities. Storing an entity in a hashmap
means the synthesized `.get(key)` materializes a stranger
in the caller's scope — the same antipattern that
`spec/semantics.md § Locus method dispatch` rejects at the
user-declared layer. For keyed-children patterns, use the
canonical alternatives:
- **Parent-child**: declare `accept(c: ChildL)` on the
parent locus. If name-based lookup is needed, pair with
a parallel `@form(hashmap)` of cell type
`type Index { key: String; child_idx: Int; }`.
- **Bus topic**: publish commands keyed by name; the parent
subscribes and dispatches to the right child.
- **Delegation**: collapse the per-child operation onto the
parent (`parent.inc_named(name)`).
- The slot name is user-chosen and is not part of the contract.
The compiler finds the form's pool slot by *position*, not by
name. Idiomatic spellings: `entries`, `bindings`, `routes`.
- `as_parent_for` on the slot is rejected — `@form(hashmap)`
owns its slot's allocator, so the borrow mechanic from
v1.x-4b doesn't compose.
- `@form(hashmap, ...)` accepts one optional kwarg:
`sync = X` (F.32-1; see "Cross-pool sync disciplines"
below). All other kwargs are rejected.
## Cross-pool sync disciplines
By default, `@form(hashmap)` is **single-pool only** — the
runtime has no synchronization on the hashmap entry points
(`lotus_hashmap_set` / `_grow` / etc), and cross-pool calls
into a plain `@form(hashmap)` receiver are typecheck-rejected
(F.32-0). The opt-in path is the `sync = ` kwarg:
| Annotation | Discipline | Status |
|---|---|---|
| `@form(hashmap)` | single-pool only | shipped |
| `@form(hashmap, sync = serialized)` | per-map `pthread_mutex_t` (F.32-1α) | shipped |
| `@form(hashmap, sync = striped)` | cell-level CAS + per-map `pthread_rwlock_t` for grow + cache-padded cells (F.32-1β2-v2) | shipped |
| `@form(hashmap, sync = lockfree)` (optional `cap = N` initial-size hint) | cell-level CAS, no rwlock or mutex on the steady-state path (F.32-1γ-v1); + `remove` via tombstones (F.32-1γ-v2 session 1); + lazy grow with brief writer/reader stall during migration (F.32-1γ-v2 session 3) | shipped |
**Discipline picker by workload:**
- **Single-pool only** (no cross-pool calls): plain
`@form(hashmap)`. Densest layout, zero sync overhead;
cross-pool calls are typecheck-rejected.
- **Cross-pool, write-heavy, cap unknown / dynamic**:
`sync = serialized`. Per-map mutex; writers serialize but
the path is short. Beats striped on 2-core / cheap-payload.
- **Cross-pool, read-heavy or per-op work is expensive**:
`sync = striped`. Rwlock lets concurrent readers run in
parallel; cache-padded cells avoid false-sharing between
reader and writer cells. Slower than serialized on 2-core
/ cheap-payload writes (rwlock overhead > parallelism gain).
- **Cross-pool, write-heavy, cap known approximately**:
`sync = lockfree` (optional `cap = N`). Pure CAS on the
steady-state hot path — no kernel-mediated sync. Fastest of
the four on the `form_hashmap_false_sharing` bench (~1.3×
faster than α serialized at 2 cores). Trade-off: when the
load factor exceeds 0.6, a grow event briefly stalls all
lockfree ops (~ms for typical caps) while the migration
runs. Steady state outside of grow remains fully lockfree.
`cap = N` is an optional initial-size hint (omitting it
starts at `LOTUS_HASHMAP_INITIAL_CAP = 8` and grows on
demand); supply 2-4× the expected peak so grows are rare.
The `serialized` discipline wraps every public entry point in
a per-map mutex. Throughput is bounded by lock contention
(~5-10 M ops/s on a 4-writer workload); correctness is
trivial. The map's `lotus_hashmap_t` struct grows by 12 bytes
(int sync_mode + pointer mu); the pthread_mutex_t is
heap-allocated at init and destroyed at dissolve.
The `striped` discipline adds cache-line padding to the cell
stride (rounds up to `LOTUS_CACHE_LINE`, 64B default) and uses
a 3-state occupancy machine (EMPTY → CLAIMED → COMMITTED) with
`__atomic_compare_exchange_n` for slot claim. A
`pthread_rwlock_t` guards the grow path (set/get hold rdlock,
grow holds wrlock). On the 2-core / cheap-payload bench
striped measures ~1.87× slower than serialized — the rwlock
overhead per op (~150 ns) exceeds α's mutex+memcpy (~90 ns)
by more than the 2-core parallelism gain compensates.
Striped's win materializes on 4+ cores or with heavier per-op
work where the rwlock overhead amortizes.
The `lockfree` discipline (F.32-1γ) drops the rwlock
entirely. `cap = N` is an optional initial-size hint (was
required pre-γ-v2 session 3 before grow shipped; now grows
transparently when load factor crosses 0.6). Under γ-v1
`remove` was a no-op; under γ-v2 session 1
`remove` is supported via tombstones (4-state cell machine:
EMPTY → CLAIMED → COMMITTED → TOMBSTONE). Pure CAS on the
occupancy byte; no kernel-mediated synchronization on the
hot path. The `form_hashmap_false_sharing` bench measures
lockfree at ~1.30× faster than serialized and ~2.54× faster
than striped on the 2-pool concurrent-write workload. Cap
should be sized to 2-4× the peak expected entry count to keep
linear-probe latency bounded and avoid early grows; the
runtime rounds the user's `cap = N` up to the next power of 2
(needed for the `& mask` probe). Omitting `cap` starts at
`LOTUS_HASHMAP_INITIAL_CAP = 8` and grows on demand.
Tombstones in γ-v2 session 1 are *not* reclaimed in place —
the probe advances past them, but inserts always land in the
next EMPTY slot rather than reusing a TOMBSTONE. Session 3's
grow path is where tombstones get compacted out: when the
load factor (`live + tombstones / cap`) crosses 0.6, the
table is rebuilt at double size and the new table omits
tombstones entirely. Consistency on remove follows the
lockfree model: a reader that observed COMMITTED before a
concurrent CAS to TOMBSTONE returns the (now-stale) value —
"the key was present at the moment we read."
Grow under γ-v2 session 3 uses a simpler design than the full
NBHM cooperative-helper migration: one writer wins a
grow_phase CAS, spin-waits for in-flight ops to drain via a
writers_in_flight counter, then runs the migration single-
threaded (no SENTINEL state, no cooperative helping). All
concurrent set/get/remove ops yield-spin during the migration
window. The hot-path cost in steady state is one atomic
load + branch-not-taken — measurably cheaper than the
NBHM-style 5-state CAS-on-every-probe. The trade-off is
tail latency on the writer that triggers grow (bounded by
the migration's O(cap) walk, ~ms for caps up to ~100k) and
brief stalls on all concurrent ops during that window.
The OLD slots buffer is freed eagerly at the end of the
migration (γ-v2 session 4). The drain-wait already guarantees
no in-flight op holds a stale pointer to OLD when grow
completes, so the use-after-free risk that the handoff doc's
QSBR design was meant to solve doesn't exist in this single-
grower variant — QSBR epoch tracking would be redundant. RSS
post-warmup is bounded by the current table size plus the
brief migration-window peak (OLD + NEW both alive for the
~ms duration of `lf_migrate`).
Cross-pool method calls into a `@form(hashmap, sync = ...)`
receiver are accepted without diagnostic — the chosen
discipline carries the substrate's safety contract. Inside a
single pool, all three sync modes pay only their respective
uncontended-fastpath costs (~30 ns for serialized,
~10 ns for lockfree's CAS).
See `notes/f32-cache-aware-delivery-plan.md` § F.32-1 for the
per-discipline implementation strategy + trade-off analysis,
and `spec/types.md` § "Single-threaded-method invariant
(F.31)" for how cross-pool calls into form-bearing receivers
interact with the placement system.
The key type `K` is derived from the resolved type of the
indexed-by field. At v1, K must be `Int` or `String`. Other
field types parse and synthesize methods but reject at codegen
with a focused diagnostic (the runtime ABI's `key_type_tag`
only enumerates these two).
## Synthesized methods
```
fn get(key: K) -> S fallible(KeyError)
fn set(value: S) -> () # infallible
fn has(key: K) -> Bool # infallible
fn remove(key: K) -> () fallible(KeyError)
fn len() -> Int # infallible
fn is_empty() -> Bool # infallible
fn key_at(i: Int) -> K fallible(IndexError)
fn entry_at(i: Int) -> S fallible(IndexError)
fn bump(key: K) -> () # infallible; S must be {key + Int counter}
```
The fallible methods return the synthesized `KeyError` payload:
```hale
type KeyError {
kind: String; # "missing_key" — only kind at v1
}
```
`KeyError` is injected into the bundle scope by the form
machinery alongside `IndexError`. The same type is shared across
all `@form(hashmap)` instantiations.
The key is not carried on the error because K varies per
hashmap. Users who want key context construct it through the
substitute motion (`or fallback(err)`), where `err: KeyError`
is in scope and any of the call's local bindings — including
the key arg — are available.
### `set`
```
fn set(value: S) -> ()
```
Inserts or replaces. `set(v)` GEPs the indexed-by field from
`v` to derive the key, then writes the whole struct at the
hashed slot. If a previous entry shared the key, it is
overwritten (`set` is unconditional — no error on duplicate).
`set` is **infallible**. OOM during the doubling realloc is a
substrate-level concern routed through the closure-violation
channel, not a `fallible(...)` return. Same shape as
`@form(vec)`'s `push`.
### `get`
```
fn get(key: K) -> S fallible(KeyError)
```
Returns the entry whose indexed-by field equals `key`. If no
such entry exists, fails with `KeyError { kind: "missing_key" }`.
```hale
let entry = registry.get(name) or raise;
let entry = registry.get(name) or default;
let entry = registry.get(name) or fallback(err);
```
### `has`
```
fn has(key: K) -> Bool
```
`true` iff an entry with this key is present. Equivalent to
"`get(key)` would succeed" but cheaper — no value copy.
### `remove`
```
fn remove(key: K) -> () fallible(KeyError)
```
Removes the entry whose indexed-by field equals `key`. If no
such entry exists, fails with `KeyError { kind: "missing_key" }`.
Idiomatic call shape:
```hale
registry.remove(name) or raise; # bubble on missing
registry.remove(name) or ignore(err); # swallow via Unit-returning handler
```
Hale doesn't surface `()` as a literal expression at v1, so
swallowing the error requires a Unit-returning handler call (or
guarding with `has` first):
```hale
fn ignore(_e: KeyError) { }
// later:
registry.remove(name) or ignore(err);
```
`remove` does not shrink the underlying buffer; capacity does
not decrease. Buffer release happens at locus dissolution.
### `len` and `is_empty`
```
fn len() -> Int
fn is_empty() -> Bool
```
`len()` returns the entry count; `is_empty()` is sugar for
`len() == 0`. Both infallible, O(1).
### `key_at` and `entry_at`
```
fn key_at(i: Int) -> K fallible(IndexError)
fn entry_at(i: Int) -> S fallible(IndexError)
```
Hash-table-order iteration (added 2026-05-16). `key_at(i)`
returns the i-th present key; `entry_at(i)` returns the i-th
present full entry value. Order is hash-table order (deterministic
for a given table state, but insertion-sensitive — adding entries
between iterations may rearrange the sequence as the table
rehashes). For a populate-then-iterate pattern the snapshot
order is reproducible.
`IndexError` payload matches the `@form(vec).get` shape (`kind:
String`, `index: Int`, `len: Int`). `out_of_bounds` is the only
`kind` produced; `index < 0` or `index >= len()` triggers it.
Per-call cost is O(cap) — the substrate walks the slots array
counting occupied entries until reaching the i-th. A full sweep
is O(cap²). Fine at small/medium scale; agents iterating
100k+-entry tables should populate a parallel `@form(vec)`
during inserts instead.
Closes the wordfreq-corpus reinvention pattern where every
program maintained a parallel keys vec to dodge the lack of
iteration.
### `bump`
```
fn bump(key: K) -> ()
```
Increment-or-init the Int counter field of the entry keyed by
`key`. Collapses the canonical 6-line pattern:
```hale
if m.has(k) {
let prev = m.get(k) or raise;
m.set(Entry { key: k, count: prev.count + 1 });
} else {
m.set(Entry { key: k, count: 1 });
}
```
into:
```hale
m.bump(k);
```
**Cell-shape requirement.** The entry type `S` MUST have exactly
two fields: the `indexed_by` key field plus one `Int` field
(the counter). Any other shape (zero Int fields, two Int fields,
or extra non-Int fields) is a codegen error pointing at the
manual pattern. The Int field's name is detected at codegen —
"count", "n", "freq", "hits", etc. all work.
`bump` is infallible. Generalizing to arbitrary update fns
(`update_with(k, init, fn(S) -> S)`) is the natural follow-up
once a use case surfaces.
## Lowering strategy
`@form(hashmap)` lowers the pool slot to an inline six-field C
struct holding open-addressing hashtable state:
```c
typedef struct {
size_t cap; // power-of-two slot count
size_t len; // live entry count
size_t key_size; // sizeof(K), set at init
size_t value_size; // sizeof(S), set at init
int key_type_tag; // 0 = Int, 1 = String
char *slots; // cap * (1 + key_size + value_size) bytes
} lotus_hashmap_t;
```
Each slot is `1 + key_size + value_size` bytes laid out as
`[occupied: u8][key: K][value: S]`. `occupied = 0` means empty;
the runtime uses **backward-shift deletion** (no tombstones) so
probes terminate as soon as an empty slot is seen.
- **Initial cap:** 8 slots, allocated at locus birth via
`lotus_hashmap_init`. Power of two so hash → index folds to
a single `& mask`.
- **Growth policy:** double `cap` when `(len + 1) > 0.7 * cap`.
Rehash every live entry through the normal `set` path (the
probe sequence changes with the new mask).
- **Shrink policy:** none. Capacity is monotonic in v1.
- **Hash functions:** 64-bit Knuth multiplicative for Int keys
(`k * 0x9E3779B97F4A7C15`), FNV-1a over the bytes for String
keys.
- **Probing:** linear with `& mask`. Backward-shift deletion
walks the cluster forward, shifting any entry whose natural
position is "before" the freed slot. Cluster boundary is the
first empty slot.
### Key extraction at the codegen surface
At each `set(value: S)` call site, codegen GEPs the indexed-by
field offset on the value alloca to produce a pointer to the
key, then passes `(slot_ptr, key_ptr, value_ptr)` to
`lotus_hashmap_set`. The runtime memcpys `key_size` bytes from
`key_ptr` into the slot's key region and `value_size` bytes
from `value_ptr` into the value region.
At `get`, `has`, `remove` sites, codegen lowers the key arg
into an alloca matching `key_size` and passes its address.
## Arena ownership
`@form(hashmap)` is **not** a separate arena-allocated structure.
The `lotus_hashmap_t` struct lives inline in the locus's struct
layout, the same way the literal F.22 pool-slot declaration
would. The `slots` buffer is malloc'd from the *system
allocator*, not from the locus's arena — matching the existing
F.22 slot contract.
Dissolution: when the locus arena is destroyed, the hashmap's
`slots` buffer is freed via `lotus_hashmap_destroy` in the F.22
dissolve cascade.
For elements of pointer-shaped types (`String`, `Bytes`) in the
cell struct, the hashmap stores the pointer by value; the
pointed-to bytes live in whatever arena they were allocated
from. Hashmap dissolution frees the slots buffer but does not
free the pointed-to bytes — those follow their owning arena's
lifetime per the standard F.22 contract.
## Complexity
| Operation | Expected | Worst case |
|---|---|---|
| `set` (no resize) | O(1) | O(N) on probe cluster |
| `set` (with resize) | O(N) amortized over inserts | O(N) per resize |
| `get` / `has` | O(1) expected | O(N) on probe cluster |
| `remove` | O(1) expected | O(N) on shift |
| `len` / `is_empty` | O(1) | O(1) |
Load factor stays ≤ 0.7 by construction. Hash quality for Int
keys (Knuth multiplicative) handles dense sequences such as
consecutive IDs without all colliding on slot 0.
## Interaction with the locus tower
A `@form(hashmap)` locus is a locus in every other respect — it
can have `params`, lifecycle bodies (`birth` / `run` / `drain` /
`dissolve`), `closure` invariants, `on_failure` routing, bus
membership, and projection by `perspective` declarations. The
form annotation *replaces* the literal F.22 pool-slot lowering
and synthesizes the six methods; it does not replace any other
locus mechanic.
## Bench protocol (future FORM-N gate)
`@form(hashmap)` gets a bench family parallel to `@form(vec)`'s
three bands once `hale-lang/bench`'s `micro/form_hashmap_*`
is wired up:
1. **Tight-loop primitive (band (a)).** A `form_hashmap_set`
microbench — 200k `set` calls on a hashmap of struct cells,
compared against an equivalent hand-written C program using
`malloc` + open-addressing tables of the same shape.
Target: 10% of the C baseline. Expected to track
`form_vec_push`'s shape closely since the subregion-elision
work applies equally.
2. **Per-op fallible (band (c)).** A `form_hashmap_get`
microbench — 200k `get(k) or raise` calls. Advisory; the
same C-function-call-boundary residual `form_vec_get` shows
will apply here. Closing it is the same work item.
3. **App bench.** A representative app rewritten to use
`@form(hashmap)` where it currently does explicit registry
walks; before / after comparison.
Not gated on FORM-4 shipping (FORM-4 was held to the band (a)
`form_vec_push` win for license); ships as a separate
milestone after a hashmap consumer surfaces concrete demand.
## Anti-patterns
### Treating `set` as keyed insert
```hale
// WRONG — set takes the whole value, not (key, value).
registry.set("foo", entry); // type error: too many args
```
```hale
// RIGHT — value carries its key as a field; substrate extracts.
registry.set(CmdEntry { name: "foo", handler: 1 });
```
The intrusive shape means the type system catches this for you
(`set` is synthesized with the single-arg signature `set(value:
S) -> ()`), but the conceptual reflex from `HashMap`
shaped languages is worth flagging.
### Ignoring the fallible return
```hale
// WRONG — get and remove return fallible(KeyError).
let v = registry.get(name); // compile error: error not addressed
registry.remove(name); // compile error: error not addressed
```
```hale
// RIGHT — address the error via one of the three motions.
let v = registry.get(name) or raise;
registry.remove(name) or ();
```
### Mutating the indexed-by field after `set`
The intrusive shape means the key is the field. If user code
keeps a reference to the value and mutates the indexed-by
field, the hashmap's invariant breaks (the cell sits in the
slot keyed by its *old* key, but `get` now looks up by its
*new* key). The v1 surface doesn't expose stored cells by
reference, so this isn't reachable from user code today.
Future iteration APIs that surface entry references will need
to gate against indexed-by-field mutation.
> Forward-looking / deferred items for this area now live in the
> decision log — see [`decisions.md` § Deferred & future
> work](./decisions.md#deferred--future-work).
## Required capacity shape
The locus MUST declare exactly one `pool` slot. The cell type is
the element type `T`; the capacity comes from the annotation arg
`cap = N`.
```hale
@form(ring_buffer, cap = 64)
locus RecentCmds {
capacity { pool history of CmdEntry; }
}
```
Rules verified at typecheck:
- Exactly one slot. Zero slots, more than one slot, or a `heap`
slot is rejected. (`heap` is the growable-contiguous shape
covered by `@form(vec)`; ring buffer recycles fixed-capacity
cells, which is the `pool` discipline.)
- The slot MUST NOT declare `as_parent_for` or `indexed_by` —
those clauses belong to other forms.
- `@form(ring_buffer, cap = N)` requires `cap`, must be a
positive integer literal. v1 doesn't const-evaluate
expressions for form args.
- The cell type T may be a primitive, a user-defined `type`, or
a generic parameter. It MAY NOT be a locus reference — same
restriction as the other forms.
## Synthesized methods
```
fn push(x: T) -> Bool # false when full
fn pop() -> T fallible(EmptyError)
fn len() -> Int # infallible
fn is_full() -> Bool # infallible
```
The fallible `pop` returns the synthesized `EmptyError` payload:
```hale
type EmptyError {
kind: String; # "empty" — only kind at v1
}
```
`EmptyError` is injected alongside `IndexError` and `KeyError` by
the form machinery; user-declared `EmptyError` wins per the
existing idempotent-injection contract.
### `push`
```
fn push(x: T) -> Bool
```
Appends `x` after the last element. Returns `true` on success,
`false` when the buffer is at capacity. Callers decide
drop-vs-backpressure semantics by inspecting the result:
```hale
let accepted = recent.push(entry);
if !accepted {
// backpressure: surface to caller, or drop, or evict-oldest
// via pop()+push() in a separate path.
}
```
`push` is intentionally Bool-returning rather than
`fallible(FullError)`. The full-buffer state is a normal
operational condition (the caller chose a bounded capacity), not
a substrate failure — surfacing it as a Bool keeps the call
shape ergonomic and avoids forcing every caller through `or`.
This is the one place in the form library where infallible-but-
returning-a-status is the right idiom; vec's `push` is truly
infallible (OOM routes through the closure-violation channel),
and hashmap's `set` is unconditional (replace-on-collision). The
ring buffer's fixed cap makes "refused" a user-observable state.
### `pop`
```
fn pop() -> T fallible(EmptyError)
```
Removes and returns the oldest element (FIFO — the one inserted
earliest among those still present). Fails with
`EmptyError { kind: "empty" }` when the buffer is empty.
```hale
let cmd = recent.pop() or raise; # bubble on empty
let cmd = recent.pop() or default_cmd; # substitute
let cmd = recent.pop() or fallback(err);
```
### `len` and `is_full`
```
fn len() -> Int
fn is_full() -> Bool
```
`len()` is the current element count, in `0..=cap`. `is_full()`
is sugar for `len() == cap`. Both infallible, O(1).
There is intentionally no `is_empty()` synthesized method on
`@form(ring_buffer)` — `pop` is the natural empty-detection
surface (the fallible return signals empty directly to the
caller addressing the error). Adding `is_empty()` would create
two redundant ways to ask the same question; defer until a
workload demonstrates a real need.
## Lowering strategy
`@form(ring_buffer)` lowers the pool slot to an inline five-field
C struct holding head/tail and a pre-allocated backing buffer:
```c
typedef struct {
size_t cap; // fixed at init; never changes
size_t head; // index of oldest element (next pop)
size_t len; // current count, 0..=cap
size_t elem_size; // bytes per element
char *buf; // cap * elem_size bytes
} lotus_ring_buffer_t;
```
- **Birth.** `lotus_ring_buffer_init` mallocs `cap * elem_size`
bytes and pins them for the locus's lifetime. The init takes
`cap` and `elem_size` as args; `cap` flows in from the form
annotation, `elem_size` from the cell type's LLVM `size_of`.
- **Push.** `lotus_ring_buffer_push` checks `len == cap`; if so
returns 0. Otherwise computes the wrap index as
`(head + len) % cap`, memcpys `elem_size` bytes from the
caller-provided source into the slot, increments `len`,
returns 1.
- **Pop.** `lotus_ring_buffer_pop` checks `len == 0`; if so
returns 0. Otherwise memcpys from `buf + head * elem_size`
into the out-pointer, advances `head` modulo cap, decrements
`len`, returns 1.
- **No growth.** Once init runs, the backing buffer is fixed
size. Push at capacity refuses; the spec contract is "fixed
capacity" not "grows on demand."
- **Dissolution.** `lotus_ring_buffer_destroy` `free`s the
backing buffer at locus arena destroy.
## Arena ownership
Same as `@form(vec)` and `@form(hashmap)`: the
`lotus_ring_buffer_t` struct lives inline in the locus struct
layout; the backing `buf` is malloc'd from the *system
allocator*, not from the locus's arena. Dissolution frees
`buf` via the F.22 dissolve cascade.
For pointer-shaped element types (`String`, `Bytes`), the ring
buffer stores the pointer by value; the pointed-to bytes live
in their owning arena per the standard F.22 contract.
## Complexity
| Operation | Cost |
|---|---|
| `push` (not full) | O(1) |
| `push` (full → refuse) | O(1) |
| `pop` (not empty) | O(1) |
| `pop` (empty → fail) | O(1) |
| `len` / `is_full` | O(1) |
All operations are constant-time and allocation-free after
locus birth. No realloc, no rehash, no compaction.
## Interaction with the locus tower
A `@form(ring_buffer)` locus is a locus in every other respect.
Same orthogonality as the other forms — `params`, lifecycle
bodies, closure invariants, `on_failure` routing, bus
membership, perspective projection all compose unchanged.
## Anti-patterns
### Forgetting to address `pop`'s fallible
```hale
// WRONG — pop returns fallible(EmptyError); the bare let drops it.
let v = recent.pop(); // compile error: error not addressed
```
```hale
// RIGHT — address with one of the three motions.
let v = recent.pop() or raise;
```
### Treating `push` as fallible
```hale
// WRONG — push returns Bool, not fallible(FullError).
let v = recent.push(x) or raise; // type error: push isn't fallible
```
The full-buffer state surfaces as a Bool return, not a fallible
payload. Inspect the Bool directly.
### Resizing expectations
The ring buffer's cap is fixed at the annotation site. There is
no `grow` or `shrink_to_fit`. Apps that need a growable bounded
buffer should pick a generous cap up front, or use
`@form(vec)` if growth is the right semantic.
> Forward-looking / deferred items for this area now live in the
> decision log — see [`decisions.md` § Deferred & future
> work](./decisions.md#deferred--future-work).
## Required capacity shape
The locus MUST declare exactly one `pool` slot with an
`indexed_by ` clause over a user-declared struct cell —
the same key surface as `@form(hashmap)` — AND the annotation arg
`cap = N`. `lru_cache` is the one form that needs BOTH a key and a
cap.
```hale
type SessionEntry { id: Int; token: String; }
@form(lru_cache, cap = 1000)
locus SessionCache {
capacity { pool sessions of SessionEntry indexed_by id; }
}
```
Rules verified at typecheck:
- Exactly one slot, of kind `pool`. Zero slots, more than one
slot, or a `heap` slot is rejected.
- The slot MUST declare `indexed_by `; the named field
must exist on the cell struct and becomes the cache key type
`K` (Int or String at v1). The cell struct is the value type
`S`.
- `cap = N` is required and must be a positive integer literal.
v1 doesn't const-evaluate expressions for form args.
- The slot MUST NOT declare `as_parent_for` — form-lowered slots
own their own allocator.
- The cell type MAY NOT be a locus reference — same restriction
as the other forms.
## Synthesized methods
```
fn put(x: S) -> () # infallible; silent LRU evict
fn get(k: K) -> S fallible(KeyError) # lookup + recency touch
fn contains(k: K) -> Bool # membership, NO recency touch
fn len() -> Int # infallible; count <= cap
```
`get`'s miss payload is the synthesized `KeyError` (shared with
`@form(hashmap)`); no new payload type is introduced.
### `put`
```
fn put(x: S) -> ()
```
Insert or update by the `indexed_by` key extracted from `x`. If
the key is already present, its value is overwritten and its
recency is refreshed. If the key is new and the cache is at
capacity, the least-recently-used entry is silently evicted first.
`put` is **infallible** — over-cap is a normal, silent operation
(eviction), not a failure. This matches the "never flagged /
bounded" contract: there is no `FullError`, unlike
`@form(ring_buffer).push` which returns a Bool.
Both insert and update mark the touched entry as recently used.
### `get`
```
fn get(k: K) -> S fallible(KeyError)
```
Looks up the value for key `k`. On a hit it returns the value
**and** marks the entry as recently used (a recency touch) — this
is what makes the policy LRU rather than FIFO: a `get` on an entry
saves it from an eviction that a purely-oldest-inserted policy
would take. On a miss it raises `KeyError`, addressed at the
call site's `or` clause.
### `contains`
```
fn contains(k: K) -> Bool
```
Membership test. Returns `true` if `k` is present. Unlike `get`,
`contains` does **NOT** touch recency — a `contains` on an entry
leaves it exactly as recently-used as it was. This distinction is
observable: after `contains(k)`, the entry `k` remains eligible
for eviction if it was the LRU entry.
### `len`
```
fn len() -> Int
```
Current live entry count, always `<= cap`. Infallible.
## Lowering strategy
`@form(lru_cache)` lowers the pool slot to an inline header struct
plus a single heap-allocated open-addressed table, managed by the
`lotus_lru_*` C runtime:
```
struct lotus_lru_t {
size_t cap; // fixed live-entry cap (never grows)
size_t len; // current live entries (<= cap)
size_t key_size;
size_t value_size;
int key_type_tag; // Int / String key ABI (shared w/ hashmap)
uint64_t tick; // monotonic access counter
size_t table_cap; // power-of-two slot count (>= 2*cap)
char *slots; // table_cap * (occupied + tick + key + value)
}
```
It deliberately does **not** reuse the `lotus_hashmap_*` family:
that family auto-grows and has no recency notion, both of which
break the cap invariant. The table is sized to the next power of
two `>= 2*cap` (load factor `<= 0.5`, so a probe always meets an
empty terminator). Each slot carries an access tick; eviction
removes the occupied slot with the minimum tick (the LRU entry)
via backward-shift compaction (no tombstones), keeping the table
reusable under unbounded insert/evict churn. Ticks are globally
unique per access, so the LRU entry is unambiguous.
The backing table is pre-allocated at locus birth
(`lotus_lru_init`) and freed at dissolve (`lotus_lru_free`); the
inline header lives in the locus struct and dies with the arena —
the same inline-header / heap-buffer split as `@form(vec)`,
`@form(hashmap)`, and `@form(ring_buffer)`.
> Forward-looking / deferred items for this area now live in the
> decision log — see [`decisions.md` § Deferred & future
> work](./decisions.md#deferred--future-work).
================================================================================
## Lexical structure
## source: spec/tokens.md
================================================================================
# Lexical structure
This document specifies the lexical layer of Hale: the set of
tokens the lexer produces and feeds to the parser. The formal
grammar in `grammar.ebnf` is defined over these tokens.
## Source encoding
Source files are UTF-8 encoded. File extension: `.hl`.
Outside string literals and comments, only ASCII characters are
permitted. The framework's mathematical primitives are spelled
out as ASCII names; the renderer can produce Unicode for human
display, but the source is ASCII-only. This commitment serves the
agent-first authorship principle (no symbol-input friction) and
simplifies tooling.
## Whitespace
- Spaces, tabs, carriage returns, and newlines are whitespace.
- Whitespace separates tokens but is otherwise insignificant.
- No off-side rule (no Python-style indentation).
- Newlines are not statement terminators; semicolons are required.
## Comments
- Line comment: `// ...` to end of line.
- Block comment: `/* ... */`. Block comments do not nest in v0.
- Doc comment: `///` lines directly above a declaration attach to
it (decorator lines like `@hot` may sit between). Rendered by
`hale doc` into the seed's API reference (spec/testing.md).
Tooling recovers doc text positionally from the source — the
lexer itself skips all comments. `/** */` stays reserved for a
future block form; `hale doc` v1 reads `///` only.
## Identifiers
- Match: `[a-zA-Z_][a-zA-Z0-9_]*`.
- Case conventions are enforced by lints, not the lexer:
- `PascalCase` for type names, locus names, perspective names.
- `snake_case` for variables, fields, function names.
- `SCREAMING_SNAKE_CASE` for constants.
- Reserved words (see below) are not legal identifiers.
## Reserved words (keywords)
### Declaration keywords
```
locus perspective type const
fn import export module
interface
```
`interface` (F.20) declares a structural interface — a named set
of method signatures. Any locus whose method set is a superset
structurally satisfies the interface (no `impl I for L`
declaration). Phase A (typecheck) and Phase B (codegen vtable
dispatch) both shipped 2026-05-11; interface values are usable as
fn params, fn returns, locus param/field values, and
`@form(vec)` cell elements. Interface elements in tuples / fixed
arrays remain gated on the broader composite-construction
coercion design (same gap as tuple-of-`LocusRef` escape).
### Locus member keywords
```
params contract bus capacity
as_parent_for indexed_by
```
`capacity` introduces an F.22 `capacity { ... }` block carrying
zero or more `pool X of T;` / `heap Y of T;` slot declarations.
`as_parent_for` and `indexed_by` are **hard keywords** used only
in slot-clause position inside a capacity block: `pool P of T
as_parent_for ChildL;` (v1.x-4 — share the slot's allocator with
a child locus's same-named slot at accept time) and `pool P of T
indexed_by field;` (v1.x-FORM-4 — names the hashmap key field on
a `@form(hashmap)` cell type).
The slot-kind words `pool` and `heap` are **contextual idents** —
they lex as ordinary Idents and the parser recognizes them only
in slot-decl head position inside a capacity block. So
`fn pool_alloc(...)`, `let heap = ...`, and `type Heap { ... }`
all stay admissible outside capacity blocks. Same F.10-style
narrowing the closure-keyword family uses for `approx` / `within`.
### Lifecycle keywords
```
birth accept release run
drain dissolve on_failure
```
### Mode keywords
```
bulk harmonic resolution
```
`mode` is a **contextual keyword**: recognized only
at locus-member position (as the leading token of a `mode_decl`
production in `grammar.ebnf` § 11). Outside that position it
lexes as an ordinary Ident, so `cam.mode: Int` and similar
param/field uses are admissible — relevant for raylib-style
bindings where `mode` is a natural API name. Same F.10-style
narrowing as `bindings`, `birth_check`, `pool`, `heap`.
`bulk` / `harmonic` / `resolution` remain reserved (they sit in
member-name position via `try_keyword_as_name`).
### Projection-class keywords
```
projection rich chunked recognition
```
### Placement keywords (F.31)
```
placement cooperative pinned pool core cores
topology node l3 reserve replicas
```
`placement` introduces the `placement { }` block on `main
locus` (F.31); `topology` introduces the `topology { }` block
(topology Phase 1b). `cooperative` / `pinned` are
placement-spec keywords inside `placement { }`. `pool`, `core`,
and `cores` are **contextual idents** — recognized only as
kwarg names inside `cooperative(pool = X)` /
`pinned(core = N)` / `pinned(cores = A..B | A..=B | {a, b, c})`
placement specs (topology Phase 1a). Phase 1b adds
`pinned(node = N)` / `pinned(l3 = name)`, plus the `topology`,
`node`, `l3`, and `reserve` block keywords. Phase 1c adds
`replicas` (a second `pinned(...)` kwarg, e.g. `pinned(cores =
A..B, replicas = K)`). All are contextual idents — outside their
positions they lex as ordinary Idents
and can name fns / vars / fields (same F.10-style narrowing the
closure / mode keyword families use). Note L3-domain names go
through the identifier rule, so a **hard** keyword (e.g. `bulk`)
can't name a domain — use a plain identifier. `cores` range
bounds and set elements are integer literals; the range reuses
the expression tokens `..` (exclusive) / `..=` (inclusive).
The pre-F.31 `schedule` keyword is gone — the `: schedule
cooperative | pinned` per-locus annotation no longer exists.
Placement is a `main`-only deployment seam (`bindings`'s
sibling), not a per-locus annotation.
### Closure keywords
```
closure epoch persists_through resets_on
```
`approx` and `within` are **contextual keywords**, recognized
only inside a `closure { ... }` block body (and only in the
specific positions the closure-assertion grammar admits them).
They lex as ordinary Idents elsewhere, so `fn approx(...)` and
`let within = ...` are admissible outside closure bodies. Same
F.10-style narrowing the mode-keyword family uses
post-dot. Shipped 2026-05-11; resolves
`notes/hale-friction.md` 2026-05-10
`closure-keyword-shadows-helper-ident`.
`captures` and `inline` (v1.x-VIOLATE, F.27) are
**contextual keywords**. `captures` is recognized only as a
clause keyword inside `closure { ... }` body, in the
`captures: f1, f2 ... ;` production. `inline` is recognized
only as an `epoch_spec` variant (immediately after `epoch` in
a closure clause). Both lex as ordinary Idents elsewhere — so
`let captures = ...` and `fn inline(...)` stay admissible
outside closure bodies. Same F.10-style narrowing.
### Recovery primitives
```
restart restart_in_place quarantine reorganize
bubble
```
`violate` (v1.x-VIOLATE, F.27) is a **contextual keyword**:
recognized only as the leading token of a statement inside a
locus method body (the `violate_stmt` production in
`grammar.ebnf` § 14). Outside that position it lexes as an
ordinary Ident, so `let violate = ...` and `fn violate(...)`
stay admissible. The optional trailing `with` in
`violate NAME with EXPR;` is the previously-reserved keyword
`with` (no longer reserved-for-future-use); recognized only as
the separator before the payload expression in this production.
### Contract keywords
```
expose consume inferred
```
### Bus keywords
```
subscribe publish on of
```
### Perspective keywords
```
stable_when serialize_as serves reperspective
```
`serves` (Phase 2a) is a **contextual keyword** —
recognized only in a locus header's post-`:` list, as the
`serves P` conformance clause (`locus RouterV1 : serves Router`).
`reperspective` (Phase 2b) is a **contextual keyword**
recognized only as a statement head followed by `self`
(`reperspective self. as ;`). Outside those positions
both lex as ordinary Idents, so `fn serves(...)` /
`let reperspective = ...` stay admissible. Same F.10-style
narrowing the placement / closure keyword families use.
`perspective` (a **hard** declaration keyword) doubles as a type
constructor: `perspective(P)` in type position is a handle to the
contract `P`. The parser recognizes it by the `perspective`
token followed by `(`.
### Statement / expression keywords
```
let mut if else
match for in while
return break continue true
false nil tier self
```
Bindings are immutable by default. `let mut x = ...` declares a
mutable binding; reassignment via `x = ...` is permitted. Without
`mut`, reassignment is a compile-time error.
`self` is meaningful only inside a lifecycle block, mode block,
or closure block. It refers to the enclosing locus's own params
and contract-exposed state. Outside such a block, `self` is a
parse error.
### Predefined type names (NOT keywords)
```
Int Uint Float Decimal
String Bool Time Duration
Bytes BytesView StringView BytesMut
```
`BytesView` / `StringView` (F.30) are non-owning views over a
`BytesBuilder`'s buffer; `BytesMut` (#3) is a raw `{ptr, len}`
writable/readable window (a `Topic.write` ring slot or a
`MirrorRing` window). See `spec/types.md`.
PascalCase per the type-name convention. The lexer emits these
as `Ident` tokens; the parser recognizes them by name in **type
position only**. In expression / namespace position, these names
are unreserved — `time::sleep` is a regular path because `time`
(lowercase) is an ordinary identifier. This eliminates the
lexical collision between primitive type names and stdlib
namespace names that would otherwise occur.
Shadowing a predefined type name with a user-defined type
(`type Int = ...`) is permitted by the grammar but produces a
compiler warning.
### Reserved for future use (not yet legal)
```
trait impl async await
macro
```
(`with` is no longer in this list — v1.x-VIOLATE recognizes it
as a contextual keyword inside the `violate_stmt` production.
`where` is no longer in this list either — Form K
recognizes it as the suffix keyword on `binding_entry` carrying
operational constraints.)
`yield` is a real statement keyword (m26b) — explicit
cooperative yield point; lowers to a bus-queue drain in
codegen. Listed under cooperative-scheduler keywords.
`terminate` is a statement keyword — ends the
current locus's lifecycle from inside one of its own methods
(the locus analogue of `return`). Only valid inside a locus
method body. See spec/semantics.md § "terminate".
### Cooperative-scheduler keywords
```
yield
terminate
```
### Fallible / error-addressing keywords (v1.x-FORM-1)
```
fallible fail or raise
```
All four are **contextual keywords**, recognized only in the
positions described below. Outside those positions they lex as
ordinary `Ident` tokens, so existing code that uses
`let fail = ...`, `fn or_else(...)`, or `let raise = ...` stays
admissible.
- **`fallible`** — recognized only immediately after a function
declaration's return type and before its body, in the
`fallible_marker = "fallible" "(" type_expr ")"` production.
Marks the function as one whose call sites MUST address the
error. *Permitted at declaration sites: free fns + stdlib-
synthesized methods over `@form(...)` containers; rejected at
typecheck on user-declared locus methods per the two-channel
rule (see `spec/semantics.md` § "Fallible call semantics"
§ "Where each channel lives").*
- **`fail`** — recognized only as the leading token of a
statement inside a fallible-fn body. Exits via the error
path, attaching the expression as the typed payload.
Symmetric to `return`.
- **`or`** — recognized as a postfix on any expression of
fallible type (the `or_clause` production). Four RHS forms:
`or raise` (propagate one frame up the call stack),
`or discard` (swallow the error and substitute Unit —
rejected at typecheck on calls whose success type is non-Unit;
sugar for the old `or noop(err)`), `or fail ` (B3 / G6 —
diverge like `raise` but with a fresh payload of the enclosing
fallible fn's declared error type), `or `
(substitute), `or ` (hand off).
- **`raise`** — recognized only as the immediate RHS of `or`.
Diverges the expression by re-entering the fallible-return
shape of the enclosing `fallible(E)` fn, with the payload
carried into the enclosing fn's error sret slot. Past every
enclosing fallible frame — at the implicit main locus's
root boundary — the runtime panics via `lotus_root_panic`.
The value-error channel is orthogonal to the closure-
violation channel; `or raise` does **not** enter the
`bubble` / `on_failure` machinery by default. See
`spec/semantics.md` § "Fallible call semantics" for the
full two-channel semantics.
The implicit binding `err` is in scope on the RHS of an `or`
clause and resolves to the fallible call's payload value
(typed as the fn's `fallible(T)` marker). This is a typecheck
rule, not a lexer rule — `err` lexes as an ordinary identifier
everywhere; the typechecker just introduces it as a binding in
the `or` RHS scope.
Rule of thumb for use:
- A stdlib fn whose failure carries useful diagnostic context
returns `fallible(T)` with a named payload type T.
- A stdlib fn whose "failure" is a benign not-found case (and
whose successful value has a natural default like 0 or "")
uses the sentinel-with-discriminator idiom instead
(`parse_int` / `can_parse_int`). Not every stdlib fn is
fallible — the marker is reserved for true error paths.
## Operators
### Arithmetic
```
+ - * / %
```
### Comparison
```
== != < > <= >=
```
### Logical
```
&& || !
```
### Bitwise
```
& | ^ << >> ~
```
### Assignment
```
= += -= *= /= %= &= |= ^=
```
### Closure / approximation
```
~~ equivalent to `approx`; tests value approximate-equal
within a stated tolerance band. Used in closure tests.
```
### Bus send
```
<- Send a typed message on a declared bus subject:
`"subject" <- value;`. The left side names a subject
declared in the locus's `bus { publish ... }`; the
right side is the typed payload. Same Erlang-shape
as `Pid ! Msg`; one-direction (subscribe is
declarative, not an operator).
```
### Member access / call / index
```
. :: ( ) [ ]
```
### Type / generic
```
< > -> => : ::
```
(Note: `<` and `>` are overloaded between comparison and generic
arguments. The parser disambiguates contextually.)
### Punctuation
```
; , { }
```
### Annotation prefix (v1.x-FORM-1)
```
@
```
`@` introduces a decorator-shaped annotation that decorates the
declaration immediately following it. v1 recognizes one such
annotation: `@form(, ...)`, which sits above a
`locus` declaration and picks an efficient lowering (see
`spec/forms.md`).
Lexically, `@` is a single-character punctuation token. The
parser routes the following `form` ident as a contextual
keyword in this position only.
Reserved for future annotation surfaces; user-defined
annotations are not in v1.
### Reserved (no v0 meaning)
```
# $ ? ?? ?:
```
## Literals
### Integer literals
- Decimal: `0`, `42`, `1_000_000` (underscores permitted as digit
separators).
- Hexadecimal: `0xFF`, `0x1A_2B`.
- Octal: `0o755`.
- Binary: `0b1010_1010`.
- Optional type suffix: `42i32`, `0xFFu64`. Default: `int`.
### Float literals
- Decimal: `3.14`, `1.0e-3`, `2.5E+10`.
- Optional type suffix: `3.14f32`, `2.5f64`. Default: `float`.
### Decimal literals
- Suffix `d`: `1.50d`, `0.05d`. Used for the built-in `decimal`
type (fixed-precision, no float artifacts; same semantics as
the `shopspring/decimal` Go library).
### Time / duration literals
- Duration suffixes: `ns`, `us`, `ms`, `s`, `m`, `h`, `d`.
Examples: `100ms`, `5s`, `1h30m`. Compound forms permitted.
- Time literals: ISO-8601 between backticks: `` `2026-05-08T12:00:00Z` ``.
### String literals
- Double-quoted: `"hello"`. Standard escape sequences:
`\n`, `\t`, `\r`, `\\`, `\"`, `\'`, `\0`, and `\xNN` for
ASCII bytes 0x00..=0x7f (added 2026-05-16; useful for
separators like `"a\x01b"`). High bytes (`\x80`+) are
rejected with a message pointing at `std::bytes::*` — the
Rust String invariant would UTF-8-encode them as two bytes
and surprise the caller. No `\u{NNNN}` Unicode escape at
v1; agents needing non-ASCII drop to byte literals.
- Raw strings: `r"..."` — no escape processing.
- Multi-line strings: `"""..."""`.
- F-strings (v1.x-10): `f"hello {name}"` — interpolates Hale
expressions inside `{...}`, each rendered via the same
formatter `println` uses. `{{` and `}}` are literal braces.
Plain `"..."` strings keep `{` and `}` as ordinary characters
for back-compat (no breaking change). Interpolation accepts
any expression; types are converted with `to_string`.
### Boolean literals
- `true`, `false`.
### Nil literal
- `nil`. Represents the absent value of an option type. Distinct
from numeric zero or empty string.
### Bytes literals
- `b"..."` — byte-string literal. Same escape table as `STRING_LIT`
(`\n`, `\t`, `\r`, `\\`, `\"`, `\0`, `\xNN`), but the body is
raw bytes — **no UTF-8 promotion**, so `\xNN` accepts the full
`0x00..0xFF` range (unlike the string literal which clamps at
`\x7f`). Embedded NULs are preserved; the value's length is
carried alongside the byte buffer, so `len(b)` and
`std::bytes::at(b, i)` work over the entire literal. Codegen
lowers via `lotus_bytes_from_buf(arena, src, len)`, which
copies the source bytes into the caller's arena.
Use cases: binary protocol headers / framing, magic bytes,
embedded fixtures. Earlier code reached for
`std::bytes::from_string("...")`, which only worked for ASCII;
the `b"..."` form replaces that workaround end-to-end.
## Built-in identifiers (not keywords)
These identifiers have semantic meaning in the standard library
and are conventionally reserved, but are not parser-reserved
keywords:
```
B c sigma phi
k_max span_max
sum prod min max
length empty
print println
to_string len abs
Int Float
```
`Int(x)` (v1.x-11) is a built-in cast — explicit Float → Int
narrowing via `fptosi` (truncate toward zero). Int arg is the
identity; other types reject. There is no implicit Float → Int
conversion; the user must commit via this constructor-shaped
call. `to_string(x)`, `len(x)`, `abs(x)`, `min(a, b)`, `max(a, b)`
are similarly bare-name builtins.
`print` and `println` are built-in functions, always in scope
without an `import`. They write to stdout. `print` does not
emit a trailing newline; `println` does. They accept any number
of arguments of any displayable type and concatenate.
`publish` is a built-in function in scope inside any locus
that declares matching `bus { publish SUBJECT of type T; }`.
The compiler verifies the subject and type at each call site.
Out of scope in loci with no publish declaration.
The framework names `(B, c, σ, φ)` use ASCII spellings in source:
`B`, `c`, `sigma`, `phi`. The framework's `k_max` is `k_max` or
its named alias `span_max`.
## Tokenization rules
- Longest match: `==` is one token, not `=` followed by `=`.
- Keywords win over identifiers: `run` is the lifecycle keyword,
not an identifier.
- Numeric literals are recognized greedily up to the first
non-numeric character (after handling `0x`, `0o`, `0b`
prefixes, decimal point, exponent, and type suffix).
- Comments are stripped before parsing (except doc comments,
which are attached to the following declaration as metadata).
## Reserved character classes
The following characters are not part of any token in v0 and are
lexer errors if encountered outside a string or comment:
```
` (outside time literal) \ (vertical bar mid-token, not || or |=)
```
(Backticks are reserved for time literals only; bare backslash
outside a string literal is illegal.)
================================================================================
## Operator precedence and associativity
## source: spec/precedence.md
================================================================================
# Operator precedence and associativity
Operators are listed from **highest** precedence (binds tightest)
to **lowest** (binds loosest). Operators on the same row have the
same precedence; their associativity is given in the right column.
| Level | Operators | Associativity | Notes |
|-------|-----------|---------------|-------|
| 14 | `()` `[]` `.` `::` | left | Call, index, field access, path |
| 13 | unary `-` `!` `~` | right | Unary minus, logical not, bitwise not |
| 12 | `*` `/` `%` | left | Multiplicative |
| 11 | `+` `-` | left | Additive |
| 10 | `<<` `>>` | left | Bit shifts |
| 9 | `&` | left | Bitwise and |
| 8 | `^` | left | Bitwise xor |
| 7 | `\|` | left | Bitwise or |
| 6 | `<` `>` `<=` `>=` | non-assoc | Ordering comparison |
| 5 | `==` `!=` | non-assoc | Equality |
| 4 | `~~` | non-assoc | Approximate equality (closure context only) |
| 3 | `&&` | left | Logical and |
| 2 | `\|\|` | left | Logical or |
| 1 | `..` `..=` | non-assoc | Range (parsed everywhere; *use* restricted to `for x in lo..hi` at typecheck) |
| 1 | `or` | right | Fallible disposition (v1.x-FORM-1; contextual). RHS is `raise` / `discard` / `fail ` / another expression. |
| 0 | `=` `+=` `-=` `*=` `/=` `%=` `&=` `\|=` `^=` | right | Assignment |
| -1 | `<-` | non-assoc | Bus send (statement-shape only) |
## Notes
### Comparison and equality are non-associative
`a < b < c` is a parse error. Use `a < b && b < c`. This avoids
the chained-comparison surprises common in C and matches Rust's
choice.
### Approximate equality binds at level 4
The `~~` operator is permitted **only inside a `closure` block's
assertion clause**. Elsewhere it is a parse error. It is
non-associative; `a ~~ b ~~ c` is invalid.
A closure assertion has the form:
```
expression ~~ expression within expression
```
The `within` clause is part of the assertion syntax, not an
operator at this precedence level. See `grammar.ebnf` for the
production.
### Generic arguments shadow comparison
The lexer emits the same tokens for `<` `>` regardless of context.
The parser disambiguates: when a `<` follows an identifier in a
type-expression context (after `:`, `->`, in a generic-args
position), it begins a generic-args list. Otherwise it is a
comparison.
This is the same approach Rust takes (with the famous turbofish
disambiguator `::<>` reserved for ambiguous expression contexts).
v0 does not provide turbofish; we'll add it if needed.
### Member access vs. path
- `.` accesses a field or method of a runtime value:
`foo.bar`, `book.bid_side()`.
- `::` accesses a name through a path: module, type, perspective:
`messages::Book`, `Strategy::default()`.
### Fallible disposition (`or`) binds looser than everything except assignment
The `or` keyword is the fallible-disposition postfix (v1.x-
FORM-1). It is **contextual** — recognized only when it
appears as a postfix on an expression that the typechecker
has marked `Ty::Fallible`. Outside that position, `or` is an
ordinary identifier (so `let or = 5;` stays admissible).
It is **right-associative** so a chain reduces step by step:
```
a() or b() or raise
// parses as: a() or (b() or raise)
```
The RHS of `or` is one of:
- the contextual keyword `raise` (diverges by re-entering the
enclosing `fallible(E)` fn's error-return path — the value-error
channel, orthogonal to closure tests; past every fallible frame
it root-panics via `lotus_root_panic`), or
- `discard` (swallow the error, substitute Unit; rejected at
typecheck when the success type is non-Unit), or
- `fail ` (diverge like `raise`, but with a fresh payload
of the enclosing fallible fn's declared error type), or
- any expression of the success type (the substitute path;
`err` is implicitly bound to the payload in this scope).
### Bus send is a statement, not an expression
`<-` does not nest in expressions. It appears only at statement
position:
```
"subject" <- value;
```
The left side is a string-literal subject (or an identifier
bound to a publish handle in a future revision); the right side
is any expression. The construct produces no value — there is no
`x = ("s" <- v)`. The level −1 row in the table is bookkeeping:
`<-` binds looser than every expression operator, so the parser
recognizes the leading expression in full before deciding the
statement is a send.
### Recovery primitives are statements, not operators
`restart`, `quarantine`, `bubble`, `violate`, etc. are
statement-level keywords; they do not participate in the
precedence table. They appear in the form:
```
restart(child);
quarantine(child) for 30s;
bubble(err);
violate fatal_io; // F.27, v1.x-VIOLATE
violate fatal_io with detail; // optional payload
```
`violate` is divergent (same type-level shape as `fail` /
`bubble`); the typechecker treats it as `Never`. Its closure-
name argument is a bare identifier (not a parenthesized
argument list), and the optional `with ` trailer carries
a user-shaped payload onto the synthesized `ClosureViolation`.
### Lifecycle and locus member declarations are not expressions
`birth`, `accept`, `run`, `drain`, `dissolve`, `on_failure`,
`mode`, `closure`, `contract`, `params`, `bus` introduce
declaration-level constructs inside a locus body and never
appear in expression position.
## Examples
```
a + b * c // a + (b * c) -- level 12 binds tighter than 11
a == b && c == d // (a == b) && (c == d) -- level 5 over level 3
a < b == c // parse error: < is non-assoc with ==
-a.field // -(a.field) -- access binds tighter than unary minus
foo(x) // generic instantiation, not (foo < T) > (x)
```
## Range and reserved precedence levels
- `..` / `..=` (range) — level 1, non-assoc. These **parse in any
expression position**; v1 restricts their *use* to `for x in
lo..hi`, so a range elsewhere is a typecheck error, not a parse
error. `..` is exclusive, `..=` inclusive.
Future operators expected to land at specific levels:
- `??` (nil-coalesce) — level 2, right-assoc
`??` is a reserved token; using it in v0 is a parse error.
Note: `?` (try-propagation) was previously reserved at level
13 but has been **cut** from the roadmap. The fallible-
disposition operator `or` covers the same use case at the
expression level without re-introducing a parallel
upward-propagation mechanism at the value layer. See
`spec/design-rationale.md`.
================================================================================
## Projects
## source: spec/projects.md
================================================================================
# Projects
An Hale project is a directory tree of `.hl` source files plus
anything those files vendor or reference. This document covers
the on-disk shape: how the directory tree composes into one or
more compiled binaries, how files within a directory share
scope, and how a project reaches into vendored libraries.
Two language-level commitments drive the shape:
- **F.19 — per-directory seed model.** Every `.hl` file in one
directory compiles as one seed; all top-level decls share
scope.
- **F.25 — cross-seed imports.** A library is a directory (or
single file) of `.hl` source; the importer names a namespace
alias and the resolver finds the source by path.
The recursion is in the import graph, not the directory tree.
The file system is presentation — convenient grouping of bits
that compose into a logical structure at parse time. Two
projects with identical import graphs can ship totally
different on-disk layouts; the lotus shape lives in the code.
See `spec/decisions.md` F.19 and F.25 for the design
rationale.
## Project shapes
Three shapes are idiomatic at v1; pick the smallest that fits.
### Single-file script
For one-off programs and tiny utilities. One file, no
directory:
```
script.hl
```
Build: `hale build script.hl` → `./script` binary.
Imports work — `import "../shared/foo" as foo;` resolves
relative to `script.hl`'s directory — but most scripts have no
imports.
### Single-app project
For an app that's developed and shipped as a unit:
```
myapp/ # the project — one seed
├── main.hl # AppL declaration + fn main()
├── .hl # sibling concerns (F.19; same seed)
├── ...
├── lib/ # vendored sub-loci (F.25)
│ ├── moa/
│ └── /
├── README.md # what the app does
└── FRICTION.md # per-app friction log
```
Build: `hale build myapp/` → `myapp/myapp` binary (next to
source; directory's basename becomes the binary name).
The center of the project is one named locus declared in
`main.hl` (per the apps-are-loci rule in `spec/styleguide.md`).
Sibling `.hl` files decompose by concern; they share top-level
scope under F.19. Vendored libraries live under `lib/` (by
convention — see "Disk is presentation" below).
There is no `src/` wrapper, no project metadata file, no
build-output directory. The directory IS the package; the
binary lands next to the source.
### Workspace (multi-binary)
For multiple related apps that share vendored libraries:
```
project/ # workspace root
├── apps/
│ ├── fitter/main.hl # one app, one seed
│ └── applier/main.hl # another app, another seed
└── lib/ # workspace-level shared libs
├── moa/
└── shared/
```
Each app under `apps/` is its own seed; `hale build
apps/fitter` produces `apps/fitter/fitter`. Imports written
inside an app like `import "lib/shared" as shared;` resolve via
F.25's workspace-root fallback — the entry-relative search
misses (no `apps/fitter/lib/`), and the resolver walks upward
to find a `Cargo.toml` and tries `/lib/shared/`.
The workspace root is identified by walking upward from the
entry source looking for `Cargo.toml`. A future milestone may
add an `.hale-workspace` sentinel for non-Cargo trees; until
then, projects shipped outside this monorepo can still use
entry-relative imports but lose the workspace fallback.
## Seeds — the per-directory model (F.19)
**A directory of `.hl` files compiles as one seed.** Every
top-level decl (locus, type, free fn, perspective, const,
interface) declared in any file in the directory is visible to
every other file in the same directory, in one shared scope.
`hale build `, `hale run `, and `hale check
` accept directory targets and bundle every `.hl` file
under them. `hale build ` keeps working for
single-file targets (scripts, one-off cases).
**File order in the merged bundle is alphabetical by filename**
(deterministic). Resolution within the seed is **order-free** —
the typechecker flattens all top-level decls into one bundle
scope before name lookup, so a fn declared in `z.hl` is
callable from `a.hl` without ceremony.
**No per-file visibility.** There is no `pub`, no Go-style
uppercase-exported convention. Anything declared at the top
level of any file is visible to every other file in the seed.
Decompose by *concern* (one file per concern, helpers grouped
with their callers); don't try to encode visibility through
file boundaries.
**No subdirectories.** A subdirectory inside a seed is NOT
part of the seed — it's a separate seed. To reach into it,
import it as a library. This keeps "what's in scope at a given
file" answerable by reading the seed's directory; subdirs would
either silently inject decls (confusing) or require their own
import mechanism (which is what cross-seed imports are for).
## Cross-seed imports (F.25)
Cross-seed imports are how one seed reaches into another. The
imported seed's decls become available under a user-chosen
alias.
### Syntax
```
import "" as ;
```
- **``** is a string literal naming the library to import.
The path is resolved per "Resolution order" below.
- **``** is an identifier naming the namespace at the
import site. It is **required** — bare `import "";` is
a parse error. Cross-seed references in the importing seed
read as `::Name`.
The alias-required rule is the same forcing-function discipline
v1.x-3 enforces for `: projection recognition` (no default
sub-mode) and v1.x-FORM-2 enforces for the two-channel rule
(substrate-facing surfaces — lifecycle, mode, closure assertions,
bus handlers — can't declare `fallible(E)`; user-declared `fn`
members and free fns can): the user names the commitment at the
surface so a downstream reader doesn't have to reconstruct the
namespace from the path.
Imports appear at the top of a file, before any top-level
declaration. Multi-file seeds may declare imports in any file;
the build merges every file's imports into one set against the
seed's directory + workspace root.
### Examples
```hale
import "lib/finance" as fin;
import "../shared-helpers" as helpers;
fn main() {
let q = fin::Quote { symbol: "ABC", price: 10 };
let h = helpers::Formatter { };
}
```
### Resolution order
The compiler tries three locations in order; the first hit wins:
1. **`/.hl`** — single-file library.
2. **`//`** — directory bundle. Every `.hl`
file in the directory is one library seed (per F.19's
per-directory model). File order in the merged bundle is
alphabetical; resolution within the seed is order-free.
3. **`//`** — workspace fallback. The
workspace root is the first directory found by walking
upward from the importer that contains a `Cargo.toml`.
If none of the three locations resolve, the build fails with a
diagnostic listing all three search paths.
### Mangling scheme
Each imported library's top-level decls are rewritten with a
flat prefix so they never collide with the importer's symbols.
The mangled form is:
```
__lib___
```
- **``** is a stable, sanitized identifier for the lib
derived from its canonical path relative to the workspace
root (the nearest ancestor directory containing `hale.toml`
or `Cargo.toml`). Two consumers importing the same lib
produce the same `lib_id` regardless of which alias each
consumer chose. Non-identifier characters in the path collapse
to `_`; runs of underscores collapse to one.
- **``** is the basename of the source file the decl
lives in, sans `.hl`. So two files in the same library can
share a decl name without colliding.
- **``** is the original decl name as written in source.
Example: `/shared/messages/messages.hl` declaring `type
Order { ... }`, imported by app A as `msgs` and by app B as `m`,
both produce `__lib_shared_messages_messages_Order` in the
merged program. The shared identity is the natural shape for
DTO seeds exchanged on a bus — both apps see Order as
symbol-identical, and the wire bytes match by construction.
Mangling is recursive: every reference to an imported decl
inside the imported seed itself — bare names in fn bodies,
struct literals, type expressions, capacity-slot element
types, `as_parent_for` clauses, etc. — is rewritten through a
unified rename map built across the whole library. Locals (let
bindings, fn params, lifecycle params, for-loop vars, pattern
bindings, generic params) shadow top-level names per ordinary
lexical scope rules; the mangler tracks scope so a local named
`Greeter` inside an imported fn body does NOT rewrite.
The user never writes the mangled form. Their import-site
references go through a per-build path-rename table that maps
`::` → `__lib___`, analogous to
the static `STDLIB_PATH_RENAMES` and `MOA_PATH_RENAMES` tables.
The codegen's `Cx::mangled_for_path` method consults all three
tables in order: static stdlib, static moa, per-build imports.
The `` is the importer's local namespace choice (used
only at the call-site reference layer); the `` is the
lib's canonical identity.
Collision avoidance: two different libs live at different paths,
get different ``s, never collide regardless of what
aliases their importers picked.
`` fallback when no workspace root is in scope (e.g., a
one-off `hale build foo.hl` outside any toml-rooted repo):
the lib's directory base name (or file stem for single-file
imports), sanitized. Less collision-safe but the only stable
thing available; rare in practice.
The mangling shape mirrors the existing hand-spelled
`__StdLangMorpheme` / `__MoaBraidId` prefixes the bundled
stdlib and moa seeds carry; cross-seed imports extend the same
discipline automatically.
### Scoped imports (A4)
If library A imports library B, B's decls become reachable
**inside A's body only** under the alias A chose for B. A's own
importers (apps or other libraries) do NOT see B unless they
import it themselves. The mechanism: the resolver recurses into
each imported library's `import` directives with the library's
own directory as the new importer dir, so relative paths resolve
the way the library author wrote them.
This is the per-library-scoped-import shape called out as future
work in the v1 IMPORT milestone — A4 lifts the v1 strict barrier
to unblock the `pond/_util/*` retrofit (libraries that share
internal helper libs without forcing every consumer to vendor
them).
"Reachable inside A's body" is full reach, including **expression
and return position**: A may name a B type as a field/param/return
type, call B's free fns, *and* instantiate B's types and loci by
qualified literal — `b::Thing { ... }`, `b::SomeLocus { ... }` —
inside A's own fns, even when A is itself only reached two hops
down (`app → A → B`) and across A's multiple files. (This is the
"G34" shape; verified at HEAD, WS3.4 2026-06-11. The re-export
rule below still holds — the *app* cannot name `b::Thing` unless
it imports B itself.)
**No re-exports.** B's decls are not visible to A's importers
unless they declare their own dependency on B. The ``
in B's mangled prefix is derived from B's canonical path, NOT
from any importer's alias — so a util library reached through
two different importers gets ONE shared identity in the merged
program (path-deduplicated by canonical path in the visited
set). 2026-05-22 changed this from the original per-importer
mangling, removing the "DTO seed across two apps" sharp edge
where the same source produced different symbols.
**Cycles.** The CLI's canonical-path `visited` set bounds the
walk; a lib that imports itself or two libs that mutually import
each other resolve once each and stop.
### No `pub` / `export`
Every top-level decl in an imported seed is exported. There is
no visibility modifier in v1. The whole imported seed becomes
available under the alias.
Adding `pub` doubles the design surface (every decl picks a
visibility; users author the modifier; the typechecker enforces
it); v1 declines that complexity until a workload demonstrates
a real need for export control.
## Disk is presentation
The on-disk hierarchy is decoupled from logical identity. Three
layers to keep straight:
- **Library identity** = the alias the importer assigns
(`foo`) plus the decl names inside the lib (`Bar`,
`Greeter`). Stable across moves. References in user code —
`foo::Bar`, `let g = foo::Greeter { ... }` — never change
when a lib moves on disk.
- **Library location** = the string in the import line
(`"lib/moa"`, `"../vendor/moa"`). Per-importer, mutable.
Moving a lib costs N edits to import lines (one per
importer) but zero edits to actual code.
- **Library convention** = the fact that vendored libs
typically live under `lib/` rather than `vendor/` or
`petals/`. The `lib/` prefix is style, not semantics. The
resolver does not privilege `lib/`; it walks entry-relative
then workspace-root for whatever path the importer wrote.
The lotus shape — the recursion tower of a project — lives in
the **import graph**, not the file tree. Parse-time
resolution + merging + mangling is where the recursion
materializes. The filesystem just stores the bits in some
convenient grouping; two projects could have identical import
graphs and totally different disk shapes.
This is by design. Refactoring the on-disk layout (moving a
lib from `lib/` to `vendor/`, splitting a monolithic lib into
two, consolidating two libs into one) only needs to update
import lines — the bulk of the code that references those libs
stays unchanged.
## Workspace root caveat
Workspace-root detection walks upward looking for `Cargo.toml`.
For Hale programs living inside this Rust monorepo (apps/,
examples/, etc.) the walk hits the workspace's top-level
Cargo.toml and the fallback works as expected.
Standalone-shipped Hale binaries — sources not under a
Cargo.toml — won't have a workspace root to fall back on. They
can still use entry-relative imports (the single-file and
directory shapes above); only the workspace-fallback path is
unavailable. A future milestone may add an `.hale-workspace`
sentinel for non-Cargo trees.
## `hale run` interaction
`hale run` and `hale build` share the same codegen path, and as
of WS3.3 they also share the same *import* path: both
the single-file form and the directory form (`hale run ./dir`)
resolve `import "..." as ...;` directives, build the per-build
path-rename table, and rewrite qualified `alias::Name` references
identically. A directory `hale run` now produces the same
merged-and-resolved program as `hale build ./dir` — it execs it
instead of writing a binary.
(Previously the directory `hale run` form bundled the directory's
files *without* threading the path-rename table, so cross-seed
`alias::Name` references — and a topic decl referenced from a
sibling file — failed under `run` though they worked under
`build`. That gap is closed; the two commands no longer diverge on
imports.)
## Git-based dependency fetching (`hale fetch`)
A project may declare git dependencies in an `hale.toml`
manifest at the repo root; `hale fetch` clones each into
`vendor//` and pins resolved commit SHAs in
`hale.lock`. The cloned source is then picked up
automatically by the import-resolution order above (path 1 of
the resolver looks at `//`, which is
exactly where the fetcher places `vendor//`).
`vendor/` is toolchain-managed and distinct from `lib/`
(hand-maintained, never touched by the fetcher). Both paths
work identically through the import resolver but keeping them
physically separate prevents `hale fetch` from clobbering
hand-vendored source on a name collision.
See `spec/packages.md` for the full surface — manifest
format, lockfile shape, pin semantics, fetch command behavior,
and library-author conventions.
## What's NOT shipped (v1 boundaries)
Explicit non-features of the v1 project / import system. A
future milestone may relax some of them when concrete friction
demonstrates the need.
- **Deduplication is by canonical path, not content.** A library
reached through two different importers gets ONE shared identity
— its `` is derived from its canonical path and
deduplicated in the resolver's visited set (the 2026-05-22
change; see § "No re-exports" above), so the same source yields
the same symbols across consumers. What's *not* done is
content-level unification: two libraries vendored at different
paths are distinct symbols even if byte-identical.
- **No registry / version ranges / semver.** Dependency pins
are exact git refs. See `spec/packages.md` § "What's NOT in
v1" for the full list of package-management non-features.
- **No `pub` / `export` keywords.** Everything top-level in an
imported seed is exported.
- **No `src/` wrapper.** Source files live at the project root.
- **No build-output directory.** The binary lands next to
source.
## Implementation entry points
The project / import surface lives in three places:
- `crates/hale-cli/src/main.rs` — `find_workspace_root`,
`resolve_import`, `collect_target_files`, `resolve_imports`,
`parse_with_imports`, `collect_ap_files`. The CLI does file
resolution + mangling + merging; passes the resulting
per-build path-rename table to
`build_executable_with_imports`.
- `crates/hale-codegen/src/mangle.rs` — `mangle_program`,
`build_seed_renames`, `mangle_with_renames`. The AST walker
rewrites decl sites and use sites with a scope-aware
shadowing stack.
- `crates/hale-codegen/src/codegen.rs` —
`build_executable_with_imports`, `Cx::mangled_for_path`,
the `import_renames` field on `Cx`.
End-to-end coverage lives in
`crates/hale-codegen/tests/cross_seed_imports.rs` using
`tests/fixtures/lib-toy/` (two-file library) and
`tests/fixtures/import-toy-consumer/main.hl` (consumer with
`import "../lib-toy" as toy;`).
## Build flags + environment
| Surface | Effect |
|---|---|
| `hale build --dev` / `HALE_DEV=1` | Latency mode: LLVM O1 pipeline + Less machine codegen instead of the O3/`target-cpu=native` release default. For edit-build-run loops. |
| `hale check --json` | NDJSON diagnostics on stdout, one object per line (`file`/`line`/`col`/`severity`/`kind`/`message`) — editor/LSP consumption. `hale check` runs in ~10 ms on the largest apps. |
| `HALE_TIME=1` | Per-phase build wall times on stderr (front-end+codegen, llvm-passes, obj-emit, emit+link). |
| `--no-warn-unbounded-alloc` | Opts a run out of the default-on memory-bound survey (see verification.md). |
| `--target-cpu native\|baseline` | Backend CPU tuning (native = host, default; baseline = portable x86-64-v3). |
================================================================================
## Package management
## source: spec/packages.md
================================================================================
# Package management
Hale's v1 package-management surface is small and explicitly
git-based. A project declares its direct dependencies in
`hale.toml`; running `hale fetch` clones each one into
`vendor//` and records resolved commit SHAs in
`hale.lock`. `import "vendor/" as alias;` directives pick
the cloned source up via the standard import resolution order
(see `spec/projects.md`).
The fetched tree lives at `vendor/` — toolchain-managed,
distinct from `lib/` which stays for hand-maintained sources
the user vendors directly. Both paths work identically through
the import resolver (each is just an `//`
hit on path 1), but keeping them physically separate prevents
`hale fetch` from clobbering hand-maintained source on a name
collision.
The single design commitment driving the shape:
- **F.26 — vendoring is the v1 dependency primitive.** Hale
ships no registry, no version-solver, no transitive
resolution. Each project lists its direct dependencies and
the consumer vendors any transitive ones explicitly.
See `spec/decisions.md` F.26 for the rationale.
## Manifest (`hale.toml`)
The manifest lives at the project root — the same directory
that hosts the top-level `.hl` sources and (after `hale
fetch`) the `vendor/` directory. It is a TOML file with one
required section, `[deps]`:
```toml
[deps]
helpers = { git = "https://github.com/me/helpers", rev = "abc123" }
finance = { git = "https://github.com/me/finance", tag = "v0.1.0" }
ui = { git = "https://github.com/me/ui", branch = "main" }
```
Each entry's key is the local namespace the consumer will use
to import the dep. The value is a table with one required field
and up to one optional pin:
| Field | Required | Description |
|----------|----------|--------------------------------------------------------|
| `git` | yes | The clone URL. Any scheme git understands works. |
| `rev` | no | Pin to a specific commit SHA. |
| `tag` | no | Pin to a named tag. |
| `branch` | no | Track a branch (lockfile still pins the resolved SHA). |
Setting more than one of `rev` / `tag` / `branch` is a manifest
error — the spec must be unambiguous. Setting none uses the
remote's default branch.
There is no `[package]` table, no top-level metadata, no
authors / description / license fields. A project is identified
by its directory name and its source.
### Pin semantics
- `rev = ""` — the consumer pins to that exact commit.
Triggers a full (non-shallow) clone, because git's
`--depth 1 --branch [` form rejects raw SHAs. After
cloning, `git checkout ` lands the working tree.
- `tag = ""` — shallow clone of the tag's commit.
Idempotent re-fetches don't update; the lockfile pins the
SHA the tag pointed to when fetched.
- `branch = ""` — shallow clone of the branch tip
at fetch time. The lockfile still pins the SHA. To pick up
upstream changes on a tracked branch, delete the lockfile
(or just the lockfile entry for that dep) and re-run
`hale fetch`.
## Lockfile (`hale.lock`)
Auto-written by `hale fetch`. Pins every declared dep to a
resolved commit SHA so re-cloning is reproducible across
machines and across time:
```toml
[[dep]]
name = "helpers"
git = "https://github.com/me/helpers"
sha = "abc1234567890abcdef..."
[[dep]]
name = "finance"
git = "https://github.com/me/finance"
sha = "deadbeefcafef00d..."
```
The lockfile is intended to be committed alongside the
manifest. A consumer running `hale fetch` on a fresh checkout
re-clones every dep at the locked SHA, producing the same
`vendor/` contents the author worked with.
If a dep listed in the manifest has no entry in the lockfile
(new dep), `hale fetch` resolves it freshly and appends to
the lockfile. If a dep is removed from the manifest, its
lockfile entry is dropped on the next fetch (the lockfile is
re-emitted from the current manifest, not edited in place).
The `hale.lock` shape is owned by the toolchain — manual
edits will be overwritten on the next `hale fetch`. To
upgrade or downgrade a dep, edit the manifest and re-run
`hale fetch`.
## The `hale fetch` command
```
hale fetch [repo-root]
```
`repo-root` defaults to the current working directory. The
behavior, per dep:
1. **First fetch (no `vendor//`).** Clone the URL into
`vendor//`, checking out the requested ref (`--depth 1`
for tag / branch / default-branch; full clone + `git
checkout` for `rev`).
2. **Re-fetch, lockfile SHA matches current HEAD.** No-op —
no network call.
3. **Re-fetch, lockfile SHA differs from current HEAD.** Run
`git fetch --tags --prune origin`, then `git checkout` the
requested ref. Updates the lockfile with the new resolved
SHA.
4. **Re-fetch, no lockfile entry for the dep.** Same as case 1
from the consumer's perspective — the dep is new to this
project even if `vendor//` was somehow already present.
5. **Collision with a hand-maintained directory.** If
`vendor//` exists but has no `.git/`, `hale fetch`
errors and refuses to overwrite it. Move or delete the
directory and re-run. This guards against silently
clobbering sources the user vendored by hand (e.g. before
adding the dep to `hale.toml`).
After processing every dep, `hale fetch` writes a fresh
`hale.lock`. The write is whole-file (no in-place editing)
so a partial / failed fetch never leaves a corrupt lockfile.
Exit codes:
- `0` — every dep resolved cleanly.
- `1` — a manifest error, a lockfile parse error, a network
failure, or a `git` invocation returning non-zero.
## Resolution order interaction
The compiler's import resolver doesn't know about
`hale.toml` or `hale.lock`. It only knows that an `import
"vendor/" as alias;` (or `import "lib/" as alias;`,
or any other path) directive looks for source on disk at the
paths described in `spec/projects.md` § Resolution order.
`hale fetch` puts the source at `/vendor//`,
which is exactly where path-1 of the resolver looks first when
the import string is `"vendor/"`.
This separation is deliberate: the fetcher is a small,
optional tool that produces an on-disk tree. The compiler
treats that tree the same way it treats hand-vendored source
under `lib/`. A project that already vendors its libraries
(committed into `lib/`) can ignore the package manager
entirely; a project that uses `hale fetch` exclusively gets
the same compile behavior with less manual maintenance; and a
project that does both keeps the trees physically separate so
the toolchain never overwrites work it didn't put there.
## Library author conventions
A library is just a git repository whose root directory holds
one or more `.hl` files. From the consumer's perspective the
clone lands at `vendor//`, which becomes one Hale seed
(per F.19). What this implies for library authors:
- **Source goes at the repo root**, not under `src/`. Nested
directories are NOT crawled — they exist for the author's
organization only.
- **Include `hale.toml` in a library only if it has FFI
bindings.** As of 2026-05-22, a library that declares
`@ffi("c")` functions ships an `hale.toml` with an `[ffi]`
section listing `link = [...]` (C libraries the consumer's
link line should pull in) and `csrc = [...]` (C glue files
compiled alongside the runtime). The consumer's build reads
the section automatically at import time — no manual
`--link` / `--csrc` flags needed. See [`spec/ffi.md`](./ffi.md)
for the FFI contract and
[`agents/binding-packages.md`](../agents/binding-packages.md)
for the authoring brief. Pure-Hale libraries (no `@ffi`)
have no reason to ship `hale.toml`; their consumers
resolve them through the standard import lookup.
v1 still has no transitive `[deps]` resolution: a library
declaring its own `[deps]` won't have those auto-fetched by
the consumer's `hale fetch`. The consumer vendors every
dependency themselves. Document any indirect requirements in
your README.
- **Tag releases** if you want consumers to pin via
`tag = "vX.Y.Z"`. The toolchain doesn't enforce semver — the
tag is just a git ref — but consumers will read your tag
history when deciding what to pin to.
- **Keep top-level decl names short and namespace-friendly.**
Consumers will see your decls as `alias::Name`; the alias is
theirs to choose, so design decls that read fluently under
any short prefix. See `spec/styleguide.md` for naming rules.
## What's NOT in v1
Explicit non-features. A future milestone may relax some of
these when concrete friction demonstrates the need.
- **No registry.** All deps are git URLs.
- **No transitive deps.** A library's own dependencies are not
followed. The consumer must vendor them too.
- **No version ranges or semver.** Pins are exact strings (a
rev, tag, or branch name). There is no
highest-compatible-version solver.
- **No `hale publish`.** Distribution happens via `git push`
to a hosting service of the author's choice.
- **No checksum verification beyond git's own.** A pinned SHA
is the integrity guarantee — git's content-addressing makes
swapping content under a fixed SHA infeasible.
- **No build scripts or post-fetch hooks.** A library is
source only; no arbitrary code runs during `hale fetch`
beyond the `git` invocations the toolchain controls.
## Implementation entry points
- `crates/hale-cli/src/pkg.rs` — `Manifest`, `DepSpec`,
`Lockfile`, `LockedDep` types (serde) and the `fetch()`
entry point.
- `crates/hale-cli/src/main.rs` — the `hale fetch`
dispatch arm.
- `crates/hale-cli/tests/pkg_fetch.rs` — integration tests
that exercise a real `git clone` against a `file://` URL.
The compiler's import-resolution path (which consumes the
cloned source) lives in `hale-cli`'s `parse_with_imports` /
`resolve_import` / `find_workspace_root` family — see
`spec/projects.md` § Implementation entry points.
================================================================================
## Standard library
## source: spec/stdlib.md
================================================================================
# Standard library
Bundled with the toolchain, no separate install required. This
document describes the **current** stdlib surface. Milestone /
phase history lives in [`../CHANGELOG.md`](../CHANGELOG.md).
## Path resolution
`.hl` source references stdlib symbols by fully-qualified path:
```hale
let p = std::process::pid();
let contents = std::io::fs::read_file("config.toml");
std::io::tcp::Listener { host: "127.0.0.1", port: 8080 };
```
The parser tokenizes `::` as a path separator and the type checker
punts namespaced paths to `Ty::Unknown`; the codegen layer
resolves `std::*` paths against a hardcoded namespace dispatcher.
There is **no general module system** at v1 — no `use`
statements, no user-defined modules, no multi-file `.hl`
packages via the std-style mechanism. `std::*` is the only
recognized prefix. Adding a new stdlib function means: declare
its libc backer in `hale-codegen`'s `declare_builtins`, add a
match arm to `lower_stdlib_path_call_expr` (or its statement
sibling), and implement one `lower_std_*` method.
Cross-binary user code uses the F.25 cross-seed-imports mechanism
(`import "path/to/lib" as alias;`) — distinct from the `std::*`
magic path; see [`decisions.md` § F.25](./decisions.md).
## Design principles
- **Batteries included.** Go's approach: if a typical Hale
program needs it, it ships. A new Hale user shouldn't need
third-party packages for table-stakes coordinator-system work.
- **One canonical implementation.** Per Go's "one obvious way":
one `std::collections::Map`, not seven. Multiple options live
in third-party packages.
- **Framework-aware.** Stdlib types use the language's projection
classes, modes, and closure tests where appropriate. The
stdlib is itself disciplined.
- **Replaceable.** Anything in stdlib can be replaced by a
third-party module; nothing in stdlib is tied into the
compiler.
`std::*` is the curated path for ships-with-the-compiler bindings
(libc + OpenSSL only at the link floor). User-extensible C-ABI
bindings live outside this surface — see [`spec/ffi.md`](./ffi.md)
for `@ffi("c")` declarations, the mechanism by which library
authors land bindings to third-party C libraries (raylib, sqlite,
curl, ...) in community repos like pond. The stdlib's narrow link
surface is preserved exactly because user code can extend the FFI
surface without touching the compiler.
## Module surface
| Namespace | Surface | Source |
|---|---|---|
| `std::process` | `pid() -> Int`, `exit(code: Int)`, `rss_bytes() -> Int`, `dump_arena_residency() -> Int` (no-op unless `LOTUS_ARENA_RESIDENCY=1`; writes per-arena residency snapshot to stderr), `dump_pool_residency() -> Int` (F.35, 2026-05-28; writes one stderr line per cooperative pool — name, async_io vs blocking mode, parked-coro count, pending cell-queue depth — for ops embedding in heartbeat ticks), `run(argv) -> ProcessOutput fallible(IoError)`, `spawn` / `wait` / `kill` / `write_stdin` / `read_stdout` / `read_stderr` over `Child`. **`try_wait(c) -> Int fallible(IoError)`** (2026-07-17) — non-blocking reap via `waitpid(WNOHANG)`: `-2` = still running (the retryable sentinel shape `recv_into` uses; poll again), `0..255` = exited, `-1` = killed by a signal; ECHILD (already reaped) surfaces `kind="not_found"`. The supervisor idiom: a periodic tick polls `try_wait` per child without ever parking its pool — closes the styleguide §7 "daemons can't non-blocking-reap" gap. **`signal(c, sig) -> () fallible(IoError)`** (promoted from pond/subprocess) — arbitrary POSIX signal to the child's pid (15=TERM, 9=KILL, 1=HUP, …); the TERM→KILL escalation remains `kill`'s job; ESRCH surfaces `kind="not_found"` (usually benign post-exit — `or discard`). Both honor the manual-`Child` convention: `pid <= 0` answers exited-with-0 / no-ops. | path-call dispatch + C primitives |
| `std::env` | `args_count()`, `arg(i)`, `arg_or(i, default)`, `var(name)`, `var_exists(name)` | path-call dispatch + main-prelude argv stash |
| `std::cli` | `Resolver` locus — layered config resolution with precedence **CLI argv > env var > fallback**. Params: `env_prefix: String = "HALE_"` (each `get(key, …)` looks up `` in the process env — `prefix="HALE_"`, `key="dir"` → `HALE_DIR`) and `argv_keys: String = ""` (newline-separated positional keys — first line maps to `argv[1]`, second to `argv[2]`, …; a blank line doesn't shift positions; a key absent from `argv_keys` skips the CLI layer). Methods: `get(key, fallback) -> String` (the highest *populated* layer wins; empty/unset at a layer falls through) and `get_int(key, fallback) -> Int` (same precedence; a non-parseable value falls through to `fallback` rather than crash). No birth/run/dissolve lifecycle — the params *are* the configuration, so re-prefixing the Resolver retargets it without touching the body. | `runtime/stdlib/cli.hl` |
| `std::time` | `monotonic() -> Duration`, `monotonic_ns() -> Int`, `sleep(d: Duration)`, `now() -> Int`, `time_from_unix(n: Int) -> Time` | `clock_gettime` + EINTR-retrying `clock_nanosleep`; `sleep` slices the request into ≤100ms intervals and folds in a cooperative bus drain after each slice, so a long keep-alive sleep doesn't starve main-pool handlers (see `spec/runtime.md` § "`time::sleep` drain semantics"); `now()` is `CLOCK_REALTIME`; `time_from_unix` formats `gmtime_r` + `strftime` ISO 8601 UTC |
| `std::decimal` | `to_float(d: Decimal) -> Float` | Direct i128 → f64 conversion at scale 9 (`mantissa × 10^-9`) — skips an ASCII round-trip |
| `std::str` | `parse_int(s) -> Int fallible(ParseError)`, `parse_float(s) -> Float fallible(ParseError)`, `parse_decimal(s) -> Decimal fallible(ParseError)`; predicate siblings `can_parse_int` / `can_parse_float` (`can_parse_decimal` is NOT dispatched — listed here historically; implement or drop, tracked in notes/typecheck-m3.md); range-bounded variants `range_eq(json, start, end_exclusive, expected) -> Bool` / `range_parse_int(json, start, end_exclusive) -> Int fallible(ParseError)` / `range_parse_decimal(json, start, end_exclusive) -> Decimal fallible(ParseError)` (2026-05-26 — operate on byte ranges within an existing `String` without materializing a substring, paired with `std::json::iter_find_*_range` for allocation-free JSON walks); `byte_at_unchecked(s, i) -> Int` (2026-05-26 — direct byte access at offset i with NO bounds check; caller must guarantee 0 ≤ i < len(s); used by stdlib scan helpers (JSON walkers) where the bound is externally known and a per-access strlen / bytes_from_string would tank perf; misuse → UB); `index_of`, `lower` / `upper`, `trim`, `substring(s, lo, hi)`, `replace`, `repeat`, `pad_left` / `pad_right`, `from_bytes`, `clone(v) -> String` (deep-copy a `StringView` to an owned blob; identity on a `String` for generic callers); `builder_new` / `builder_append` / `builder_len` / `builder_finish` (String-builder primitives — for binary-safe accumulator use `std::bytes::BytesBuilder`) | `lotus_str_*` C runtime primitives |
| `std::bytes` | `at(b, i) -> Int fallible(IndexError)`, `slice(b, lo, hi) -> Bytes`, `from_string(s) -> Bytes`, `from_int(v) -> Bytes`, `concat(a, b) -> Bytes`, `clone(v) -> Bytes` (deep-copy a view to an owned blob). **Word-scan + masked-XOR** (2026-06-13, fast-protocol-I/O #4): `find_byte(b, off, needle) -> Int` returns the first index `>= off` whose byte equals `needle` (low 8 bits) or `-1` (non-fallible; `memchr` word-at-a-time scan — the length/delimiter-framing primitive for HTTP CRLF etc.). `read_*` / `at` / `find_byte` also accept a **`BytesMut`** raw `{ptr,len}` window (a `MirrorRing.readable()` window or a `Topic.write` slot) — read directly via their `_raw` siblings (length is the window length, no `[i64 len]` prefix), so a mirror-ring parse is zero-copy. `BytesBuilder.xor_mask(src: Bytes, key: Int)` (and its primitive `std::bytes::builder::__xor_mask_into(handle, src, key) -> Int`) appends `src` XOR'd with a repeating 4-byte key (`masked[i] = src[i] ^ key[i % 4]`, key bytes packed little-endian) in one reserve + word-at-a-time pass — the WebSocket masking primitive, replacing a per-byte `from_int` + `append` loop. **Binary-pack readers** (2026-06-06, shm-ring-interop Proposal A): `read_u8` / `read_u16_{le,be}` / `read_u32_{le,be}` / `read_u64_{le,be}` and the signed `read_i8` / `read_i16_{le,be}` / `read_i32_{le,be}` / `read_i64_{le,be}` (sign-extended), each `(b, off) -> Int fallible(IndexError)`; plus `read_f32_le` / `read_f64_{le,be}` `-> Float fallible(IndexError)`. Fixed-width scalar reads at a byte offset, bounds-checked (`[off, off+width)` past the buffer raises `IndexError { kind: "out_of_bounds", index: off, len }`, same error as `at`). Endianness is explicit (`_le` is the x86-native common case); a `u64` with the top bit set wraps to a negative `Int` (i64). **Binary-pack writers** (2026-06-08, shm-ring-interop A1): the mirror `write_u8` / `write_u16_{le,be}` / `write_u32_{le,be}` / `write_u64_{le,be}`, signed `write_i8` / `write_i16_{le,be}` / `write_i32_{le,be}` / `write_i64_{le,be}`, and `write_f32_le` / `write_f64_{le,be}`, each `(buf, off, val) -> Int fallible(IndexError)` — a fixed-width scalar write at a byte offset into a **`BytesMut`** raw window (a `Topic.write` slot or a `MirrorRing.writable()` window), bounds-checked identically to the readers (`[off, off+width)` past the window raises `IndexError`), returning the offset past the write. These back the `Topic.write(max) { … }` zero-copy ring producer and the `repr:`-tagged `Type::set_field`. Growing-buffer accumulator surface lives on the `BytesBuilder` locus — see [§ Builders vs Bytes](#builders-vs-bytes--the-recv-loop-pattern) | `lotus_bytes_*` C runtime primitives |
| `std::text` | `md_to_html(md) -> String`, `base64::encode` / `base64::decode` / `base64::url_encode` (RFC 4648 §5 URL-safe, unpadded — for JWT/JWS, OAuth, webhooks), `Sink` interface + `StdoutSink` / `StringSink` / `FileSink` loci, byte-class predicates (`is_alpha` / `is_digit` / `is_alnum` / `is_whitespace` / `is_word_char`), `tokenize_words_into(s, target_vec)` | `runtime/stdlib/text.hl` + C runtime |
| `std::io::fs` | `read_file(path) -> String`, `write_file(path, s) -> ()`, `write_file_append(path, s) -> Int` (returns bytes appended — asymmetric with write_file's Unit, kept for back-compat), `read_bytes -> Bytes`, `file_size -> Int`, `mkdir`, `rename`, `unlink`, `mktemp(dir, prefix) -> String`, `list_dir_count -> Int`, `list_dir_at(path, i) -> String` — all `fallible(IoError)`. (`list_dir` is listed in older notes but not dispatched — use list_dir_count + list_dir_at.) `file_exists(path) -> Bool` is a predicate (non-fallible). One-shot path-call surface: each call opens, does the op, closes. For held-open shapes use `std::io::file::File`. | `lotus_fs_*` C runtime primitives |
| `std::io::file` | `File` locus (held-open fd with auto-dissolve close). `open(path, mode) -> File fallible(IoError)`; `read_line(f) -> String` (returns "" at EOF or error — pair with `at_eof`); `at_eof(f) -> Bool`; `write_bytes(f, b)`, `write_line(f, s)`, `seek(f, offset)` all `fallible(IoError)`. Mode strings `"r"` / `"w"` / `"a"` / `"r+"` / `"w+"`. Returned Strings live in the bus payload arena. | `lotus_file_*` C primitives + `runtime/stdlib/file.hl` |
| `std::io::stdin` | `read_line() -> String`, `read_line_status() -> Int` (status `-1` = EOF/IO error; `0` = OK including empty-line). `read_byte(timeout_ms: Int) -> Int` — `poll` up to `timeout_ms` then a 1-byte `read`; returns `0..255` = the byte, `-1` = timeout, `-2` = EOF/error (sentinel; a timeout is a control outcome, not an error). `timeout_ms <= 0` is a pure poll. For interactive raw-mode input paired with `std::term::RawMode`. | POSIX `getline` / `poll`+`read` + payload-arena copy |
| `std::io::stdout` | `write_bytes(s: String) -> Int` — `fflush(stdout)` then a raw `write(1, ...)` that bypasses the prelude's `_IOLBF` line-buffering, so a multi-line frame isn't flushed per newline. Returns bytes written, `-1` on error (sentinel). The `fflush` keeps output ordered consistently with buffered `println`. | `lotus_term_write_stdout` C primitive |
| `std::term` | `is_tty(fd: Int) -> Bool` (POSIX `isatty` — probe whether an fd is a terminal, e.g. so a logger can decide on color without vendoring an FFI shim). `RawMode` — an RAII guard locus: `let r = std::term::RawMode { };` puts stdin in raw mode (no echo/canon/ISIG — unbuffered byte input, Ctrl-C as byte `0x03`) at birth and restores the saved termios at dissolve (scope exit). Soft-fails (runs unstyled) when stdin isn't a tty. `RawMode` also registers a runtime `atexit` termios restore, so a panic / unhandled error (which `exit()` through `atexit`) restores the terminal too — no FFI glue needed for terminal hygiene. `size() -> TermSize` (record `{ cols: Int; rows: Int }` via `ioctl(TIOCGWINSZ)`; `{ 0, 0 }` when stdout isn't a tty — poll per frame, no SIGWINCH handling). POSIX-only; non-tty / non-POSIX degrades soft. | `lotus_term_*` C primitives + `runtime/stdlib/term.hl` |
| `std::io::tcp` | `Listener` locus, `Stream` locus with `send` / `send_bytes` (Unit success) / `recv` / `recv_bytes` — all `fallible(IoError)` since #209 (EOF and recv-timeout return empty, only genuine errors fail; see § Stream methods below), `recv_into(fd, buf: Bytes, max_bytes) -> Int` (caller-provided builder destination). Path-calls `listen_socket`, `connect`, `accept_one` are `fallible(IoError)`. `connect` accepts dotted-quad hosts directly and falls back to hostname resolution via `getaddrinfo(AF_INET)`. **Exclusive bind (2026-07-15, downstream handoff item 4):** `listen_socket` sets `SO_REUSEADDR` (so a restart can rebind a port still lingering in `TIME_WAIT`) but deliberately **not** `SO_REUSEPORT` — so a second live process binding the same host:port fails with `EADDRINUSE` instead of silently sharing the port with the kernel round-robining connections between two divergent-state servers (a split-brain foot-gun). Intentional multi-process load balancing across a shared port is not the default; it would need an explicit opt-in if a workload ever wants it. Send/recv timeouts: `set_recv_timeout(fd, d: Duration) -> () fallible(IoError)` wraps `SO_RCVTIMEO`; `set_send_timeout(fd, d: Duration)` wraps `SO_SNDTIMEO`. `d = 0` disables (blocking default). After `set_recv_timeout(fd, 100ms)` a `recv_bytes` on a quiet socket returns ~100ms instead of blocking forever — unblocks recv loops that need periodic silence-detection / heartbeat / watchdog work. **Sub-millisecond timeouts aren't real:** `SO_RCVTIMEO` is rounded to the kernel's scheduling tick, so a `set_recv_timeout(fd, 50us)` parses but an idle recv still returns after ~1–1.5ms, not 50µs. Poll loops that need finer cadence must amortize their probes rather than lean on a sub-ms deadline (the async_io timed park inherits the same floor — its deadline comes from the same `SO_RCVTIMEO` value). Shares the `sock_set_timeout_ns` helper with the udp siblings (P4). **async_io parking (2026-07-14, downstream handoff):** on a `where async_io` pool, `recv_into` / `recv_stamped_into` / `recv_bytes` / `Stream.recv` park the coroutine on EPOLLIN until the fd is readable or the fd's `set_recv_timeout` deadline expires — `-2` (or `recv_bytes`'s empty return) means the *deadline expired*, never an instant would-block; with no timeout set they park indefinitely. The park deadline mirrors the socket's `SO_RCVTIMEO` (read back via `getsockopt` at call entry), so timeout semantics are identical on every pool type and liveness machinery built on the `-2` sentinel (pond/websocket ping-pong) composes unchanged. **`Stream.release_fd() -> Int`** (2026-07-19): flips `owns_fd` off (dissolve becomes a no-op) and returns the fd — the ownership hand-off primitive behind `std::http` takeover and any live-connection transfer to a longer-lived owner. Nagle control: `set_nodelay(fd, on: Bool) -> () fallible(IoError)` sets `TCP_NODELAY` on a connected fd — `on = true` disables Nagle so small writes hit the wire immediately instead of waiting up to ~40ms to coalesce, the first thing a latency-sensitive request/response or market-data socket reaches for. Previously impossible from Hale (`std::io::tcp` had no setsockopt surface); the generic udp `set_option_*` / `get_option_int` work on any fd if a less-common TCP option is needed. Kernel RX timestamps: `set_rx_timestamps(fd, on: Bool) -> () fallible(IoError)` enables `SO_TIMESTAMPNS` once at setup; `recv_stamped_into(fd, buf: Bytes, max_bytes) -> Int` is the timestamped sibling of `recv_into` (identical `>0` / `0` EOF / `-1` fatal / `-2` retryable contract) that issues one `recvmsg(2)` capturing the kernel's wire-arrival timestamp alongside the bytes — no extra syscall on the hot path. Read the stamps with `last_recv_kernel_ns() -> Int` / `last_recv_user_ns() -> Int` immediately after the call (errno-style thread-local, same idiom as `udp::last_source_*`). `last_recv_user_ns` is a `CLOCK_REALTIME` stamp taken at `recvmsg` return; `last_recv_kernel_ns` is the kernel's `SCM_TIMESTAMPNS` value, or **`0` when the platform/path delivered none** — notably loopback TCP and any NIC without RX software/hardware timestamping (it never returns garbage, so `>= 0` is the contract). The cmsg is parsed defensively (first control message only, length-validated, no `CMSG_NXTHDR` walk — some libcs loop forever on a zero-length cmsg). **Bus-routed I/O observability**: `Stream` gains a `log_subject: String = ""` param and a `bus { publish "io.tcp.**" of type std::io::tcp::LogEvent; }` declaration. When `log_subject` is set, every send / recv / close on that Stream publishes a `LogEvent { phase, detail, bytes, fd }` on the configured subject. Empty `log_subject` (the default) gates the publish with a single `len(s) > 0` branch — zero hot-path cost. Users wire any subscriber locus they want (`subscribe "io.tcp.**" as on_evt of type std::io::tcp::LogEvent`) — stderr sink, structured log, metrics, ring buffer; the bus is the indirection so no `Logger` interface or per-Stream sink locus is needed. Closes the "I/O lib is silent by default with no hook" friction. | `lotus_tcp_*` C primitives |
| `std::io::udp` | `bind(host, port) -> Int fallible(IoError)` (`host=""` → INADDR_ANY); `send(fd, host, port, msg)`, `recv(fd, max_bytes)`, `recv_into(fd, buf: Bytes, max_bytes)` (`>0` bytes / `0` empty datagram / `-1` fatal / `-2` retryable — a `set_recv_timeout` expiry or async_io park deadline; **2026-07-14 behavior change**: EAGAIN previously fell into `-1` fatal, and on `where async_io` pools `recv_into` now parks until readable or deadline like the tcp sibling), `close(fd)`. Multicast (2026-05-26 P1): `join_group(fd, group, iface) -> () fallible(IoError)` (iface=`""` → INADDR_ANY), `leave_group(fd, group, iface)`, `set_multicast_ttl(fd, ttl)` (0..255), `set_multicast_loop(fd, enabled: Bool)` (whether the sender receives its own packets), `set_multicast_iface(fd, addr)`. Transparent setsockopt pass-through (P2): `set_option_int(fd, level, name, value)`, `set_option_bool(fd, level, name, enabled)`, `get_option_int(fd, level, name) -> Int` — paired with `std::io::sockopt::()` named constants below for the `level` / `name` args. Source-bearing recv + timeouts (2026-05-26 P4): `recv_with_source(fd, max_bytes) -> Bytes fallible(IoError)` populates a thread-local source cache; `last_source_host() -> String` / `last_source_port() -> Int` read the cache from the most-recent recv_with_source on the current thread (errno-style; read immediately after recv). `set_recv_timeout(fd, d: Duration) -> () fallible(IoError)` / `set_send_timeout(fd, d: Duration)` wrap `SO_RCVTIMEO` / `SO_SNDTIMEO` (they take a struct timeval so can't ride `set_option_int`); `d = 0` disables (blocking default). Datagram boundaries preserved. **async_io parking (2026-07-15, downstream handoff item 3):** the Bytes-returning `recv` / `recv_with_source` (like `recv_into` before them) park the coroutine on EPOLLIN on a `where async_io` pool — bounded by the socket's `set_recv_timeout` deadline, or indefinitely when no timeout is set — instead of blocking the single pool worker inside `recvfrom`. This is what lets N reader loci, each parked on its own socket, share one async pool concurrently: previously the first blocking `recv` pinned the worker and every reader queued behind it on the same pool never started (with no timeout, never at all). Off async pools it stays a plain blocking `recvfrom`, and a park-deadline expiry surfaces the same "no datagram" result (NULL → the fallible error path) a blocking `SO_RCVTIMEO` timeout already produced. **`std::io::udp` is the raw-socket primitive, not a bus transport** — its `recv` / `send` calls don't carry the bus's typed-payload-dispatch contract. For UDP-as-bus see the `std::bus` row's `udp://host:port` substrate transport (shipped 2026-05-26): single URL scheme covers IPv4 unicast and multicast, dispatch goes through the same `LOTUS_BUS_CONFIG` route as `unix://`, lossy delivery (publisher-side "sendto returned" durability; subscribers best-effort). **`Reader` handle (2026-07-16) — the ergonomic event-driven ingest default.** `std::io::udp::Reader { addr, port, cap }` bundles a bound socket + a single reused `BytesBuilder` and exposes `next() -> BytesView fallible(IoError)`: it binds lazily on the first call (so a bind failure propagates through `next()`'s fallible channel), clears + refills the buffer in place per call (no per-datagram allocation), and returns a **zero-copy `BytesView`** aliasing the buffer — valid until the next `next()`. On a `where async_io` pool `next()` parks on EPOLLIN (kernel-woken, no busy-poll, no `SO_RCVTIMEO` quantum); on a classic pool it blocks the worker in `recvfrom`. It parks until a datagram arrives, so its only failures are a bind failure (first call) or a fatal recv error — both genuinely exceptional, so `or raise` is the disposition (`or discard` doesn't apply — `next()` yields a value). This is the hand-rolled "bind + BytesBuilder + `recv_into` + `.view()`" fast path baked into one handle so it's the path of least resistance; unlike the allocating `recv` it copies no per-datagram payload (the view aliases the buffer). `dissolve()` closes the socket; the nested `BytesBuilder` frees with it. | `lotus_udp_*` C primitives |
| `std::io::sockopt` | Named-constant getters returning the platform's numeric value for each setsockopt level / name. Use as the `level` / `name` args to `std::io::udp::set_option_*` / `get_option_int`. Each is a zero-arg fn (`std::io::sockopt::SO_RCVBUF()` etc.) so the value tracks the kernel headers; cross-platform without hardcoding. Shipped: `SOL_SOCKET`, `IPPROTO_IP`, `IPPROTO_IPV6`, `IPPROTO_TCP`, `IPPROTO_UDP`, `SO_REUSEADDR`, `SO_REUSEPORT`, `SO_RCVBUF`, `SO_SNDBUF`, `SO_RCVTIMEO`, `SO_SNDTIMEO`, `SO_BROADCAST`, `SO_KEEPALIVE`, `SO_LINGER`, `SO_PRIORITY`, `SO_BINDTODEVICE`, `IP_TTL`, `IP_TOS`, `IP_MULTICAST_TTL`, `IP_MULTICAST_LOOP`, `IP_MULTICAST_IF`, `IP_ADD_MEMBERSHIP`, `IP_DROP_MEMBERSHIP`, `IP_PKTINFO`. PMTU surface added 2026-05-27: `IP_MTU_DISCOVER` + `IP_PMTUDISC_DONT` / `IP_PMTUDISC_WANT` / `IP_PMTUDISC_DO` / `IP_PMTUDISC_PROBE` (Linux-only; returns -1 on platforms missing the constant — caller can detect and skip). TCP option added 2026-06-13: `TCP_NODELAY` (use with `IPPROTO_TCP` as `level`; or reach for the `std::io::tcp::set_nodelay` convenience). | `lotus_sockopt_` C getters |
| `std::io::tls` | Client-side TLS via system OpenSSL. `connect(host, port) -> Int fallible(IoError)` does the TCP connection + TLS 1.2+ handshake with SNI + system-trust-store cert verification. **Socket upgrade** (2026-07-14): `upgrade(fd: Int, host: String, verify: Bool) -> Int fallible(IoError)` wraps an *already-connected* TCP fd (from `std::io::tcp::connect`) in a client TLS session — the STARTTLS-style path a protocol takes after speaking a plaintext prologue on the socket (e.g. pond/pq sends the pgwire `SSLRequest`, reads the `'S'` byte, then upgrades the same fd). The returned handle is fully interchangeable with `connect`'s across `send_bytes` / `recv_bytes` / `recv_into` / `close` / `set_nodelay` / etc. SNI (`SSL_set_tlsext_host_name`) is **always** sent. `verify = true` authenticates the peer: hostname-checked against the cert via `SSL_set1_host` against the system trust store (identical to `connect`). `verify = false` gives **sslmode=require semantics** — encrypt *without* authenticating the peer (`SSL_VERIFY_NONE` per-connection override, no hostname check), for endpoints whose CA is not in the system trust store (e.g. AWS RDS); SNI is still sent. **FD-ownership asymmetry (important):** `upgrade` does **not** close `fd` on handshake failure — the caller already owned the fd (it came from `tcp::connect`, not from this call), so teardown stays with the caller (`std::io::tcp::close_fd(fd)` in the `or` handler). This is the *opposite* of `connect`, which dials its own fd and therefore closes it on any failure. On success the returned handle owns the fd (`close` closes it), same as `connect`. **Caveats:** pass a *hostname*, not an IP literal, for `host` when `verify = true` — `SSL_set1_host` treats the value as a DNS name for certificate hostname matching, and using an IP address as SNI is discouraged (some servers reject or mismatch it). Do **not** `upgrade` the same fd twice — each returned handle assumes sole ownership of the fd at `close` time, so a double-upgrade produces two handles that both believe they own the fd, leading to a double-close. **STARTTLS over-read:** before calling `upgrade`, read *exactly* the plaintext prologue and no further (pond/pq reads the single `'S'` byte and stops). Any bytes consumed past the prologue are gone from the kernel receive buffer, so the peer's TLS `ServerHello` — which follows immediately on the wire — would be handed to the caller as plaintext and the handshake would then read a truncated, corrupt stream (the classic STARTTLS plaintext-injection footgun). The current design is safe because `upgrade`'s recvmsg BIO and the `tcp::recv_*` primitives share **no** read buffer — every read pulls straight from the socket, so there is nowhere to accidentally buffer post-prologue bytes — but any future read-ahead/buffered `tcp` reader would break this property, so a consumer must keep its prologue reads unbuffered and byte-exact right up to the `upgrade` call. **`Int` is overloaded — a raw fd and a TLS handle are not interchangeable even though both are typed `Int`.** TLS handles are small table indices (0, 1, 2, …) that numerically alias real fd numbers, so the type system cannot catch a mix-up: passing a TLS *handle* to `upgrade` (which expects a raw fd) runs `SSL_connect` on whatever kernel fd happens to share that number (e.g. fd 0/1/2 = stdio), and passing a raw *fd* to `tls::send_bytes` / `recv_bytes` indexes the handle table with a value that isn't a handle — **both are silent misuse** (no error raised, just corrupt/garbage I/O), not a type error. Only ever pass `upgrade` an fd from `tcp::connect`, and only ever pass the `tls::*` send/recv/close primitives a handle returned by `tls::connect` / `tls::upgrade`. `connect` is now internally `dial + upgrade(fd, host, verify=1)`, so the `connect` behavior is unchanged and its network tests double as regression coverage for the shared upgrade path. `send_bytes` / `recv_bytes` / `recv_into` / `close` over the handshaked connection. Send/recv timeouts: `set_recv_timeout(handle, d: Duration) -> () fallible(IoError)` / `set_send_timeout(handle, d: Duration)` wrap `SO_RCVTIMEO` / `SO_SNDTIMEO` on the connection's underlying socket fd (the handle-aware siblings of the `std::io::tcp` ones, which take a raw fd). With a recv timeout set, `recv_into` returns the `-2` "timed out, retryable" sentinel (not `-1`/fatal) when `SSL_read` yields `SSL_ERROR_WANT_READ` — so a long-lived client can bound a blocking read and run connection-liveness work (ping/pong) instead of hanging forever on a half-open connection. On a `where async_io` pool, `recv_into` / `recv_stamped_into` park the coroutine on the raw fd (EPOLLIN, or EPOLLOUT on a renegotiation `WANT_WRITE`) until readable or the recv-timeout deadline — same 2026-07-14 parking semantics as `std::io::tcp`; `-2` means the deadline expired, never an instant would-block. **Fast-path siblings**: `set_nodelay(handle, on: Bool) -> () fallible(IoError)` (TCP_NODELAY on the underlying fd — reuses the tcp primitive) and `recv_stamped_into(handle, buf: Bytes, max) -> Int` with `last_recv_kernel_ns() -> Int` / `last_recv_user_ns() -> Int`, the TLS siblings of the `std::io::tcp` versions. `set_rx_timestamps(handle, on: Bool) -> () fallible(IoError)` enables `SO_TIMESTAMPNS`. The kernel timestamp rides the *socket* recvmsg but `SSL_read` sits in front of the socket, so the TLS connection uses a custom BIO whose read does `recvmsg` + the defensive `SCM_TIMESTAMPNS` cmsg walk, capturing the stamp on the socket fill that feeds `SSL_read` — `last_recv_kernel_ns` is the last segment's kernel RX stamp (0 when the path delivered none, e.g. no NIC RX timestamping), `last_recv_user_ns` is a `CLOCK_REALTIME` stamp at `SSL_read` return. (The recvmsg cmsg is the path that carries `SO_TIMESTAMPNS`; the `SIOCGSTAMPNS` ioctl reads `sk_stamp`, which that option does not populate.) Process-global `SSL_CTX` runs with `SSL_MODE_RELEASE_BUFFERS` — OpenSSL releases its read/write buffers between records so long-running TLS clients don't accumulate ~32 KiB per idle connection. **Allocation (audit, fast-protocol-I/O #6):** `recv_into` is zero-alloc on the Hale side — `SSL_read` decrypts straight into the caller's reserved buffer, no per-record malloc in the binding (the `tcp`/`udp` `recv_into` siblings likewise; pinned by `crates/hale-codegen/tests/recv_zero_alloc.rs` via the `std::diag` counter). The only per-record TLS allocation is OpenSSL-internal: with `SSL_MODE_RELEASE_BUFFERS` set, a released read buffer is re-malloc'd on the next record — a deliberate memory-vs-malloc tradeoff. An always-busy latency-critical connection that prefers zero per-record malloc over frugal idle memory would clear that mode (retain the buffers); that knob isn't exposed yet (no consumer). The `lotus_tls.c` TU compiles separately so helper tests linking `lotus_arena.c` directly don't drag in libssl/libcrypto. | `lotus_tls_*` in `runtime/lotus_tls.c` |
| `std::shm` | In-band record-header field delivery for a foreign ring (2026-06-13, shm-ring-interop). When a `layout:`-bound subscriber's `ring_layout` declares `record_header_bytes` with in-band header scalars (a per-record fixed header before the payload — e.g. a sequence number and a producer wire-arrival timestamp), `last_record_seq() -> Int` / `last_record_kernel_ns() -> Int` / `last_record_user_ns() -> Int` read the header fields of the record currently being delivered, called from inside the handler (errno-style thread-local, the same read-immediately idiom as `tcp::recv_stamped` — the value is the most-recent record's, valid for the duration of the handler). Each returns `0` when the bound layout declares no corresponding header field. The names map to the layout's declared header scalars by role; the layout's `recheck post_copy` guard ensures a torn header isn't surfaced. | `lotus_shm_*` in `runtime/lotus_shm_ring.c` |
| `std::diag` | Test-time gate counters. `heap_alloc_count() -> Int` returns the cumulative number of heap allocations (malloc / realloc / calloc / mmap) the runtime has made; `syscall_count(name: String) -> Int` returns the cumulative count of a wrapped I/O syscall (`"recv"`, `"recvmsg"`, `"read"`, `"write"`, `"send"`, `"sendto"`). Read a counter before and after a steady-state region and assert the delta — the runtime/test-time complement to compile-time `--warn-unbounded-alloc` ("this loop did zero heap allocs" / "exactly one read per poll"). Both return `-1` when the counting shim is absent (sanitizer builds — TSan/ASan interceptors collide with the `-Wl,--wrap` shim), so a caller can distinguish "gate unavailable in this build" from a real `0`. Counters are process-wide and monotonic; `syscall_count` returns `-1` for an unrecognized name. The wrap shim is compiled into every default (`-O2`) build at the cost of one relaxed-atomic increment per allocation / wrapped syscall — only the runtime's own calls are routed (libc- and libssl-internal I/O is untouched). | `lotus_diag_*` + `__wrap_*` in `runtime/lotus_arena.c` |
| `std::http` | `Request` + `Response` types (`Response.headers: String` carries CRLF-joined user-supplied headers — no trailing CRLF — for Set-Cookie / CORS / custom headers); `parse_request`, `write_response`; case-insensitive symmetric `header(receiver, name)` lookup; `Handler` interface (`fn handle(req: Request) -> Response`); `Server` locus with `shutdown()` (cross-thread safe — see [§ Server.shutdown](#servershutdown--interruptible-accept-loop)) and optional `ready_signal: String` for piped oracles. **Request reassembly** (2026-07-14, downstream handoff): the per-connection loop reads until the `\r\n\r\n` header terminator and then until `Content-Length` body bytes arrive, so clients that split headers and body across TCP segments (python urllib et al.) are served whole. Guards: 1 MiB total-request cap (a declared `Content-Length` over the cap answers `413`; overflow before a complete header block closes without a response) and a 5s recv timeout (bounds a stalled client on classic/pinned pools; inert on `async_io`, where a stalled conn parks only its coroutine). Keep-alive remains unsupported (`Connection: close` hardcoded); `Transfer-Encoding: chunked` is not parsed. **Connection takeover / Upgrade (2026-07-19):** `Request.conn_fd` carries the live connection fd into the handler (`-1` outside a Server), and `Response { takeover: true }` makes the Server write ONLY the status line + the response's `headers` + a blank line — no Content-Type/Content-Length/`Connection: close`, no body — and return WITHOUT closing the fd (`Stream.release_fd()` disarms the per-connection scope close). From that moment the handler owns the connection: stash `req.conn_fd` (typically publish it to a session locus on its own pool) and drive it through the raw-fd tcp surface or a borrowed `Stream { conn_fd: fd, owns_fd: false }`. Status-agnostic (101 for WebSocket-class upgrades — `__status_phrase` knows `101 Switching Protocols` — or a CONNECT tunnel's 200). Caveats: the conn loop's 5s recv timeout is still armed on the fd (clear via `set_recv_timeout(fd, 0)`), and a handler that sets `takeover` without stashing the fd leaks it — the accept/release daemon warn class. This is the surface WebSocket promotion was blocked on. **Router** (promoted from pond/router, 2026-07-17): `Router` locus — `add(method, pattern, h)` registers `METHOD /path/:capture` patterns against `RouteHandler` loci (`fn handle(ctx: Context) -> Response`; first match wins, register specific-before-general; method matching is case-insensitive at register time), `use(m)` registers `Middleware` (`before(ctx)` forward / `after(ctx, resp)` backward — onion order), `dispatch(req)` runs the chain, and `handle(req)` satisfies the `Handler` interface so `Server { handler: router }` plugs in directly. `Context` bundles the parsed `Request` (`ctx.req`, raw target incl. query string) with `RouteParams` (`ctx.params`); `path_param(params, name)` / `query_param(params, name)` return the capture / `k=v` value or `""` (sentinel shape; values NOT URL-decoded at v1). Patterns bound at 8 captures (`bounded[String; 8]` — exceeding it raises at register-authored routes); the 404 default is the overridable `not_found: RouteHandler` param. Trailing-slash tolerant on both sides; no implicit wildcard suffix. **Client** (promoted from pond/http/client, 2026-07-17): one-shot free fns `get(url)` / `post(url, body, content_type)` / `request(req)` — all `fallible(HttpError)`, `Connection: close`, read-to-close — plus the pooled `Client` locus (`user_agent` / `timeout_ms` / `max_retries` / `max_body` params; same fallible method surface; opt-in `keep_alive: true` switches to framed reads — Content-Length or `Transfer-Encoding: chunked` — over a 4-slot per-host:port connection pool with retry-and-backoff; the pool is deliberately hand-rolled, NOT `@form(lru_cache)`: an fd-owning cache needs an eviction hook and take-semantics the form doesn't offer). Client-side types are distinct from the server side on purpose: `ClientRequest` (`method` / `url: Url` / packed `headers` / `body: Bytes`) and `ClientResponse` (`status` / packed `headers` / `body: Bytes` — Bytes for binary safety, embedded NULs survive); `parse_url(s) -> Url fallible(HttpError)` decomposes scheme/host/port/path (query rides in `path`; no userinfo/fragments; no URL-decoding). `HttpError` kinds: bad_url / unsupported_scheme / connect_failed / send_failed / recv_failed / bad_response / too_large / retries_exhausted. https rides `std::io::tls` — **placement caveat**: TLS recv blocks the worker thread (no async_io park yet), so loci making https calls belong on `pinned` or a classic cooperative pool. Not implemented at v1: redirects (3xx returns as-is), proxies, compression. **Bus-routed observability**: `Server` gains a `log_subject: String = ""` param and a `bus { publish "io.http.**" of type std::io::tcp::LogEvent; }` declaration. When `log_subject` is set, listen-start / accept / listen-close events publish on the configured subject; empty (default) keeps the hot path at a single `len > 0` branch per event. Reuses the `std::io::tcp::LogEvent` type so one subscriber can observe both TCP and HTTP layers. | `runtime/stdlib/http.hl` |
| `std::json` | `Builder` locus (streaming output assembly — see [§ json::Builder](#stdjsonbuilder--streaming-output-api)); `escape_string` / `unescape_string` (RFC 8259); `find_string_field` / `find_int_field` / `find_bool_field` (flat-object lookup); `find_field_raw(json, name) -> String` (bracket-balanced raw substring over nested objects/arrays — the recursive-descent primitive); `ArrayIter` + `array_first` / `array_next`, and the span-bearing `ArrayIterSpan` cursor `array_first_span(json, start) -> ArrayIterSpan` / `array_next_span(it) -> ArrayIterSpan` (carry the element's byte range rather than an owned substring — the allocation-free array-walk sibling of the object cursor below). Range-bearing iter family: `iter_find_field_range(it, json, name) -> JsonFieldRange` and `iter_find_string_field_range(it, json, name) -> JsonFieldRange` return `{ok, start, end_pos}` instead of an owned-String substring; paired with the `std::str::range_*` family for fully allocation-free per-element walks on large arrays (the high-throughput workload class where per-field allocation dominates arena pressure). No nested-tree shape at v1 — re-feed substrings into the same surface for nested walks. Single-pass object member cursor: `object_first(json) -> ObjectIterSpan` / `object_next(it, json)` walk `{...}` members once, with `obj_key_eq(it, json, name) -> Bool` / `obj_key_len(it) -> Int` for key dispatch and `obj_value_int` / `obj_value_bool` / `obj_value_string` / `obj_value_raw(it, json)` reading the current value from its source range (no per-field rescan; nested objects/arrays on unmatched keys are skipped whole by the depth scan). This is the substrate a compiler-generated, schema-specialized parser drives — and the seam a future SIMD structural index slots under. | `runtime/stdlib/json.hl` |
| `std::test` | `assert(cond, msg)`, `assert_eq_int`, `assert_eq_str` | `runtime/stdlib/test.hl` |
| `std::log` | `Logger`, `LogEvent`, `StdoutSink` (subscribes `log.**`). **Sinks promoted from pond/logfmt (2026-07-18):** `FileSink` — appends every event to `path`, rotates by size (`max_size_bytes`, `keep_files`; chain shifts via atomic `rename(2)` overwrite, oldest evicted, active file recreated on next append), I/O failures captured in the `last_error_kind/errno/path` scratch triple (the styleguide-2.7 convention), also wears the `std::text::Sink` shape (write/line/newline with the same rotation). `ConsoleSink` — dim HH:MM:SS + colored width-5 level badge + dim path + message; WARN/ERROR on stderr (StdoutSink's lane split); color AUTO (tty probe on stderr; FORCE_COLOR/CLICOLOR_FORCE override; NO_COLOR always wins; `color: false` = never). pond's OtlpSink stays in pond (vendor-protocol integration is app-tier). | `runtime/stdlib/log.hl` |
| `std::metrics` | Prometheus-shaped metrics (promoted from pond/metrics, 2026-07-18). `Registry` locus (`namespace` prefix; **owns its storage** — the `MetricMap` (`@form(hashmap, sync = serialized)`) and `HistogramList` (`@form(vec)`) are param-default children, so `Registry { namespace: "app" }` is the whole construction and a Registry returned from a builder fn keeps its series alive; explicit `store:`/`histograms:` overrides remain accepted but a let-bound override dissolves at the constructing fn's scope exit — prefer the owned default). Factory free fns `counter(reg, name, labels)` / `gauge(reg, name, labels)` / `histogram(reg, name, bounds, labels)` are **idempotent on (name, labels)** — a repeat call returns a handle to the same series without resetting it; handles reference the storage slots directly (resolve once at boot, cache the handle as a field, mutate from the hot path — styleguide S12). `Counter` (`inc` / `add`, monotonic by convention), `Gauge` (`set` / `add` / `sub` / `inc` / `dec`), `Histogram` (`observe(v)` — cumulative buckets + implicit `+Inf` + `sum` + `count`; bucket bounds passed as a space-separated ascending String `"0.005 0.01 0.05"`, parsed once at registration, max 32 buckets with over-cap clamping). Labels via `labels_empty()` / `labels_one(k, v)` / `labels_two(...)` / `labels_append(l, k, v)`. `render()` emits Prometheus text exposition (`# TYPE` lines, `ns_name{k="v"} value` samples, histogram `_bucket{le=...}` / `_sum` / `_count`); `Endpoint { registry: reg }` satisfies `std::http::Handler` and answers any request with the rendering under `Content-Type: text/plain; version=0.0.4`, so `std::http::Server { handler: Endpoint { ... } }` is a complete /metrics scrape target. The MetricMap is `sync = serialized` because the canonical topology scrapes from one pool while handlers write from another. | `runtime/stdlib/metrics.hl` |
| `std::math` | `sqrt`, `exp`, `log`, `floor`, `ceil`, `pow`, `tanh`, `nan`, `is_nan`, `inf`, `sin`, `cos`, `tan`, `asin`, `acos`, `atan`, `atan2`, `int_to_float`, `float_to_int`, `round`, `trunc` | path-call dispatch into libm (`nan`/`inf`/`is_nan` are IEEE 754 sentinels; trig added 2026-05-23 for spatial / animation code). `int_to_float`/`float_to_int` are named `sitofp`/`fptosi` conversions — round-toward-zero, callable in any expression position; see `spec/types.md` § Explicit numeric conversions. `round(Float) -> Int` / `trunc(Float) -> Int` are the Float→Int siblings that return `Int` directly: `round` is round-half-away-from-zero (`3.7 → 4`, `2.5 → 3`, `-2.5 → -3`), `trunc` is round-toward-zero (an alias of `float_to_int`). Both lower to pure LLVM (`fptosi`, plus a compare/select half-shift for `round`) — **no libm symbol**, so unlike `floor`/`ceil` (which stay libm and return `Float`) they need no host import on the `wasm32` target. |
| `std::crypto` | `sha1(b) -> Bytes` (20-byte), `sha256(b) -> Bytes` (32-byte), `hmac_sha256(key, msg) -> Bytes` (32-byte), `sha512(b) -> Bytes` (64-byte) / `hmac_sha512(key, msg) -> Bytes` (64-byte) (the 64-bit-word SHA-2 sibling — FIPS 180-4 SHA-512 + RFC 2104 HMAC over a 128-byte block; same non-fallible shape as `hmac_sha256`, hand-rolled, no libcrypto; added 2026-06-25 for venue order-entry auth, which sign with HMAC-SHA512), `crc32(b) -> Int` (4-byte IEEE 802.3 checksum returned as Int; reversed polynomial `0xEDB88320`, init `0xFFFFFFFF`, final XOR `0xFFFFFFFF` — the zlib / Python `binascii.crc32` variant; added 2026-05-27). `ecdsa_p256_sign(key, message) -> Bytes` / `ecdsa_p256_verify(pubkey, message, sig) -> Bool` (ES256 — ECDSA over NIST P-256 + SHA-256; `key` is a PEM EC private key, SEC1 or PKCS#8; `pubkey` is PEM SPKI; signature is raw `r‖s`, 64 bytes, the JWS/COSE form JWT wants; added 2026-06-03 for venue/JWT auth). `ecdsa_p256_sign` has two forms: the bare call returns an empty Bytes on failure (the `base64::decode` convention — `len(sig) == 0` ⇒ failed), and in an `or` context it is `Bytes fallible(CryptoError)`, so `let sig = std::crypto::ecdsa_p256_sign(key, msg) or raise;` propagates a structured `CryptoError { kind: String, detail: String }` (`kind` = the op tag `"ecdsa_p256_sign"`; `detail` = the failure reason) — read it via `or handler(err)` / `or fail err` / `or ` exactly like `IoError` / `ParseError`. The hashes + crc32 are hand-rolled (no libcrypto); ECDSA is OpenSSL-backed (rides the libssl/libcrypto link TLS already pulls). | `lotus_crypto_*` (hashes in `runtime/lotus_arena.c`; ECDSA in `runtime/lotus_tls.c`) |
| `std::os` | `getrandom(n: Int) -> Bytes fallible(IoError)` (CSPRNG; `getrandom(2)` with `/dev/urandom` fallback) | `lotus_os_getrandom` C primitive |
| `std::rand` | `next_int(max: Int) -> Int` — a uniform-ish integer in `[0, max)` drawn from a shared xorshift64\* generator; `seed_from_time()` re-seeds that generator from the wall clock. **Not cryptographic** (deterministic PRNG, process-shared state) — for security-sensitive randomness use `std::os::getrandom`. | `lotus_rand_*` C runtime |
| `std::ts` | Tree-sitter parse substrate (m96 — the `std::ts::*` routes back the higher-level `Lang` locus). `parse_go(src: String) -> Int` parses Go source and returns an opaque **tree handle** (`Int`); `root_node(tree) -> Int` returns the root **node handle**. Node navigation (all handles are `Int`): `node_child_count(node)` / `node_named_child_count(node)`, `node_child(parent, i)` / `node_named_child(parent, i)`, `node_is_named(node) -> Int` (`0`/`1`). Kind, text, spans: `node_kind(node) -> String`, `node_text(node) -> String`, `node_start_byte(node) -> Int` / `node_end_byte(node) -> Int`. Go is the only bundled grammar at v1; the tree-sitter shim staticlib is linked into the build (gating the link on actual `std::ts` use is future work). | `lotus_ts_*` + tree-sitter shim |
| `std::bus` | `__StdBusAdapter` interface (contract for user-supplied bus transports — a single `fn send(subject: String, bytes: Bytes)` method); `__local_dispatch(subject, bytes)` primitive lets an adapter relay received wire-bytes into the local handler set. **Substrate transport URL schemes** (resolved at runtime by `lotus_bus_load_config` from `LOTUS_BUS_CONFIG=`): `unix://` (AF_UNIX SEQPACKET; m58); `udp://:` (2026-05-26 — IPv4 UDP, single scheme covers unicast and multicast: addresses in `224.0.0.0/4` trigger `IP_ADD_MEMBERSHIP` on the subscribe side, everything else takes the plain unicast bind/sendto path; lossy delivery — publishers get "sendto returned" durability, subscribers best-effort; gap recovery is a deployment concern via app-layer repeaters, MoldUDP-style). Each LOTUS_BUS_CONFIG line: `subject = :` where role is `listen` or `connect`. Other protocol-layer transports (NATS, MQTT, raw-TCP-with-framing) come in via user adapters (the `__StdBusAdapter` route above). **Payload size**: the substrate handles bus payloads up to ~64 KB (`LOTUS_PAYLOAD_MAX`, sized to UDP datagram max). Mailbox cells inline payloads ≤ 512 B (`LOTUS_PAYLOAD_INLINE` — zero malloc on the hot path); larger payloads route through a per-cell `malloc` that the drain path frees after the handler returns. The UDP transport's wire buffer is sized at LOTUS_PAYLOAD_MAX; the kernel-side socket receive buffer takes its size from `SO_RCVBUF` (default ~208 KB on Linux, raise via `LOTUS_BUS_UDP_RCVBUF=` env). UDP `sendto` failures log once per errno class to stderr (`EMSGSIZE` typically means path-MTU mismatch; see `std::io::sockopt::IP_MTU_DISCOVER` for the DF-bit knob). | `runtime/stdlib/bus.hl` + `lotus_bus_*` C runtime |
Hale doesn't use parametric stdlib collection types (`Map`, `Vec`, `Set`, etc.). Storage is locus-shaped via the
`@form(...)` annotation machinery — see
[`forms.md`](./forms.md). v1 ships `@form(vec)`
(contiguous-buffer), `@form(hashmap)` (intrusive open-addressing,
String / Int keys), and `@form(ring_buffer)` (fixed-capacity
FIFO).
## Builders vs Bytes — the recv-loop pattern
`Bytes` and `std::bytes::BytesBuilder` are deliberately distinct
types because their runtime ABIs are **incompatible**:
- **`Bytes` blob.** Single contiguous allocation: `[i64 len][u8 data[len]]`.
The handle points at the length prefix. `lotus_bytes_len(b)`
reads `*(int64_t*)b`. `lotus_bytes_at(b, i)` reads
`((u8*)b)[8 + i]`. Stable across the value's lifetime.
- **`BytesBuilder` locus.** Owns a `lotus_bytes_builder_t`
header `{cap, buf, mutation_epoch}` whose body lives in a
separately malloc'd region pointed to by `buf` and can move
on realloc (the header is stable; the body is not).
The two ABIs cannot be unified without giving up stable handles
(the body has to be relocatable; the Bytes blob layout doesn't
tolerate that). Lifting the builder into its own locus type
turns `std::bytes::at(builder, i)` into a typecheck error rather
than the silent footgun it was when the builder shared the
`Bytes` static type.
The discipline that follows:
1. **Pick one role per binding.** A binding is either a
`BytesBuilder` (long-lived growable buffer with methods
`append` (chunk: Bytes) / `append_str` (s: String — append the
string's bytes verbatim in one strlen + memcpy; `String` only,
a non-NUL-terminated `StringView` must be materialized first) /
`len` / `shift_front` / `clear` / `snapshot` /
`finish` / `view` / `text_view`, plus the binary-pack writers
below) or a `Bytes` (immutable length-prefixed blob with
functions `at` / `slice` / `len` / `concat`). The typechecker
enforces this; no implicit coercion between them.
**Binary-pack writers** (2026-06-06, shm-ring-interop Proposal A
— the inverse of `std::bytes::read_*`): `b.append_u8(n)`,
`b.append_u16_{le,be}(n)` / `u32` / `u64`, the signed
`b.append_i8`/`i16_{le,be}`/`i32`/`i64` (identical byte pattern —
width is what matters), `b.append_f32_le(x)` /
`b.append_f64_{le,be}(x)` (x: Float), and `b.append_pad(to_align)`
(zero-fill to the next `to_align` boundary). Each appends the low
`width` bytes in the named endianness; a realloc failure routes
through `violate alloc_failed` like `append`.
2. **Cross between roles via explicit calls.** `BytesBuilder →
Bytes` is either `b.snapshot()` (copies into the bus
payload arena — stable across mutations) or `b.view()` (no
copy, aliases the builder's buffer — valid until the next
mutation; the right choice for parser passes). `Bytes →
BytesBuilder` is `let b = std::bytes::BytesBuilder { ... };
b.append(bytes)` (copies). The Builder → Bytes direction has
a zero-cost path via `view()`; the reverse does not.
3. **Long-lived accumulators live as locus state.** Either a
method-body `let`-binding (dissolves at scope exit) or a
param-typed field on the owning service locus (dissolves via
the F.29 cascade at the parent's dissolve). Method-body
`let` is simpler when the lifetime fits in one method call;
field-typed is needed when the buffer must outlive a single
call (e.g., state held across bus subscription callbacks).
4. **Read syscalls write directly into the builder.** The
`recv_into` family takes a `BytesBuilder` as `buf` and
writes into its tail. Combined with `b.shift_front` after
each peeled frame (streaming) or `b.clear()` at message
boundaries (per-message accumulator), the steady-state recv
loop is zero-alloc against `g_bus_payload_arena`.
Canonical pattern, a held-open socket locus that accumulates
inbound frames:
```hale
locus FrameClient {
params { sock: Int = -1; recv_chunk: Int = 4096; }
run() {
let rx_buf = std::bytes::BytesBuilder {
initial_cap: 4096,
};
loop {
let got = std::io::tcp::recv_into(
self.sock, rx_buf, self.recv_chunk);
if got <= 0 { break; }
// peel complete frames off the front via rx_buf.len() /
// rx_buf.shift_front(consumed); for a per-frame snapshot
// use rx_buf.snapshot() only at the point of handoff to
// logic that needs a Bytes view (parsers that read via
// std::bytes::at / slice).
}
// rx_buf dissolves here → buffer freed, no explicit cleanup
}
}
```
Try writing `std::bytes::at(rx_buf, 0)` inside that loop — it
fails at typecheck (`at` expects `Bytes`, got
`__StdBytesBytesBuilder`). The discipline is mechanical, not
documentary.
The same pattern with the builder held as locus state (per
F.29), for cases where the accumulator must survive across
multiple method calls (bus callbacks, message state held
between handler firings):
```hale
locus WsClient {
params {
sock: Int = -1;
recv_chunk: Int = 4096;
rx_buf: std::bytes::BytesBuilder
= std::bytes::BytesBuilder { initial_cap: 4096 };
last_msg: std::bytes::BytesBuilder
= std::bytes::BytesBuilder { initial_cap: 4096 };
}
fn read_one() {
let got = std::io::tcp::recv_into(
self.sock, self.rx_buf, self.recv_chunk);
// ... peel frames, append payload bytes into self.last_msg
// via self.last_msg.clear() + .append(...) between message
// boundaries — no per-frame allocation.
}
// No dissolve() needed for rx_buf / last_msg — the cascade
// fires their dissolve when WsClient itself dissolves.
}
```
Consumer reads `self.last_msg` via a contract that exposes
`b.view()` — zero-copy across the F.14 interface.
## `~~std::panic~~` — not a thing
Hale doesn't have `panic(msg)`, `assert(cond)`, or any other
value-level "bail from this function" primitive. Failure is
structural, not parametric:
1. Declare a **closure** in the relevant locus asserting the
invariant you want enforced.
2. When the assertion fails at the closure's epoch, the runtime
constructs a `ClosureViolation` and routes it to the parent's
`on_failure` handler per F.9.
3. The parent picks one of `restart` / `restart_in_place` /
`quarantine` / `reorganize` / `bubble`, or absorbs the
violation by returning without calling any of them.
4. A violation that bubbles past `main` exits the process
non-zero with the violation report on stderr.
That covers every legitimate use of `panic`. "Impossible state"
becomes "a closure asserting state is possible." "Bail from this
function" is a category error in Hale — functions return
values; failure lives at the locus level.
## Form-synthesized error types
Beyond the explicit `std::*` namespace, the resolver injects
form-specific error payload types into the top scope when any
locus in the bundle uses the corresponding form. These behave
like ordinary user types after injection — they can be the
target of `fallible(...)`, declared as fn parameters / fields,
or pattern-matched in `match`. They are NOT importable via
`std::*` (they're not in a namespace); their names live at the
top level.
| Form / source | Synthesized type | Fields |
|---|---|---|
| `@form(vec)` | `IndexError` | `kind: String`, `index: Int`, `len: Int` |
| `@form(hashmap)` | `KeyError` | `kind: String` |
| `@form(ring_buffer)` | `EmptyError` | `kind: String` |
| `std::io::fs` / `std::io::tcp` | `IoError` | `kind: String`, `errno: Int`, `path: String` |
| `std::str::parse_int` / `parse_float` / `parse_decimal` | `ParseError` | `kind: String`, `input: String` |
Idempotency: if a user / library declares a type with the same
name, the user declaration wins. The injection only runs if the
target name isn't already in scope.
### `IoError`
`std::io::fs::*` (except `file_exists`) and the path-call surface
of `std::io::tcp::*` (`listen_socket`, `connect`, `accept_one`)
return `fallible(IoError)`. Agents address failures uniformly:
```hale
let s = std::io::fs::read_file(path) or raise;
let n = std::io::fs::file_size(path) or 0;
std::io::fs::mkdir(out_dir) or show(err);
```
The `kind` tag is errno-derived — `"not_found"`,
`"permission_denied"`, `"is_dir"`, `"already_exists"`,
`"would_block"`, `"connection_refused"`, `"timeout"`,
`"host_unreachable"`, `"broken_pipe"`, `"interrupted"`, etc.,
with `"io"` as the catch-all for unmapped codes. `errno` carries
the raw platform errno for callers that want it; `path` carries
the file path / connection target / empty string for socket-fd
ops without a useful path label.
`Stream.send` / `Stream.send_bytes` / `Stream.recv` /
`Stream.recv_bytes` are *locus methods* on
`std::io::tcp::Stream`, all `fallible(IoError)` since
2026-07-15 (#209; they previously used a legacy -1/0-sentinel
shape predating open-question #24 lifting the "no fallible on
locus methods" restriction). Semantics:
`send(msg: String)` / `send_bytes(b: Bytes)` succeed with Unit
(the old Int return was only ever a 0/-1 status), so
fire-and-forget callers write `s.send(x) or discard;` and
strict ones `or raise`. `recv(max) -> String` /
`recv_bytes(max) -> Bytes` **fail only on a genuine I/O
error** (connection reset, broken pipe, bad fd, …): a clean
EOF and a `set_recv_timeout` expiry both return the empty
value as before — timeout is a liveness signal, not an error.
The error path is built from the errno the primitive records
in a thread-local (`__last_io_status` / `__io_error_kind`, the
same taxonomy the fallible path-calls use). `IoError` itself
is now declared in the stdlib seed (`io_tcp.hl`), so user code
can construct / `fail` it directly.
`std::io::stdin::read_line` keeps its sentinel shape (path-call
pairing with `read_line_status` for EOF-vs-error distinction).
**`Stream` fd ownership** (`owns_fd: Bool = true`).
By default a `Stream` *owns* its `conn_fd` and closes it on
dissolve — the contract the `Listener` / `http::Server`
accept-loop helpers rely on for per-connection cleanup
(`__handle_one_connection` wraps the accepted fd in a Stream
whose scope-exit dissolve closes it). Set `owns_fd: false` to
*borrow* an fd owned elsewhere: a transient
`Stream { conn_fd: self.conn_fd, owns_fd: false }` built only to
call `send`/`recv` against a long-lived connection. A borrowed
Stream's dissolve is a no-op (no `__close_fd`, no `close`
LogEvent), so building one per operation against a persistent
connection — e.g. a WebSocket conn locus that wraps its fd per
frame — no longer closes the shared fd out from under the next
operation. Owning a fd from two live Streams at once is still a
double-close bug; `owns_fd: false` is precisely the opt-out for
the borrow case.
### `ParseError`
`std::str::parse_int(s)` / `parse_float(s)` / `parse_decimal(s)`
return `fallible(ParseError)`. The non-fallible siblings exist
only as `can_parse_*` predicate spellings; every parsing call
site must address the failure with `or`. `ParseError` carries:
- `kind: String` — `"empty"` (s was `""`), `"trailing_chars"`
(s parsed a prefix and had junk after), `"invalid"` (no
leading numeric prefix), `"overflow"` (`parse_int` only —
magnitude exceeds Int range).
- `input: String` — the original `s` (truncated to a reasonable
preview if very long), for diagnostic surfaces.
```hale
let n = std::str::parse_int(s) or 0;
let n = std::str::parse_int(s) or raise;
let n = std::str::parse_int(s) or self.report(err);
```
Reach for the predicate sibling `can_parse_int(s) -> Bool` only
when you genuinely want to branch *before* parsing. In most
cases `or` is shorter.
The qualified form `std::str::ParseError` resolves to the same
struct as bare `ParseError` — useful in projects that also
declare a local error type with the same name.
## `std::process::rss_bytes()` — observability
Returns the calling process's **peak** resident-set size in
bytes via `getrusage(RUSAGE_SELF)`. There's no syscall for
*current* RSS that doesn't go through `/proc/self/statm`; for
alarm thresholds peak is usually what matters anyway. Returns
0 if `getrusage` rejects (vanishingly rare). On Linux the
underlying value is reported in KiB and we multiply by 1024.
```hale
let bytes = std::process::rss_bytes();
println("rss=", to_string(bytes));
```
For the *current* RSS, parse `/proc/self/statm`'s line one
field two via `read_file` (size-tolerant for synthesized
files). Both surfaces ship; pick by use case (peak for
alarms, current for heartbeat gauges).
## `Server.shutdown()` — interruptible accept loop
`std::http::Server` exposes a `shutdown()` method that calls
`shutdown(SHUT_RDWR)` on the listen socket, forcing any thread
blocked in `accept()` to return immediately with an error. The
accept loop in `run()` detects the negative return and breaks;
`dissolve()` does the actual `close()`.
`shutdown()` is **safe to call from any thread, including
cross-scheduler** — that's the whole point. A cooperative-
scheduled Server can't pump its own `shutdown()` call because
the scheduler is parked in `accept()`, so the wake-up must come
from outside. Typical pattern (e.g. a pinned gateway with a
duration-bounded recv loop, sharing a process with a
cooperative metrics endpoint):
```hale
locus App {
params {
gateway: Gateway = Gateway { duration_s: 60 };
metrics: std::http::Server = std::http::Server {
port: 9100, handler: MetricsHandler { }
};
}
run() {
// gateway is pinned — runs on its own thread.
// metrics is cooperative — its run() blocks in accept.
// When gateway's run() finishes, signal metrics to
// wind down from the pinned thread.
self.metrics.shutdown();
}
}
```
The accept loop treats any negative `accept_one` return as a
clean shutdown signal, so even degenerate fd closes (external
fd-closes, etc.) terminate gracefully.
## `Server.ready_signal` — synchronization for piped oracles
`std::http::Server` accepts an optional `ready_signal: String = ""`
param. When non-empty, the server emits it via `println` from
`birth()` immediately after `listen_socket` succeeds and before
the accept loop begins. Test harnesses, oracles, and shell
scripts that pipe the server's stdout (`./bin | grep -m1 READY`)
key off this line:
```hale
std::http::Server {
port: 8080,
handler: Routes { },
ready_signal: "READY"
};
```
Pair with the line-buffered stdout setup (the prelude installs
`setvbuf(stdout, NULL, _IOLBF, 0)`) so a single `println` is
flushed even under pipes.
## `std::json::Builder` — streaming output API
`Builder` is a `@form(...)`-less locus that accumulates a JSON
document into an internal buffer. It tracks scope state in a
single context stack (one char per open scope: `O`/`A` for
object/array with at least one value already emitted, `o`/`a`
for empty). The Builder inserts separators (`,` between
siblings, `:` between key and value) automatically.
Methods, grouped:
- **Scopes:** `begin_object()`, `end_object()`,
`begin_array()`, `end_array()`.
- **Object entries (key + value in one call):** `field(name,
value)` for the common string case; `string_field`,
`int_field`, `bool_field`, `null_field` for explicit typing.
- **Array entries / bare values:** `value(v)` (string), plus
`string_value`, `int_value`, `bool_value`, `null_value`.
- **Nested scopes inside an object:** `begin_object_field(name)`
/ `begin_array_field(name)` — emit `"name":` then open the
sub-scope.
- **Finish:** `result() -> String` returns the assembled buffer.
```hale
let b = std::json::Builder { };
b.begin_object();
b.field("name", "alice");
b.int_field("age", 30);
b.begin_array_field("tags");
b.value("admin"); b.value("ops");
b.end_array();
b.end_object();
let out = b.result(); // {"name":"alice","age":30,"tags":["admin","ops"]}
```
The flat-object readers (`find_*_field`, `array_first/next`)
are the input side of the same v1 commitment: JSON is a wire
format, not a tree value type, and the API surface reflects
that.
## `read_file` for synthesized files
`std::io::fs::read_file(path)` uses a growing buffer internally
(4 KiB initial, doubling, 64 MiB cap) rather than pre-sizing
from `fstat`. Synthesized files under `/proc` and `/sys` report
`st_size=0` from `fstat`, so a fstat-then-read approach would
return an empty String for `/proc/self/statm`, FIFOs, sockets,
and similar synthetic sources. The growing-buffer shape reads
real bytes from all of them.
The 64 MiB cap is a runaway guard, not a memory budget — real
`/proc` / config files are 4–64 KiB. Callers hitting the cap
probably want a streaming API; the cap surfaces as
`IoError { kind="io", errno=EFBIG }`.
## What's not in stdlib (third-party territory)
- ML / learning libraries
- Database drivers (Postgres, etc.)
- Web frameworks beyond basic HTTP
- Image / audio / video processing
- Cloud SDKs (AWS, GCP, etc.)
- GUI / TUI frameworks
- Cryptography beyond TLS + SHA-2 basics
- Compression formats beyond ones used in stdlib
These live in the Hale package ecosystem (per
[`packages.md`](./packages.md)).
## Open decisions
1. **Module organization** — flat (`std::collections`,
`std::string`) vs hierarchical (`std::collections::Map`).
Probably the Go-style middle ground (`std/collections/map.go`).
2. **What's exported by default vs deep-imported.** `import
std;` for everything? `import std::time;` only? Probably
the latter: explicit per-module imports.
3. **API stability commitments.** Go's stdlib is famously
stable. v0 stdlib is unstable; v1 marks specific APIs as
`stable`; only stable APIs survive long-term.
4. **Versioning.** Stdlib versioned with the language? Or
independently? Probably with the language for v0; consider
independent versioning when stable.
## Why batteries-included
- **Lower adoption barrier.** New users don't need to evaluate
third-party packages for table-stakes functionality.
- **Discipline propagation.** Stdlib uses framework primitives
correctly; new code following stdlib examples inherits the
discipline.
- **Ecosystem trust.** When the language ships a `std::crypto`,
it's vetted; trust transfers to programs that use it.
- **Cross-language consistency.** Programs from different teams
share the same vocabulary because they share the same stdlib.
Cost: stdlib is permanently load-bearing once shipped. Bad
decisions are hard to undo. Discipline at design time matters
more here than in third-party.
================================================================================
## spec/ffi.md — Foreign-function interface (`@ffi("c")`)
## source: spec/ffi.md
================================================================================
# spec/ffi.md — Foreign-function interface (`@ffi("c")`)
User-extensible bindings to external C-ABI libraries. Library
authors declare extern symbols in `.hl` source via an `@ffi("c")`
annotation; the compiler emits LLVM `declare` for the signature
and the linker resolves against C source files supplied at build
time. No stdlib expansion is required to bind a new library.
## Syntax
```hale
@ffi("c") fn raylib_init_window(w: Int, h: Int, title: String) -> ();
@ffi("c") fn raylib_should_close() -> Bool;
@ffi("c") fn raylib_clear_background(c: Color) -> ();
```
Grammar:
```
ffi_annotation ::= '@' 'ffi' '(' STRING ')'
ffi_fn_decl ::= ffi_annotation 'fn' Ident '(' params ')' ('->' type_expr)? ';'
```
The annotation precedes the `fn` keyword. The fn body MUST be
absent — the declaration terminates with `;`. The compiler
synthesizes an empty body internally so downstream passes keep
the same `FnDecl` shape; user code MAY NOT write a `{...}` block.
The ABI string is the literal `"c"` (native C-ABI binding) or
`"js"` (a WASM host import — see [§ WASM host interface](#wasm-host-interface)).
Any other ABI string is rejected at parse time.
## Position
`@ffi("c")` is valid only on **top-level free fn declarations**.
The annotation is rejected on:
- Locus methods (`locus L { fn ...; }`).
- Mode bodies (`mode bulk { ... }`, `mode harmonic { ... }`, ...).
- Perspective method signatures.
- Interface method signatures.
- Closure declarations.
The position restriction matches the substrate's expectation that
the C-ABI boundary crosses at top-level program scope only; locus
and perspective methods carry implicit Hale-side context
(`self`, scratch arena, lifecycle hooks) that doesn't translate
to C.
## Restrictions
An `@ffi("c")` fn declaration MUST NOT be:
- **Generic.** Type parameters require monomorphization; the
C-ABI boundary is monomorphic by definition. Declare separate
`@ffi` fns per type if needed.
- **Fallible.** `fallible(E)` is an Hale internal channel; C
functions report failure via error sentinels in the return
value, and the Hale wrapper above translates to `fallible(E)`
if exposed to user code.
- **Defaulted.** Parameter defaults are not portable across the
C-ABI boundary; the wrapper layer applies defaults before the
call.
The parser rejects all three with a diagnostic at the annotation
or marker position.
## Type marshalling
The typechecker validates `@ffi("c")` parameter and return types
against a portable subset. LLVM lowers each Hale type to a
matching C-ABI representation at the call boundary:
| Hale type | LLVM type | C type | Notes |
|---|---|---|---|
| `Int` | `i64` | `int64_t` | 64-bit signed throughout. |
| `Float` | `double` | `double` | 64-bit IEEE 754. |
| `Bool` | `i32` | `int32_t` | Hale's i1 zero-extends to i32 at the call, truncates back at the return. Avoids C `_Bool` cross-platform ambiguity. |
| `String` | `ptr` | `const char *` | NUL-terminated. Caller owns; callee MUST NOT retain past the call. A `StringView` does **not** implicitly coerce to a `String` parameter — it isn't NUL-terminated, so a `char*`-expecting callee would `strlen` past its end. Pass a view as a `StringView` parameter (→ `lotus_view_t`, length-carrying) or materialize it first via `std::str::clone`. |
| `Bytes` | `ptr` | `void *` (header) | Points at Hale's `[int64 len][payload]` header — callee uses `lotus_bytes_len(p)` / `lotus_bytes_data(p)` (declared in `lotus_arena.h`) to inspect. Caller owns. |
| `BytesView` / `StringView` | `{ ptr, i64 }` (struct by value) | `lotus_view_t` | 16-byte F.30b view layout. C glue MAY use `lotus_view_data` to recover the payload pointer + length. |
| `Duration` / `Time` | `i64` | `int64_t` | Both are 64-bit nanosecond counts under the hood. |
| `()` (unit) | `void` | `void` | Return-position only — declared as `-> ()` or omitted entirely. Empty-tuple return type accepted but normalized to `()`. |
| User struct (`type T { ... }`) | `ptr` | `const T *` (param) / `T *out` (sret return) | Passed by pointer at the boundary; struct returns use a hidden sret first arg (see User-type structs section below). Layout match is the library author's responsibility. |
Reserved at Stage 1 (typecheck rejects with a clear diagnostic):
- `Decimal` — i128 mantissa with platform-variable ABI. Marshal as
`Int` (raw mantissa) or `Float` (lossy conversion) at the
Hale side; the wrapper handles the scale.
- `Uint` — Hale-internal type; declare as `Int` at the FFI
signature.
- Projections / fixed-size arrays / tuples — no portable C struct
layout for these v0 shapes.
- `fallible(E)` — internal channel; see Restrictions above.
- Function-pointer types — wrap as a struct/handle at the C side.
- `LocusRef`, `Cell` — Hale-internal.
### User-type structs
User-type structs (`type Color { r: Int = 0; ... }`) are passed
**by pointer** at the C-ABI boundary, not by value. The Hale
side already stores user structs as heap pointers, so the natural
mapping is `ptr` at the LLVM level. C glue authors write:
```c
// Param-position: const T * (or T * if the callee mutates).
void raylib_clear_background(const Color *c) {
ClearBackground((::Color){
(uint8_t)c->r, (uint8_t)c->g,
(uint8_t)c->b, (uint8_t)c->a,
});
}
```
Struct returns use **sret-style**: Hale allocates the return
slot in the caller's arena and passes a pointer as a hidden first
argument. The LLVM-level fn signature is `void foo(T *out,
)`; the C glue writes the struct into `*out`:
```c
// Return-position: hidden T *out first param, returns void.
void vec3i_scale(Vec3i *out, const Vec3i *v, int64_t k) {
out->x = v->x * k;
out->y = v->y * k;
out->z = v->z * k;
}
```
The Hale-side call expression
```hale
let scaled = vec3i_scale(v, 10);
```
sees the sret slot's pointer as its result — same value-shape
as any other struct-returning expression. The sret transformation
is hidden from user code; only the C glue author sees it.
**Why pointer + sret instead of by-value:** SysV / Win64 / aarch64
all classify struct-by-value differently based on size. A
portable implementation would need a per-platform ABI-lowering
pass. The pointer convention sidesteps that entirely — every
target lowers `ptr` the same way — at the cost of one
dereference per arg on the C side. For the workloads Hale is
shaped for (locus methods, bus dispatch, FFI to system
libraries), that cost is negligible compared to the portability
win.
**Layout contract:** the Hale struct's field order + types must
match the C struct on the other side. The library author
guarantees this. Future spec iteration may add a compile-time
layout-assertion mechanism (`@ffi_layout("c")` on the `type`
decl); today the contract is documented but not machine-checked.
## Calling convention
`@ffi` fns differ from regular Hale free fns at the LLVM ABI
level:
- **No implicit `__caller_arena` first parameter.** Regular free
fns receive the caller's `current_arena_ptr()` as an implicit
prefix; `@ffi` fns do not.
- **No fallible sret slots.** `@ffi` fns can't be `fallible(E)`,
so the sret-pair the substrate emits for fallible returns is
absent.
- **No monomorphization.** `@ffi` fns can't be generic.
The LLVM symbol name is the literal Hale fn name as written.
There is no `__std_*` mangling, no per-import alias prefix, no
generic-instantiation suffix. The library author's C glue
exports a function with that exact name; the linker resolves
directly.
## Lifetime rules
The Hale-side caller of an `@ffi` fn owns every pointer it
passes. The C-side callee MUST:
- NOT retain `String` / `Bytes` / view pointers past the call
boundary. If C needs persistent storage, it must copy into its
own malloc'd memory.
- NOT free or write through any pointer received from Hale.
Arena-owned pointers are read-only at the C side.
If a C function needs to RETURN heap-allocated `String` or
`Bytes`, the convention matches stdlib primitives that allocate
return values: call `lotus_arena_alloc(lotus_caller_arena_or_global(),
size, align)` to land the storage in the caller's arena, then
return the pointer. The caller's arena outlives the C-side
function frame, so the returned pointer survives.
Exceptions MUST NOT cross the FFI boundary. C code that fails
returns an error sentinel (NULL, -1, etc.); the Hale-side
wrapper translates to a `fallible(E)` shape if the error needs
to propagate.
## Build surface
The `hale build` CLI accepts repeatable flags that thread the
library author's C glue + link surface through to clang:
```
hale build mydir/ --link raylib --csrc pond/raylib/glue.c \
--link curl --csrc pond/curl/glue.c
```
- `--link ` — appended as `-l` to the clang link
line. The system's dynamic linker resolves at runtime.
- `--csrc ` — passed directly to clang as a translation
unit compiled alongside the C runtime. The library author's
`.c` glue file goes here. May be repeated for multiple files.
Both flags are optional; programs that don't use `@ffi`
declarations don't need either.
### `hale.toml [ffi]` auto-pickup (Stage 2)
When `hale build` resolves an `import` against a directory
that contains an `hale.toml`, it reads the file's `[ffi]`
section and appends those values to the build's link surface
automatically. Library authors ship:
```toml
# pond/raylib/hale.toml
[ffi]
link = ["raylib"]
csrc = ["glue.c"]
```
Consumers then just `import`:
```hale
// myapp/main.hl
import "vendor/raylib" as ray;
fn main() {
let w = ray::Window { width: 1280, height: 720 };
...
}
```
`hale build myapp/` reads `vendor/raylib/hale.toml`, picks
up `link=["raylib"]` + `csrc=["glue.c"]`, and threads them
through to the clang invocation. The CLI flags from the prior
section still work as additive overrides (CLI first, then toml-
sourced); duplicates are tolerated. Single-file imports
(`import "helpers"` → `helpers.hl`) have no companion toml and
contribute nothing.
De-duplication: a lib referenced under two aliases or via
multiple files in the same seed contributes its FFI flags once
per unique resolved directory.
Transitive FFI is NOT walked at Stage 2: only the entry's
top-level imports are scanned for `hale.toml`. If a directly-
imported lib itself imports another `@ffi`-using lib, the
transitive lib's `[ffi]` must be re-declared (or surfaced via
manual `--link` / `--csrc`) at the entry. Resolved if a workload
surfaces the need.
## Library-author surface
A binding library typically ships:
1. A `.hl` file with `@ffi("c") fn ...;` declarations + the
user-facing Hale wrapper (locus, types, idiomatic
signatures).
2. A `.c` file exporting the C-side symbols declared in the
`.hl`. Often a thin shim from Hale's snake_case to upstream
C naming.
3. (Stage 2) An `hale.toml [ffi]` section declaring
`link = [...]` and `csrc = [...]`.
Example skeleton (pond/raylib):
```hale
// pond/raylib/raylib.hl
@ffi("c") fn raylib_init_window(w: Int, h: Int, title: String) -> ();
@ffi("c") fn raylib_close_window() -> ();
locus Window {
params { width: Int = 1280; height: Int = 720; title: String = ""; }
birth() { raylib_init_window(self.width, self.height, self.title); }
dissolve() { raylib_close_window(); }
}
```
```c
// pond/raylib/glue.c
#include
#include "raylib.h"
void raylib_init_window(int64_t w, int64_t h, const char *t) {
InitWindow((int)w, (int)h, t);
}
void raylib_close_window(void) { CloseWindow(); }
```
```toml
# pond/raylib/hale.toml (Stage 2)
[ffi]
link = ["raylib"]
csrc = ["glue.c"]
```
## Diagnostic surface
Parser errors:
- `expected ; (an @ffi fn declaration has no body), got LBrace`
— body block written after the signature; convert to `;`.
- `unsupported FFI ABI "" — Stage 1 accepts only "c"`
- `\`@ffi\` fn must not be generic — the C-ABI boundary is
monomorphic`
- `\`@ffi\` fn must not be \`fallible(...)\` — C functions
return an error sentinel, the Hale wrapper above translates
to \`fallible(E)\` if needed`
- `expected \`fn\` after \`@ffi(...)\` annotation`
- `expected \`fn\` after \`@export\` annotation`
- `\`@export\` and \`@ffi\` are mutually exclusive — an FFI import
is not a module export`
Typecheck errors:
- `\`@ffi\` fn \`\` parameter \`]\` has type Decimal —
Decimal (i128) has platform-variable ABI; marshal as Int/Float
at the Hale side instead`
- `\`@ffi\` is only valid on top-level free fns at Stage 1, not
on locus methods`
Codegen errors:
- `@ffi fn \`\` parameter \`\`: type is not yet
wired for FFI codegen at Stage 1` — user-type structs, arrays,
etc. fall here.
- `@ffi fn \`\`: parameter defaults are not supported
across the C-ABI boundary`
- `@export fn \`