Forms
Reference material, synced from the compiler repo’s
spec/. The guide is the gentler path in.
A form is a compiler-recognized annotation on a locus
declaration that picks an efficient lowering for the locus’s
storage and synthesizes a standard method set. Forms are the
mechanism Hale uses in place of parametric collection types
(Map<K, V>, Vec<T>, etc.). See
spec/decisions.md for The Design’s grounding (F.0
form-before-parameter, F.22 capacity).
This document specifies the form annotation system in general
(syntax, contract, verification) and the @form(vec) contract
in detail. Subsequent forms (@form(hashmap),
@form(ring_buffer), @form(lru_cache), @form(heap)) get their
own sections as they’re committed.
Annotation syntax
Section titled “Annotation syntax”form_annotation = "@form" "(" form_name [ "," form_arg { "," form_arg } ] ")"form_name = LOWER_IDENTform_arg = IDENT "=" expressionA form annotation sits on the line above a locus declaration,
like the existing @projection annotation:
@form(vec)locus ItemList<T> { capacity { heap items of T; }}form_name— the form identifier. Lowercase, single word. The v1 form library is fixed (see “v1 form library” below); user-defined forms are deferred to a future release.form_arg— keyword arguments specific to the form. Used for tuning knobs that don’t change storage discipline (e.g.cap = 100for@form(lru_cache)/@form(ring_buffer)).- One form per locus. Composition (
@form(vec) @form(ordered)) is rejected in v1.
Form-specific configuration that does change storage
discipline goes on the capacity slot, not in the annotation
arguments — see “indexed_by and slot clauses” below.
Form contract
Section titled “Form contract”Each form specifies three things the compiler verifies and implements:
- Required capacity shape. What slots the locus must declare, of what kinds, holding what cell types. Verified at typecheck.
- Synthesized method set. Names, parameter types, return types of methods the form provides. Injected at typecheck so call sites resolve normally.
- Lowering strategy. What C-runtime substrate the compiler emits in place of the literal F.22 pool / heap lowering.
If the locus’s shape doesn’t match the form’s required capacity, the compiler emits a focused diagnostic and rejects the program. Example:
error[FORM-SHAPE]: @form(vec) requires exactly one `heap` slot; found `pool entries of CmdEntry` instead. --> registry.hl:3:1 | 3 | @form(vec) | ^^^^^^^^^^ 4 | locus Registry { capacity { pool entries of CmdEntry; } } | ---------------------------- | expected `heap items of T`Synthesized methods
Section titled “Synthesized methods”The form synthesizes its standard method set. The user does not declare them; call sites resolve as if they were declared.
@form(vec)locus ItemList<T> { capacity { heap items of T; } // push, get, set, pop, len, is_empty come from @form(vec).}
fn main() { let l = ItemListL_Int { }; l.push(42); let head = l.get(0) or raise; println(head); // 42}The user CAN add additional methods on top of the
synthesized standard set. Naming a user method that collides
with a synthesized method (e.g. user writes their own push)
is rejected at v1 — override is deferred to v2.
@form(set) — membership without a value
Section titled “@form(set) — membership without a value”A set is a hashmap whose value is carried but never surfaced: the
cell IS the key. It reuses the hashmap slot and the whole
lotus_hashmap_* runtime, sync disciplines included, so
@form(set, sync = striped) works exactly as the hashmap does.
type Item { key: String; }
@form(set)locus Seen { capacity { pool items of Item indexed_by key; } }The capacity shape is identical to @form(hashmap). Only the
synthesized surface differs:
| method | signature |
|---|---|
insert(v) |
value struct in; the key comes from indexed_by |
contains(k) |
Bool, not a fallible — “absent” is the ordinary answer to a membership question |
remove(k) |
fallible(KeyError), mirroring the hashmap’s |
len() / is_empty() |
as elsewhere |
That surface is the reason the form exists. Membership through
@form(hashmap) means writing get(k) or false at every call site —
the value plumbing leaking back out of a container whose value nobody
wants.
indexed_by and slot clauses
Section titled “indexed_by and slot clauses”Form configuration splits between slot clauses and annotation arguments. The dividing line:
- Slot clause — if the configuration changes how cells are laid out or accessed. A storage-discipline concern.
- Annotation argument — if the configuration is a policy / tuning knob the form’s runtime consults; the underlying storage shape is the same regardless.
// Storage discipline — slot clause.@form(hashmap)locus CmdRegistry { capacity { pool entries of CmdEntry indexed_by name; } // ^^^^^^^^^^^^^^^ // slot clause}
// Policy / tuning — annotation argument.@form(lru_cache, cap = 100)locus SessionCache { capacity { pool sessions of SessionEntry indexed_by id; }}indexed_by is a slot clause because indexing IS a storage
commitment — it changes the pool’s layout and access path. A
@form(hashmap) key field is an Int or a String (an identity, a
range or a quantity is its Int): the runtime hashes those two, and
any other field type is refused at the indexed_by name (“@form(hashmap)
key `Row.k` is a `Bool`: a hashmap’s key is an `Int` or a
`String`”). Until 2026-10-13 the check admitted it and the build
refused the synthesized get / set.
The same clause on a struct field, checks: bounded[Check; 16] indexed_by name;, is the hashmap model’s value-level projection (2026-10-17): no
form and no storage of its own, a bounded list whose elements carry their
key, which the api codecs carry as a map keyed by it (spec/api.md §
Codecs, spec/types.md § bounded[T; N]). Its key is a String field.
indexed_by name -> status names the element’s one other field, which
crosses alone as each key’s value: a map of values, whose element is a
record of the key and that value.
@form(heap)’s ordered_by min <field> / ordered_by max <field>
is one for the same reason: the order decides where every cell sits.
ordered_by on a slot of any other locus is a type error.
Default lowering (no form annotation)
Section titled “Default lowering (no form annotation)”A locus without @form(...) gets the literal F.22 default
lowering: pool slots become lotus_pool_t* chunked free-list;
heap slots become a lotus_heap_t* doubly-linked live list
(individual O(1) alloc/free, wholesale-freed at dissolve — the
contiguous doubling buffer is the @form(vec) specialization,
not the default). The user’s
own methods run as written; no synthesis, no shape verification
beyond the normal capacity-slot machinery.
The form annotation is the user’s opt-in to a specific efficient lowering. Without it, you get the predictable F.22 default.
Form-annotated loci as application-layer storage substrate
Section titled “Form-annotated loci as application-layer storage substrate”A @form(...) locus occupies a different position in The
Design’s taxonomy than a user-declared locus, and the
distinction is load-bearing for the two-channel failure rule
(spec/semantics.md § “Fallible call semantics” § “Where
each channel lives”):
- User-declared loci are substrate-facing — they
participate in the locus tower’s lifecycle (bus
subscriptions, modes, contract reads, lifecycle methods).
Their methods communicate failure structurally via
closure assertions +
on_failurerouting. @form(...)loci are application-layer storage substrate — they realize a substrate-honest container shape that application code uses to hold data. Their synthesized methods (@form(vec).get,@form(vec).pop, future@form(...)accessors) operate per-access and may be declaredfallible(E), addressing failure at the immediate caller’sorclause.
This is why the synthesized @form(vec) get / pop
methods carry fallible(IndexError) while user-declared
locus methods cannot. The @form(...) annotation is the
declaration-site marker that “this locus is application-
layer storage substrate, not a substrate-structural
participant.” The synthesized method surface gets the
application-layer failure channel; the underlying form-vec
locus still respects every other substrate invariant
(arena ownership, dissolve cascade, capacity slot
discipline).
Perspectives and forms
Section titled “Perspectives and forms”Perspectives reflect on structure, not on lowering.
The form annotation changes how the compiler lays out memory and synthesizes methods. It does not change:
- The locus’s name or place in the tower.
- The set of fields declared in
params. - The capacity slot declarations.
- The
closure/on_failure/busblocks.
Perspectives that reflect on a form-lowered locus see the structural view: the capacity slots, the params, the method signatures (synthesized or user-written, treated uniformly). They do not see the underlying C struct layout.
Performance commitment
Section titled “Performance commitment”The form machinery commits to three distinct perf shapes, distinguished because they measure different things and have different bands:
(a) Tight-loop primitive cost. A form-lowered primitive (e.g.
@form(vec).push) must run within 10% of a hand-written equivalent in idiomatic C on a microbench that exercises the primitive in isolation.
(b) Amortized workload cost. A form-lowered workload that mixes the form’s primitives with real per-call work (the shape real apps exhibit) must run within 2× of an equivalent idiomatic C program.
(c) Per-op fallible-method cost. A form-lowered fallible primitive (e.g.
@form(vec).get/.pop/@form(hashmap).get/.remove) measured in isolation pays the C-function-call boundary to thelotus_*primitive plus the fallible-ABI plumbing. No 10% commitment at v1; isolated-microbench numbers may show 10–50× behind C. The contract is that fallible primitives are correct, predictable, and competitive when amortized (the (b) band).Update: the isolated gap is largely closed.
@form(vec).get/.set/.pop/.pushare now inlined directly at codegen — bounds-check + typed GEP + load/store, nolotus_*C-call boundary..getindexed by a counted-loop variable (for i in 0..v.len(), the vec unmutated in the body) additionally drops the bounds check entirely — it’s provably in-bounds — so the read vectorizes. The remaining cross-boundary calls (arena allocation, etc.) inline under opt-inLOTUS_LTO=1(seeruntime.md). The “IR-level inlining or LTO” this band once deferred to is shipped.
These bands track the same underlying performance reality at
different observer-perspectives (per The Design’s
form/parameter cut): (a) measures primitive layout correctness,
(b) measures whether the substrate amortizes well at scale, (c)
measures the codegen-pattern overhead per primitive call.
@form(vec) is the canonical benchmark target (see “Bench
protocol” under the @form(vec) section below).
Current standing (2026-06-28, vs Go at matched iteration counts):
| Bench | Hale vs Go | Band | Status |
|---|---|---|---|
form_vec_push (500k push) |
4.83× | (a) | beats Go ✓ |
vec_amortized (push + fold, 200k) |
3.75× | (b) | beats Go ✓ |
form_vec_get (200k get) |
2.60× | (c) | beats Go ✓ |
fn_scratch_work (1k calls w/ work) |
7.05× | (b) | beats Go ✓ |
Hale vs Go = Go time ÷ Hale time (> 1 → Hale faster). The earlier
(2026-05-13) snapshot put form_vec_get at 0.026× and vec_amortized
at 0.42×; those were a benchmark iteration-count mismatch — the
Hale variants ran 20–25× more work than the .go/.js/.py
siblings (see the bench repo’s N-audit) — compounded by
pre-optimization codegen. At matched N, with the .get/.set/.pop/
.push inlines + counted-loop bounds-check elimination + native-CPU/O3
defaults, Hale leads Go on all four. The formal within-10%/2×-of-C
verification for bands (a)/(b) still awaits the C twins noted in the
bench harness.
If a form fails its applicable band, the lowering is redesigned before shipping more forms. The point of the form machinery is not to be clever — it’s to be roughly as fast as the C the user would have written by hand, with all the locus tower’s structural benefits on top.
@form(vec)
Section titled “@form(vec)”A contiguous, growable buffer of T. The Hale analogue of
Vec<T> / std::vector<T> / Go slices. First form committed
for v1; canonical benchmark target for the 10% perf gate.
Required capacity shape
Section titled “Required capacity shape”The locus MUST declare exactly one heap slot. Its cell type
becomes the vec’s element type T.
@form(vec)locus ItemList<T> { capacity { heap items of T; }}Rules verified at typecheck:
- Exactly one slot. Zero slots, more than one slot, or any
poolslot is rejected. - The slot MUST be a
heapslot. (poolis the unordered free- list shape;vecis the contiguous shape — they’re different storage disciplines, so apooldeclaration with@form(vec)is a contradiction.) - The slot name is user-chosen and is not part of the contract.
The compiler finds the form’s heap slot by position, not by
name. Idiomatic spellings:
items,entries,bytes,xs.
The cell type T may be:
- A primitive (
Int,Float,Bool,Decimal,Time,Duration,String,Bytes). - A user-defined
type(struct or enum). - A generic parameter (
heap items of Tinside a generic locusItemList<T>); monomorphization (m63) produces a concrete@form(vec)instance per binding.
The cell type MAY NOT be a locus reference — vecs hold values,
not loci. If you want a vec of child loci, use the F.22 pool
projection-class machinery instead; that’s the structural shape
for parent-owns-children.
Synthesized methods
Section titled “Synthesized methods”fn push(x: T) -> () # infalliblefn get(i: Int) -> T fallible(IndexError)fn set(i: Int, x: T) -> () fallible(IndexError)fn pop() -> T fallible(IndexError)fn len() -> Int # infalliblefn is_empty() -> Bool # infalliblefn sort() -> () # infallible; T in {Int, Float, String}fn sort_by(cmp: fn(T, T) -> Bool) -> () # infalliblefn sort_desc_by(cmp: fn(T, T) -> Bool) -> () # infalliblefn clear() -> () # infalliblefn truncate(n: Int) -> () # infallibleThe fallible methods return the locus-defined IndexError
payload type:
type IndexError { kind: String; # "out_of_bounds" or "empty" index: Int; # the requested index (0 for empty-pop) len: Int; # the vec's len at fail time}IndexError is defined in the synthesized form preamble; the
user does not declare it. The same type is shared across all
@form(vec) instantiations (it’s a flat record, not parametric
over T).
fn push(x: T) -> ()Appends x after the last element. Amortized O(1). The
synthesized lowering grows the underlying buffer by doubling
when capacity is exhausted; the realloc cost amortizes across
N appends to O(N) total.
push is infallible. OOM during the doubling realloc is a
substrate-level concern: the C runtime traps malloc failure and
re-raises as a closure violation, not as a fallible(...)
return on push. From the language surface, push never
errors. (See spec/runtime.md for the OOM trap convention.)
fn get(i: Int) -> T fallible(IndexError)Returns the element at index i (0-based). If i < 0 or
i >= len(), fails with IndexError { kind: "out_of_bounds", index: i, len: self.len() }.
Idiomatic call sites:
let head = vec.get(0) or raise; # bubble on emptylet first = vec.get(0) or default_value; # substitutelet nth = vec.get(i) or handle_oob(err); # custom handlerfn set(i: Int, x: T) -> () fallible(IndexError)Overwrites the element at index i (0-based) with x. If
i < 0 or i >= len(), fails with IndexError { kind: "out_of_bounds", index: i, len: self.len() }. set does not
extend the vec — index must be inside the current length;
appending new elements uses push.
vec.set(0, new_first) or raise;vec.set(i, x) or noop(err); # swallow OOBReplaced-element reclamation (iris handoff P1, 2026-07-27). Vec elements are pointer-storage: each
setdeep-copies the new value into the form owner’s arena. The REPLACED element (and its non-surviving String and Bytes fields, with aliasing dedup — the hashmap retire-cell discipline) now retires onto the arena’s reuse freelist immediately, and the deep-copy allocation consults that freelist — steady-statesetchurn ping-pongs between reused blocks instead of growing the arena (~33 B/set and a progressively slower containment walk, pre-fix). Bytes fields since GH #1037: before it a struct’s replaced Bytes stayed (~47 B a set). A field that holds a block of its own — a nested record, a tuple, a fixed array (a record’s field of scalars is inline and owns no block; inside a tuple or another array, even a scalar array is a block of its own) — retires with everything it owns, at any depth: before, the walk stopped at the top-level String and Bytes fields, so a replaced element’s nested record (its block and its own String and Bytes) stayed in the arena for good (~110 B a set for one nested record carrying a String and Bytes, ~210 B for two levels). A block the new element still reaches survives, wherever in it, and a block reached twice is freed once. (@form(hashmap)cells retire through the same descriptor, deferred to the activation boundary — see@form(hashmap)§ Cell reclamation.)The vec owns its elements (GH #577).
getreturns the CALLER’s copy of a heap-bearing element (a String, Bytes, a struct with such fields), allocated in the caller’s current arena;setandpushstore the VEC’s own copy, whatever arena the value came from — a value read from the same vec, or one value pushed twice, never leaves two slots sharing one struct. So agetresult stays whole across a latersetof its slot, and the retire above frees only what the vec held. Scalars and locus refs are copied by value as before. (Before this,gethanded back the slot’s pointer and a same-arena value passed throughsetuncopied; two items swapped throughgetandsetsegfaulted.) The copy goes down to the String and Bytes fields: one whose block already lives in the vec’s arena, such as the locus’s own String param (push(Row { id: self.a })), is copied as well, not shared, as a@form(hashmap)cell’s store copies it. A slot that shared the param’s block freed it under the param when the slot was next replaced (2026-10-16).The copy is an allocation in the caller’s arena, freed with the frame — the same cost and lifetime as a hashmap
get. A frame that reads a heap-bearing element millions of times before it returns holds every copy until then; a hot walk usesfor x in v.items, which visits the elements in place without copying (and must notsetthe vec it walks).
fn pop() -> T fallible(IndexError)Removes and returns the last element. If len() == 0, fails
with IndexError { kind: "empty", index: 0, len: 0 }.
Like get, pop returns the CALLER’s copy of a heap-bearing
element, allocated in the caller’s current arena; the element the
vec held — its block, its String and Bytes fields, and its nested
records’ blocks with what they own — is then freed onto the arena’s
reuse freelist, as a replaced element is on set (GH #1037). A vec used as a queue (push, then pop once
handled) therefore holds its arena flat. (Before this, pop handed
back the slot’s own pointer and freed nothing: ~40 B a cycle for a
bare String cell, ~62 B for a struct carrying one, for as long as
the owner lived.)
pop does not free the underlying buffer — capacity does not
shrink. Buffer release happens at locus dissolution.
clear and truncate
Section titled “clear and truncate”fn clear() -> ()fn truncate(n: Int) -> ()truncate(n) keeps the first n elements and drops the rest; an
n at or past len() changes nothing, and an n below zero drops
every element. clear() is truncate(0). Both are infallible.
Each dropped element is reclaimed exactly as pop reclaims the one it
removes — its block, its String and Bytes fields, and its nested
records, tuples, arrays and payload enums with what they own, at any
depth, back onto the arena’s reuse freelist — and the buffer keeps its
capacity, so a scratch vec hoisted into a locus and cleared once per
message holds both its buffer and its arena flat. A value read out
before the drop (a get or pop result) is the caller’s own copy and
stays whole. (Friction sweep 2026-10-12: there was no way to empty a
vec but a pop loop, while bounded[T; N] had clear.)
len and is_empty
Section titled “len and is_empty”fn len() -> Intfn is_empty() -> Boollen() returns the number of elements currently in the vec.
is_empty() is sugar for len() == 0. Both are infallible and
O(1).
fn sort() -> ()Sorts the vec in place in ascending order. The cell type T MUST
be one of Int, Float, or String; any other T (struct, enum,
bytes, etc.) is a typecheck error suggesting sort_by(cmp).
String comparison is lexicographic on the underlying byte
sequence (i.e., the C strcmp ordering); Float comparison
treats NaN as equal-to-anything to keep the ordering total
(no panic on NaN-bearing inputs).
sort is infallible. The substrate uses C qsort under the
hood — average O(N log N), worst-case O(N²) on pathological
inputs.
sort_by and sort_desc_by
Section titled “sort_by and sort_desc_by”fn sort_by(cmp: fn(T, T) -> Bool) -> ()fn sort_desc_by(cmp: fn(T, T) -> Bool) -> ()Sort the vec in place under a user-supplied strict-less-than
comparator. The comparator’s semantics: cmp(a, b) == true
means “a should come before b in the result.” This is the
strict-< shape — cmp(a, a) SHOULD return false for any
a; a reflexive <= produces an unstable ordering but does
not panic.
sort_desc_by(cmp) is equivalent to sort_by(|a, b| cmp(b, a))
with the arg order swapped under the hood, so the same user
predicate produces the reverse ordering. Provided as a
convenience for the common “descending under the same key
extractor” pattern.
Both methods are infallible from the language surface. If cmp
itself faults (e.g., raises via or raise on a fallible call
inside the comparator body), the fault propagates through the
sort and the vec is left in an unspecified but valid state
(every element still present, ordering partially applied).
fn by_x(a: Point, b: Point) -> Bool { return a.x < b.x; }points.sort_by(by_x); # ascending by xpoints.sort_desc_by(by_x); # descending by x — same cmpThe cell type T may be any sortable shape, including structs and
enums. The substrate uses qsort_r with a per-(cell-type,
direction) trampoline synthesized at codegen time; the cookie
threads the caller’s arena pointer through so the comparator’s
body can use stdlib calls that allocate.
Lowering strategy
Section titled “Lowering strategy”@form(vec) lowers the heap slot to a three-field C struct:
typedef struct { size_t cap; // allocated capacity (elements) size_t len; // number of valid elements T* buf; // contiguous element array} lotus_vec_<T>_t;- Initial capacity:
0(no allocation at birth). Firstpushallocates the initial buffer. - Initial buffer size on first push:
4elements. Chosen as a small constant that avoids the malloc-per-element shape without over-allocating for short-lived vecs. - Growth policy: double
capon overflow. New buffer is malloc’d; old elements arememcpy’d; old buffer is freed. - Shrink policy: none. Capacity is monotonic in v1. (A
shrink_to_fitmethod may be added later if a workload surfaces the need.)
Element storage is by-value: a @form(vec) of Int is a
contiguous int64_t[]; a @form(vec) of type Pair { x: Int; y: Int; } is a contiguous array of {int64_t, int64_t}
records. No per-element heap allocation, no per-element header.
For elements of pointer-shaped types (String, Bytes), the
vec stores the pointer by value; the pointed-to bytes live in
whatever arena they were allocated from. Dissolution of the vec
frees the buffer but does not free the pointed-to bytes — those
follow their owning arena’s lifetime per the standard F.22
contract.
Arena ownership
Section titled “Arena ownership”@form(vec) is not a separate arena-allocated structure.
The three-field lotus_vec_* struct lives inline in the locus’s
struct layout, the same way the literal heap items of T
declaration would. The growable buffer (the buf field) is
malloc’d from the system allocator, not from the locus’s
arena — this is the existing F.22 heap-slot contract, unchanged.
Dissolution: when the locus arena is destroyed, the vec’s
buf is freed via the F.22 dissolve cascade (the synthesized
destructor emits free(self.items.buf) for each formed heap
slot).
Interaction with the locus tower
Section titled “Interaction with the locus tower”A @form(vec) locus is a locus in every other respect. It can:
- Have
params { ... }with defaults. - Have
birth,run,drain,dissolvelifecycle bodies. - Declare
closure { ... }invariants. - Route failures via
on_failure(child, err). - Participate in a
bus. - Be projected by
perspectivedeclarations.
These are orthogonal to the form annotation. The form replaces the literal F.22 heap-slot lowering; it does not replace any other locus mechanic.
Bench protocol (FORM-3 gate)
Section titled “Bench protocol (FORM-3 gate)”@form(vec) is the canonical benchmark target. The three perf
bands above map to three benches that ship under
micro/ in the sibling hale-lang/bench repo:
- Tight-loop primitive (band (a), 10% gate).
form_vec_push— 1Mpushon a@form(vec)ofInt, compared against an equivalent hand-written C program usingmalloc+ doubling realloc and rawint64_t[]indexing. Wall-clock and peak RSS. Status 2026-05-13: 1.00× ratio vs Go (effectively at C parity); gate met after thelotus_arena_create_subregionelision for non-allocating fn bodies (notes/form-perf-checkpoint.mddocuments the path). - Amortized workload (band (b), 2× gate).
vec_amortized— 200k push + 200k fold over the result, timed in one region. Status 2026-05-13: 0.42× ratio vs Go (2.4× behind) — outside the 2× band; investigation pending. - Per-op fallible (band (c), advisory).
form_vec_get— 200k indexed reads viavec.get(j) or raise. Status 2026-05-13: 0.026× ratio vs Go (~38× behind isolated). Documented residual; advisory only at v1 — the gap is the C-function-call boundary tolotus_vec_getplus the fallible-ABI plumbing. Closing it requires either inlining the primitive’s logic in IR at codegen time, or LTO. Both deferred until a workload measures the cost. - App bench. A representative app rewritten to use
@form(vec)where it currently does explicit F.22 pool walks. Wall-clock and RSS compared before / after, with the form-lowered version targeted to be no worse than the F.22 baseline. Not yet wired.
The microbench harness lives in the sibling hale-lang/bench
repo; its run.sh resolves the hale binary via
$HALE_BIN → hale on PATH → ../hale/target/release/hale.
Bench sources are sibling .hl / .go / .js / .py files
under micro/ and app/.
If a bench fails its applicable band, the lowering is
redesigned before further forms are added to the library.
@form(hashmap) shipped via v1.x-FORM-4 without a parallel
bench under this protocol (the band (a) win on
form_vec_push was held to license the form-machinery
extension); a micro/form_hashmap_* family in hale-lang/bench
is the natural follow-up if perf becomes load-bearing for
hashmap consumers.
Anti-patterns
Section titled “Anti-patterns”Hand-rolling the contract on a form-annotated locus
Section titled “Hand-rolling the contract on a form-annotated locus”// WRONG — @form(vec) synthesizes push; user declaration// collides with the synthesized name.@form(vec)locus ItemList<T> { capacity { heap items of T; } fn push(x: T) -> () { /* ... */ } // rejected}The compiler rejects this at typecheck with error[FORM-COLLIDE]: @form(vec) synthesizes push; user declaration shadows the synthesized method (override is deferred to v2).
Ignoring the fallible return
Section titled “Ignoring the fallible return”// WRONG — `get` returns fallible(IndexError); the bare let// binding drops the error.let v = vec.get(i); // compile error: `vec.get` can fail (IndexError) and this call says nothing about it// RIGHT — address the error with one of the three motions.let v = vec.get(i) or raise;Treating the form annotation as syntactic sugar
Section titled “Treating the form annotation as syntactic sugar”// WRONG — assumes @form(vec) is "just like" hand-writing the// methods over a literal F.22 heap. The lowering is different;// the storage layout is different; the perf characteristics// are different.The form is a contract, not sugar. It commits to a specific
lowering and performance shape. Code that depends on
implementation details of the literal F.22 heap-slot lowering
(e.g. memory addresses of individual cells across pushes) will
not behave the same under @form(vec).
Open questions
Section titled “Open questions”Spec-level questions not blocking the current @form(vec)
contract; will be answered as workloads surface demand.
- Iteration surface — SHIPPED 2026-07-02.
for x in v.items { ... }iterates a@form(vec)(fully inline buf walk: len + buf loaded once, one GEP + load per element, no C calls — vectorizes for scalar cells) andfor e in m.entries { ... }iterates a@form(hashmap)(cluster-aware slot-cursor walk vialotus_hashmap_iter_next: O(cap) for a full walk, where the index-basedkey_at/entry_atrescan from slot 0 per call — O(cap×len)). The loop variable is a per-iteration COPY for hashmap entries and a REFERENCE to the vec-owned cell for vec struct cells (scalars are copies by value). Mutating the form inside the body is unsupported (a grow rehashes/reallocs under the cursor).break/continuework. Ring-buffer iteration is still deferred (wrap-aware oldest-first walk). The pseudo-field is the form’s, whatever the capacity slot is named: a vec declaredheap cells of Intis still iterated asv.items, andv.cellsis refused with a hint naming.items(a slot is not a field; friction sweep 2026-10-12). - Bulk operations.
extend(other: @form(vec)),clear(),truncate(n: Int). Useful but not foundational. Add after the core methods land.
@form(hashmap)
Section titled “@form(hashmap)”A keyed associative store: each entry is a struct value S that
carries its own key as one of its fields. The Hale analogue of
Map<K, V> / std::unordered_map / Go map[K]V — but
intrusive: the value type S carries the key inside it rather
than the map storing separate (K, V) pairs. Shipped as the second
form in v1, following @form(vec).
Required capacity shape
Section titled “Required capacity shape”The locus MUST declare exactly one pool slot, with an
indexed_by <field> clause naming a field of the cell type to
serve as the key:
type CmdEntry { name: String; handler: Int;}@form(hashmap)locus CmdRegistry { capacity { pool entries of CmdEntry indexed_by name; }}Rules verified at typecheck:
- Exactly one slot. Zero slots, more than one slot, or a
heapslot is rejected. - The slot MUST be
pool. (Hashmap recycles entry cells as inserts / removes flow — thepooldiscipline.heapis the growable-contiguous shape covered by@form(vec).) - The slot MUST declare
indexed_by <field>. The named field must exist on the cell type. - The cell type MUST be a user-declared
typestruct. Primitives, enums, type aliases, and qualified paths are rejected — the substrate needs the cell’s field layout to GEP the key out at insert time, which only resolves cleanly for struct cells. - The cell type MAY NOT be a locus reference. Cells are data;
loci are managed entities. Storing an entity in a hashmap
means the synthesized
.get(key)materializes a stranger in the caller’s scope — the same antipattern thatspec/semantics.md § Locus method dispatchrejects at the user-declared layer. For keyed-children patterns, use the canonical alternatives:- Parent-child: declare
accept(c: ChildL)on the parent locus. If name-based lookup is needed, pair with a parallel@form(hashmap)of cell typetype Index { key: String; child_idx: Int; }. - Bus topic: publish commands keyed by name; the parent subscribes and dispatches to the right child.
- Delegation: collapse the per-child operation onto the
parent (
parent.inc_named(name)).
- Parent-child: declare
- The slot name is user-chosen and is not part of the contract.
The compiler finds the form’s pool slot by position, not by
name. Idiomatic spellings:
entries,bindings,routes. as_parent_foron the slot is rejected —@form(hashmap)owns its slot’s allocator, so the borrow mechanic from v1.x-4b doesn’t compose.@form(hashmap, ...)accepts one optional kwarg:sync = X(F.32-1; see “Cross-pool sync disciplines” below). All other kwargs are rejected.
Cross-pool sync disciplines
Section titled “Cross-pool sync disciplines”By default, @form(hashmap) is single-pool only — the
runtime has no synchronization on the hashmap entry points
(lotus_hashmap_set / _grow / etc), and cross-pool calls
into a @form(hashmap) receiver whose discipline does not
synchronize are typecheck-rejected (F.32-0). The opt-in path is
the sync = kwarg. A top-level map with no sync = argument
gets the discipline sync inference picks from the pools each of
its instances is called from (F.32-1∞), and sync = none keeps a map
unsynchronized whatever inference would pick; see semantics.md
§ “A form’s sync discipline” for the rule both follow.
| Annotation | Discipline | Status |
|---|---|---|
@form(hashmap) |
inferred: none (single-pool only) unless one of its instances is called from several pools | shipped |
@form(hashmap, sync = none) |
single-pool only; inference does not run | shipped |
@form(hashmap, sync = serialized) |
per-map pthread_mutex_t (F.32-1α) |
shipped |
@form(hashmap, sync = striped) |
cell-level CAS + per-map pthread_rwlock_t for grow + cache-padded cells (F.32-1β2-v2) |
shipped |
@form(hashmap, sync = lockfree) (optional cap = N initial-size hint) |
cell-level CAS, no rwlock or mutex on the steady-state path (F.32-1γ-v1); + remove via tombstones (F.32-1γ-v2 session 1); + lazy grow with brief writer/reader stall during migration (F.32-1γ-v2 session 3) |
shipped |
Discipline picker by workload:
-
Single-pool only (no cross-pool calls): plain
@form(hashmap). Densest layout, zero sync overhead; cross-pool calls are typecheck-rejected. -
Cross-pool, write-heavy, cap unknown / dynamic:
sync = serialized. Per-map mutex; writers serialize but the path is short. Beats striped on 2-core / cheap-payload.Reads are owned snapshots; replaced clones retire (2026-08-03, downstream handoff P1). Every read path on a serialized map whose cell owns anything (
get,entry_at,foriteration) copies it into the caller’s arena — its String and Bytes fields, and each nested record, tuple or out-of-line array with what that owns — inside the same critical section that read the cell. (Until the map retired Bytes and nested blocks it copied String fields only; the rest was never freed, so a raw pointer to it could not dangle.) The value a reader holds is therefore a plain owned value — it stays intact regardless of later writes, and a reader never observes a writer’s in-place mutations through a previously-read result. That ownership is what lets the map retire replaced clones exactly like async = nonemap (the anchor-retirement machinery inmemory.md): previously a serialized map never retired, because a raw cell pointer held by an off-pool reader could outlive the writer’s activation — the leak was the safety mechanism. A continuously-churned serialized map now holds steady-state RSS instead of growing per replace.stripedandlockfreemaps read the same way and retire the same way; what stands in for the mutex is the cell’s own claim (below).striped/lockfree: the read copies under the cell’s claim. These maps have no lock a reader and a writer of one cell both take, so a read of a cell that owns anything takes the cell’s claim — the sameCOMMITTED → CLAIMEDCAS a writer’s update takes — copies the value and everything it owns into the reader’s arena, and releases it (lotus_hashmap_cell_read). An update, and a lockfreeremove‘sCOMMITTED → TOMBSTONE, are CASes fromCOMMITTED, so neither starts while a copy runs, and once one succeeds no reader can reach the old cell’s blocks: the writer retires them inside its claim, deferred to its activation boundary as on a serialized map. The choice is deep reads, not an epoch: a reader on another pool never holds a view into a cell another pool may replace, so the runtime needs no quiescent-state tracking, at the price of a copy per read (a cell holding a String, a nested record and Bytes: ~13 → ~39 ns aget, serialized ~33 ns; a cell of scalars keeps the plain read). The claim also makes such a read whole — before, a read racing an update of the same key could copy half of each value. Readers of one key serialize on its claim; readers and writers of distinct keys never meet. A stripedremoveand grow run under the write lock, a lockfree grow after its drain, so neither races a read.key_atof a String-keyed retiring map (any synced discipline) copies the key out under the read’s lock as well, since a removed cell’s key is retired. Every such copy is a block of the reader’s own whatever arena it lands in: the walk uses the cell-owned clones, which copy a pointer already in the destination rather than pass it through. (The plain clones’ same-arena fast path handed a read with no method scratch — a closure assertion on the map locus, reading intoself.__arena— the cell’s own String and Bytes, which a same-key replace on another pool then retired and its flush reused under the reader.) A lockfree tombstone keeps its String key until the next grow’s migration — a probe that loaded the cell before theremovemay still be comparing it — and the migration, with no op in flight, retires it. (Before, neither discipline retired anything: between 1 000 and 100 000 overwriting sets of 4 keys a cell holding a String, a nested record and Bytes grew the map’s arena 196 608 → 16 056 320 bytes.) -
Cross-pool, read-heavy or per-op work is expensive:
sync = striped. Rwlock lets concurrent readers run in parallel; cache-padded cells avoid false-sharing between reader and writer cells. Slower than serialized on 2-core / cheap-payload writes (rwlock overhead > parallelism gain). -
Cross-pool, write-heavy, cap known approximately:
sync = lockfree(optionalcap = N). Pure CAS on the steady-state hot path — no kernel-mediated sync. Fastest of the four on theform_hashmap_false_sharingbench (~1.3× faster than α serialized at 2 cores). Trade-off: when the load factor exceeds 0.6, a grow event briefly stalls all lockfree ops (~ms for typical caps) while the migration runs. Steady state outside of grow remains fully lockfree.cap = Nis an optional initial-size hint (omitting it starts atLOTUS_HASHMAP_INITIAL_CAP = 8and grows on demand); supply 2-4× the expected peak so grows are rare.
The serialized discipline wraps every public entry point in
a per-map mutex. Throughput is bounded by lock contention
(~5-10 M ops/s on a 4-writer workload); correctness is
trivial. The map’s lotus_hashmap_t struct grows by 12 bytes
(int sync_mode + pointer mu); the pthread_mutex_t is
heap-allocated at init and destroyed at dissolve.
The striped discipline adds cache-line padding to the cell
stride (rounds up to LOTUS_CACHE_LINE, 64B default) and uses
a 3-state occupancy machine (EMPTY → CLAIMED → COMMITTED) with
__atomic_compare_exchange_n for slot claim. A
pthread_rwlock_t guards the grow path (set/get hold rdlock,
grow holds wrlock). On the 2-core / cheap-payload bench
striped measures ~1.87× slower than serialized — the rwlock
overhead per op (~150 ns) exceeds α’s mutex+memcpy (~90 ns)
by more than the 2-core parallelism gain compensates.
Striped’s win materializes on 4+ cores or with heavier per-op
work where the rwlock overhead amortizes.
The lockfree discipline (F.32-1γ) drops the rwlock
entirely. cap = N is an optional initial-size hint (was
required pre-γ-v2 session 3 before grow shipped; now grows
transparently when load factor crosses 0.6). Under γ-v1
remove was a no-op; under γ-v2 session 1
remove is supported via tombstones (4-state cell machine:
EMPTY → CLAIMED → COMMITTED → TOMBSTONE). Pure CAS on the
occupancy byte; no kernel-mediated synchronization on the
hot path. The form_hashmap_false_sharing bench measures
lockfree at ~1.30× faster than serialized and ~2.54× faster
than striped on the 2-pool concurrent-write workload. Cap
should be sized to 2-4× the peak expected entry count to keep
linear-probe latency bounded and avoid early grows; the
runtime rounds the user’s cap = N up to the next power of 2
(needed for the & mask probe). Omitting cap starts at
LOTUS_HASHMAP_INITIAL_CAP = 8 and grows on demand.
Tombstones in γ-v2 session 1 are not reclaimed in place —
the probe advances past them, but inserts always land in the
next EMPTY slot rather than reusing a TOMBSTONE. Session 3’s
grow path is where tombstones get compacted out: when the
load factor (live + tombstones / cap) crosses 0.6, the
table is rebuilt and the new table omits tombstones entirely.
It is rebuilt at double size when the live entries alone are
more than half that threshold (live / cap > 0.3), and at its
own size otherwise — a table filled by tombstones (a set/remove
churn) needs only the compaction. (Before, every rebuild
doubled, so a churn of set + remove grew the slots without
bound.) Consistency on remove follows the
lockfree model: a reader that observed COMMITTED before a
concurrent CAS to TOMBSTONE returns the (now-stale) value —
“the key was present at the moment we read.”
Grow under γ-v2 session 3 uses a simpler design than the full NBHM cooperative-helper migration: one writer wins a grow_phase CAS, spin-waits for in-flight ops to drain via a writers_in_flight counter, then runs the migration single- threaded (no SENTINEL state, no cooperative helping). All concurrent set/get/remove ops yield-spin during the migration window. The hot-path cost in steady state is one atomic load + branch-not-taken — measurably cheaper than the NBHM-style 5-state CAS-on-every-probe. The trade-off is tail latency on the writer that triggers grow (bounded by the migration’s O(cap) walk, ~ms for caps up to ~100k) and brief stalls on all concurrent ops during that window.
The OLD slots buffer is freed eagerly at the end of the
migration (γ-v2 session 4). The drain-wait already guarantees
no in-flight op holds a stale pointer to OLD when grow
completes, so the use-after-free risk that the handoff doc’s
QSBR design was meant to solve doesn’t exist in this single-
grower variant — QSBR epoch tracking would be redundant. RSS
post-warmup is bounded by the current table size plus the
brief migration-window peak (OLD + NEW both alive for the
~ms duration of lf_migrate).
Cross-pool method calls into a @form(hashmap, sync = ...)
receiver whose discipline is serialized, striped or
lockfree, written or inferred, are accepted without diagnostic
— the discipline carries the substrate’s safety contract; one
into a sync = none receiver is rejected. Inside a
single pool, all three sync modes pay only their respective
uncontended-fastpath costs (~30 ns for serialized,
~10 ns for lockfree’s CAS).
See notes/f32-cache-aware-delivery-plan.md § F.32-1 for the
per-discipline implementation strategy + trade-off analysis,
and spec/types.md § “Single-threaded-method invariant
(F.31)” for how cross-pool calls into form-bearing receivers
interact with the placement system.
The key type K is derived from the resolved type of the
indexed-by field. At v1, K must be Int or String. Other
field types parse and synthesize methods but reject at codegen
with a focused diagnostic (the runtime ABI’s key_type_tag
only enumerates these two).
Synthesized methods
Section titled “Synthesized methods”fn get(key: K) -> S fallible(KeyError)fn set(value: S) -> () # infalliblefn has(key: K) -> Bool # infalliblefn remove(key: K) -> () fallible(KeyError)fn len() -> Int # infalliblefn is_empty() -> Bool # infalliblefn key_at(i: Int) -> K fallible(IndexError)fn entry_at(i: Int) -> S fallible(IndexError)fn bump(key: K) -> () # infallible; S must be {key + Int counter}The fallible methods return the synthesized KeyError payload:
type KeyError { kind: String; # "missing_key" — only kind at v1}KeyError is injected into the bundle scope by the form
machinery alongside IndexError. The same type is shared across
all @form(hashmap) instantiations.
The key is not carried on the error because K varies per
hashmap. Users who want key context construct it through the
substitute motion (or fallback(err)), where err: KeyError
is in scope and any of the call’s local bindings — including
the key arg — are available.
fn set(value: S) -> ()Inserts or replaces. set(v) GEPs the indexed-by field from
v to derive the key, then writes the whole struct at the
hashed slot. If a previous entry shared the key, it is
overwritten (set is unconditional — no error on duplicate).
set is infallible. OOM during the doubling realloc is a
substrate-level concern routed through the closure-violation
channel, not a fallible(...) return. Same shape as
@form(vec)’s push.
fn get(key: K) -> S fallible(KeyError)Returns the entry whose indexed-by field equals key. If no
such entry exists, fails with KeyError { kind: "missing_key" }.
The entry returned is the caller’s value: writing into it never
changes the cell (§ Reading a cell).
let entry = registry.get(name) or raise;let entry = registry.get(name) or default;let entry = registry.get(name) or fallback(err);fn has(key: K) -> Booltrue iff an entry with this key is present. Equivalent to
“get(key) would succeed” but cheaper — no value copy.
remove
Section titled “remove”fn remove(key: K) -> () fallible(KeyError)Removes the entry whose indexed-by field equals key. If no
such entry exists, fails with KeyError { kind: "missing_key" }.
Idiomatic call shape:
registry.remove(name) or raise; # bubble on missingregistry.remove(name) or ignore(err); # swallow via Unit-returning handlerHale doesn’t surface () as a literal expression at v1, so
swallowing the error requires a Unit-returning handler call (or
guarding with has first):
fn ignore(_e: KeyError) { }// later:registry.remove(name) or ignore(err);remove does not shrink the underlying buffer; capacity does
not decrease. Buffer release happens at locus dissolution.
len and is_empty
Section titled “len and is_empty”fn len() -> Intfn is_empty() -> Boollen() returns the entry count; is_empty() is sugar for
len() == 0. Both infallible, O(1).
key_at and entry_at
Section titled “key_at and entry_at”fn key_at(i: Int) -> K fallible(IndexError)fn entry_at(i: Int) -> S fallible(IndexError)Hash-table-order iteration (added 2026-05-16). key_at(i)
returns the i-th present key; entry_at(i) returns the i-th
present full entry value. Order is hash-table order (deterministic
for a given table state, but insertion-sensitive — adding entries
between iterations may rearrange the sequence as the table
rehashes). For a populate-then-iterate pattern the snapshot
order is reproducible.
IndexError payload matches the @form(vec).get shape (kind: String, index: Int, len: Int). out_of_bounds is the only
kind produced; index < 0 or index >= len() triggers it.
Per-call cost is O(cap) — the substrate walks the slots array
counting occupied entries until reaching the i-th. A full sweep
is O(cap²). Fine at small/medium scale; agents iterating
100k+-entry tables should populate a parallel @form(vec)
during inserts instead.
Closes the wordfreq-corpus reinvention pattern where every program maintained a parallel keys vec to dodge the lack of iteration.
fn bump(key: K) -> ()Increment-or-init the Int counter field of the entry keyed by
key. Collapses the canonical 6-line pattern:
if m.has(k) { let prev = m.get(k) or raise; m.set(Entry { key: k, count: prev.count + 1 });} else { m.set(Entry { key: k, count: 1 });}into:
m.bump(k);Cell-shape requirement. The entry type S MUST have exactly
two fields: the indexed_by key field plus one Int field
(the counter). Any other shape (zero Int fields, two Int fields,
or extra non-Int fields) is a type error at the bump call
pointing at the manual pattern. The Int field’s name is detected,
not fixed — “count”, “n”, “freq”, “hits”, etc. all work.
bump is infallible. Generalizing to arbitrary update fns
(update_with(k, init, fn(S) -> S)) is the natural follow-up
once a use case surfaces.
Reading a cell
Section titled “Reading a cell”A value you read is yours. What get and entry_at return —
and what for e in m.entries binds — is the reader’s value: writing
into it, at any depth, never changes the cell. Only the map’s own
operations (set, remove, bump) change a cell (ruling of
2026-10-13; the same rule holds for @form(lru_cache)’s get).
The same rule holds for a local bound from any place, a field or an
element or a parameter (spec/types.md § A local bound from a place is
yours); a @form(vec) walk, for x in v.items, still visits the vec’s
cells in place.
How the read is made is the compiler’s business, and it keeps it
cheap. A synced map (serialized, striped, lockfree) reads a
deep copy (§ Cross-pool sync disciplines). A map that reads in place
(no sync =, or sync = none) and an LRU cache copy the cell’s bytes
into the reader’s frame — its top-level fields are the reader’s own —
and leave its nested blocks (a nested record, a tuple, an out-of-line
array, a payload enum, and the Strings and Bytes in them) shared with
the cell for as long as sharing cannot be seen. The first write into
a shared part copies that part into the reader’s frame first (copy on
write: a flag per local and part, so a part is copied once however
many writes follow), so let mut c = m.get(1) or R { }; c.inner.k = 7; leaves the cell’s inner as it was. Before this rule the write
went straight into the cell’s block: the cell changed with no set.
The get-modify-set idiom stays allocation-free. A write is left in the cell’s block — no copy — when it is part of a get-modify-set the compiler can prove no one observes:
let mut e = m.get(k) or Cell { id: k };e.big[j] = Level { p: x, q: x }; // lands in the cell's arraye.inner.n = e.inner.n + 1; // and in its nested recordm.set(e); // which the set keepsThat holds when the local is set back into the same map — m.set(e),
or a literal whose key field is the local’s (id: e.id) or the read’s
key (id: k) — on every path after the write (a statement of a block
the write is in, no return, break, fail or or raise between),
with nothing that may read the map between the write and the set
(another read of a map that may be this one, a call that may read a
form’s cell, transitively, a publish or yield whose handlers may),
and with the local not used after the set. A callee the compiler
cannot identify may read anything. Anything else copies:
| Shape | What is copied, and the reason reported |
|---|---|
| no set of the value follows the write | the part, at its first write: copy: the value is not set back into its map |
| the set is in a branch or loop the write is not, or a path leaves before it | the part: copy: the set is not on every path after the write |
| the map (or one that may be it) is read between the write and the set | the part: copy: the cell is read again before the set |
| the set’s literal keys another cell, or the key field was written | the part: copy: the set may key another cell |
| the value is used after the set (or after any write of its map) | the whole value, at its binding: copy: the value outlives the set |
| the value, or a part of it, is passed to a call that may write through its parameter | the whole value at its binding (an argument with no local, where it is evaluated): copy: the value is passed to a call that may write through its parameter |
a binding into the cell (a part bound to a local, a for element, a match payload, an entry of for e in m.entries) is written |
that binding, whole: copy: a binding into the cell is written |
A write to the value’s own top level (e.count = 3, e.name = s)
is never a copy: those fields were copied with the read.
Every copy is visible. Each one is a site of the allocation
survey (hale check --dump-alloc-summary prints cell-copy (<reason>)),
so it counts toward @budget(alloc_per_call = N) and the alloc
effect, and the hot-path lint reports it as it reports any other
allocation there: a warning in a loop or a bus handler, an error
anywhere in a @hot fn.
Lowering strategy
Section titled “Lowering strategy”@form(hashmap) lowers the pool slot to an inline six-field C
struct holding open-addressing hashtable state:
typedef struct { size_t cap; // power-of-two slot count size_t len; // live entry count size_t key_size; // sizeof(K), set at init size_t value_size; // sizeof(S), set at init int key_type_tag; // 0 = Int, 1 = String char *slots; // cap * (1 + key_size + value_size) bytes} lotus_hashmap_t;Each slot is 1 + key_size + value_size bytes laid out as
[occupied: u8][key: K][value: S]. occupied = 0 means empty;
the runtime uses backward-shift deletion (no tombstones) so
probes terminate as soon as an empty slot is seen.
- Initial cap: 8 slots, allocated at locus birth via
lotus_hashmap_init. Power of two so hash → index folds to a single& mask. - Growth policy: double
capwhen(len + 1) > 0.7 * cap. Rehash every live entry through the normalsetpath (the probe sequence changes with the new mask). - Shrink policy: none. Capacity is monotonic in v1.
- Hash functions: 64-bit Knuth multiplicative for Int keys
(
k * 0x9E3779B97F4A7C15), FNV-1a over the bytes for String keys. - Probing: linear with
& mask. Backward-shift deletion walks the cluster forward, shifting any entry whose natural position is “before” the freed slot. Cluster boundary is the first empty slot.
Key extraction at the codegen surface
Section titled “Key extraction at the codegen surface”At each set(value: S) call site, codegen GEPs the indexed-by
field offset on the value alloca to produce a pointer to the
key, then passes (slot_ptr, key_ptr, value_ptr) to
lotus_hashmap_set. The runtime memcpys key_size bytes from
key_ptr into the slot’s key region and value_size bytes
from value_ptr into the value region.
At get, has, remove sites, codegen lowers the key arg
into an alloca matching key_size and passes its address.
Arena ownership
Section titled “Arena ownership”@form(hashmap) is not a separate arena-allocated structure.
The lotus_hashmap_t struct lives inline in the locus’s struct
layout, the same way the literal F.22 pool-slot declaration
would. The slots buffer is malloc’d from the system
allocator, not from the locus’s arena — matching the existing
F.22 slot contract.
Dissolution: when the locus arena is destroyed, the hashmap’s
slots buffer is freed via lotus_hashmap_destroy in the F.22
dissolve cascade.
For elements of pointer-shaped types (String, Bytes) in the
cell struct, the hashmap stores the pointer by value; the
pointed-to bytes live in whatever arena they were allocated
from. Hashmap dissolution frees the slots buffer but does not
free the pointed-to bytes — those follow their owning arena’s
lifetime per the standard F.22 contract.
Cell reclamation
Section titled “Cell reclamation”set stores the cell’s own copy of everything the value points at
(its String and Bytes fields, and each nested record, tuple or fixed
array with what that owns — a record’s field of scalars is inline and
part of the cell itself, while an array inside a tuple or another
array is a block of its own, as for @form(vec)), in the map locus’s
arena. When a map replaces a cell (set of a present key) or
removes one, under any sync discipline, everything the old cell
owned is retired, at any depth, through the retire descriptor
@form(vec) uses; a block the new cell still reaches survives, and a
block reached twice is retired once. Retirement is deferred to the
owner’s activation boundary (anchor retirement, memory.md), so a
value a plain get handed out stays whole for the rest of the
activation; a synced map’s reads are owned copies (above — a
striped or lockfree map’s taken under the cell’s claim). A
churned map’s arena therefore stays flat. Before, only top-level
String fields retired: a replaced cell’s Bytes and its nested records
(their blocks and their own String and Bytes) stayed for good — a
cell holding a nested record carrying a String and Bytes grew the
arena from 64 KiB to 10.7 MiB over 100,000 sets.
From a map’s first replace, the Strings its cells store are field
blocks of their size class (memory.md § anchor retirement, the
field blocks of GH #1490: allocated at the class’s whole capacity and
recorded in the arena’s capacity table), and a replace or a remove
retires each one whole onto its class’s list, where the next String of
the class takes it. Until 2026-10-17 a cell’s Strings were retired at
their strlen+1 through the general freelist, which hands a block to a
request up to a quarter larger, so a block reused for a shorter value
lost its slack at its next retire, and a map whose rows were replaced
with Strings of varying length grew without bound (about 2.3 KB per
10,000 replaces, downstream handoff: too slowly for a process’s RSS to
show over 10,000 requests). A map that is only inserted into
allocates no table and its Strings keep their exact size. Bytes keep
the general freelist.
An enum field follows its representation. A unit-only enum
(enum { Bid, Ask }) is an inline tag, part of the cell like an
Int. An enum with a payload variant points at its own block (the tag
and the payload), which the cell owns and retires like a nested record:
the descriptor carries one sub-descriptor per variant and the walk
takes the one the block’s tag names, so a replaced Named(String)
frees its String and a replaced Num(Int) only its block; a
@form(vec) element’s enum is retired the same way. (Before, a
hashmap set of a cell with a unit-only enum field was an internal
compiler error, a replaced payload enum’s block was never retired —
at least 16 bytes a replace, in a map or a vec — and a vec get of
an element holding one always failed.) An enum in another enum’s
payload (Outer::X(Inner::A(s))) is a block the outer enum’s block
points at, copied and retired the same way at any depth. (Before, a
cell holding one failed to build: the copy’s variant switch joined
its result from each variant’s first block, and the nested enum’s own
switch had added blocks after it.)
A nested block in the cell belongs to that cell alone. A value that
carries a nested block from the same key’s current cell, at the same
field (let e = m.get(k); m.set(Cell { k: k, big: e.big, ... })),
keeps it, so a get-modify-set allocates nothing for it; any other
nested block — one read from another key’s cell, one shared with a
field — is copied. A write the reader makes into such a block before
the set lands in the cell’s block only when no one can see it (§
Reading a cell); otherwise the part is copied at the write. (Before, a nested block already in the map’s arena
was stored as-is whatever held it, so two cells could share one; and a
cell whose only heap part was a nested record of scalars stored the
pointer it was handed, into the setting method’s scratch, which a
later get read after the method had returned.)
Complexity
Section titled “Complexity”| Operation | Expected | Worst case |
|---|---|---|
set (no resize) |
O(1) | O(N) on probe cluster |
set (with resize) |
O(N) amortized over inserts | O(N) per resize |
get / has |
O(1) expected | O(N) on probe cluster |
remove |
O(1) expected | O(N) on shift |
len / is_empty |
O(1) | O(1) |
Load factor stays ≤ 0.7 by construction. Hash quality for Int keys (Knuth multiplicative) handles dense sequences such as consecutive IDs without all colliding on slot 0.
Interaction with the locus tower
Section titled “Interaction with the locus tower”A @form(hashmap) locus is a locus in every other respect — it
can have params, lifecycle bodies (birth / run / drain /
dissolve), closure invariants, on_failure routing, bus
membership, and projection by perspective declarations. The
form annotation replaces the literal F.22 pool-slot lowering
and synthesizes the six methods; it does not replace any other
locus mechanic.
Bench protocol (future FORM-N gate)
Section titled “Bench protocol (future FORM-N gate)”@form(hashmap) gets a bench family parallel to @form(vec)’s
three bands once hale-lang/bench’s micro/form_hashmap_*
is wired up:
- Tight-loop primitive (band (a)). A
form_hashmap_setmicrobench — 200ksetcalls on a hashmap of struct cells, compared against an equivalent hand-written C program usingmalloc+ open-addressing tables of the same shape. Target: 10% of the C baseline. Expected to trackform_vec_push’s shape closely since the subregion-elision work applies equally. - Per-op fallible (band (c)). A
form_hashmap_getmicrobench — 200kget(k) or raisecalls. Advisory; the same C-function-call-boundary residualform_vec_getshows will apply here. Closing it is the same work item. - App bench. A representative app rewritten to use
@form(hashmap)where it currently does explicit registry walks; before / after comparison.
Not gated on FORM-4 shipping (FORM-4 was held to the band (a)
form_vec_push win for license); ships as a separate
milestone after a hashmap consumer surfaces concrete demand.
Anti-patterns
Section titled “Anti-patterns”Treating set as keyed insert
Section titled “Treating set as keyed insert”// WRONG — set takes the whole value, not (key, value).registry.set("foo", entry); // type error: too many args// RIGHT — value carries its key as a field; substrate extracts.registry.set(CmdEntry { name: "foo", handler: 1 });The intrusive shape means the type system catches this for you
(set is synthesized with the single-arg signature set(value: S) -> ()), but the conceptual reflex from HashMap<K, V>
shaped languages is worth flagging.
Ignoring the fallible return
Section titled “Ignoring the fallible return”// WRONG — get and remove return fallible(KeyError).let v = registry.get(name); // compile error: the call says nothing about its failureregistry.remove(name); // compile error: the call says nothing about its failure// RIGHT — address the error via one of the three motions.let v = registry.get(name) or raise;registry.remove(name) or ();Mutating the indexed-by field after set
Section titled “Mutating the indexed-by field after set”The intrusive shape means the key is the field. If user code
keeps a reference to the value and mutates the indexed-by
field, the hashmap’s invariant breaks (the cell sits in the
slot keyed by its old key, but get now looks up by its
new key). The v1 surface doesn’t expose stored cells by
reference, so this isn’t reachable from user code today.
Future iteration APIs that surface entry references will need
to gate against indexed-by-field mutation.
Forward-looking / deferred items for this area now live in the decision log — see
decisions.md§ Deferred & future work.
Required capacity shape
Section titled “Required capacity shape”The locus MUST declare exactly one pool slot. The cell type is
the element type T; the capacity comes from the annotation arg
cap = N.
@form(ring_buffer, cap = 64)locus RecentCmds { capacity { pool history of CmdEntry; }}Rules verified at typecheck:
- Exactly one slot. Zero slots, more than one slot, or a
heapslot is rejected. (heapis the growable-contiguous shape covered by@form(vec); ring buffer recycles fixed-capacity cells, which is thepooldiscipline.) - The slot MUST NOT declare
as_parent_fororindexed_by— those clauses belong to other forms. @form(ring_buffer, cap = N)requirescap, must be a positive integer literal. v1 doesn’t const-evaluate expressions for form args.- The cell type T may be a primitive, a user-defined
type, or a generic parameter. It MAY NOT be a locus reference — same restriction as the other forms.
Synthesized methods
Section titled “Synthesized methods”fn push(x: T) -> Bool # false when fullfn pop() -> T fallible(EmptyError)fn len() -> Int # infalliblefn is_full() -> Bool # infallibleThe fallible pop returns the synthesized EmptyError payload:
type EmptyError { kind: String; # "empty" — only kind at v1}EmptyError is injected alongside IndexError and KeyError by
the form machinery; user-declared EmptyError wins per the
existing idempotent-injection contract.
fn push(x: T) -> BoolAppends x after the last element. Returns true on success,
false when the buffer is at capacity. Callers decide
drop-vs-backpressure semantics by inspecting the result:
let accepted = recent.push(entry);if !accepted { // backpressure: surface to caller, or drop, or evict-oldest // via pop()+push() in a separate path.}push is intentionally Bool-returning rather than
fallible(FullError). The full-buffer state is a normal
operational condition (the caller chose a bounded capacity), not
a substrate failure — surfacing it as a Bool keeps the call
shape ergonomic and avoids forcing every caller through or.
This is the one place in the form library where infallible-but-
returning-a-status is the right idiom; vec’s push is truly
infallible (OOM routes through the closure-violation channel),
and hashmap’s set is unconditional (replace-on-collision). The
ring buffer’s fixed cap makes “refused” a user-observable state.
fn pop() -> T fallible(EmptyError)Removes and returns the oldest element (FIFO — the one inserted
earliest among those still present). Fails with
EmptyError { kind: "empty" } when the buffer is empty.
let cmd = recent.pop() or raise; # bubble on emptylet cmd = recent.pop() or default_cmd; # substitutelet cmd = recent.pop() or fallback(err);len and is_full
Section titled “len and is_full”fn len() -> Intfn is_full() -> Boollen() is the current element count, in 0..=cap. is_full()
is sugar for len() == cap. Both infallible, O(1).
There is intentionally no is_empty() synthesized method on
@form(ring_buffer) — pop is the natural empty-detection
surface (the fallible return signals empty directly to the
caller addressing the error). Adding is_empty() would create
two redundant ways to ask the same question; defer until a
workload demonstrates a real need.
Lowering strategy
Section titled “Lowering strategy”@form(ring_buffer) lowers the pool slot to an inline five-field
C struct holding head/tail and a pre-allocated backing buffer:
typedef struct { size_t cap; // fixed at init; never changes size_t head; // index of oldest element (next pop) size_t len; // current count, 0..=cap size_t elem_size; // bytes per element char *buf; // cap * elem_size bytes} lotus_ring_buffer_t;- Birth.
lotus_ring_buffer_initmallocscap * elem_sizebytes and pins them for the locus’s lifetime. The init takescapandelem_sizeas args;capflows in from the form annotation,elem_sizefrom the cell type’s LLVMsize_of. - Push.
lotus_ring_buffer_pushcheckslen == cap; if so returns 0. Otherwise computes the wrap index as(head + len) % cap, memcpyselem_sizebytes from the caller-provided source into the slot, incrementslen, returns 1. - Pop.
lotus_ring_buffer_popcheckslen == 0; if so returns 0. Otherwise memcpys frombuf + head * elem_sizeinto the out-pointer, advancesheadmodulo cap, decrementslen, returns 1. - No growth. Once init runs, the backing buffer is fixed size. Push at capacity refuses; the spec contract is “fixed capacity” not “grows on demand.”
- Dissolution.
lotus_ring_buffer_destroyfrees the backing buffer at locus arena destroy.
Arena ownership
Section titled “Arena ownership”Same as @form(vec) and @form(hashmap): the
lotus_ring_buffer_t struct lives inline in the locus struct
layout; the backing buf is malloc’d from the system
allocator, not from the locus’s arena. Dissolution frees
buf via the F.22 dissolve cascade.
For pointer-shaped element types (String, Bytes), the ring
buffer stores the pointer by value; the pointed-to bytes live
in their owning arena per the standard F.22 contract.
Complexity
Section titled “Complexity”| Operation | Cost |
|---|---|
push (not full) |
O(1) |
push (full → refuse) |
O(1) |
pop (not empty) |
O(1) |
pop (empty → fail) |
O(1) |
len / is_full |
O(1) |
All operations are constant-time and allocation-free after locus birth. No realloc, no rehash, no compaction.
Interaction with the locus tower
Section titled “Interaction with the locus tower”A @form(ring_buffer) locus is a locus in every other respect.
Same orthogonality as the other forms — params, lifecycle
bodies, closure invariants, on_failure routing, bus
membership, perspective projection all compose unchanged.
Anti-patterns
Section titled “Anti-patterns”Forgetting to address pop’s fallible
Section titled “Forgetting to address pop’s fallible”// WRONG — pop returns fallible(EmptyError); the bare let drops it.let v = recent.pop(); // compile error: the call says nothing about its failure// RIGHT — address with one of the three motions.let v = recent.pop() or raise;Treating push as fallible
Section titled “Treating push as fallible”// WRONG — push returns Bool, not fallible(FullError).let v = recent.push(x) or raise; // type error: push isn't fallibleThe full-buffer state surfaces as a Bool return, not a fallible payload. Inspect the Bool directly.
Resizing expectations
Section titled “Resizing expectations”The ring buffer’s cap is fixed at the annotation site. There is
no grow or shrink_to_fit. Apps that need a growable bounded
buffer should pick a generous cap up front, or use
@form(vec) if growth is the right semantic.
Forward-looking / deferred items for this area now live in the decision log — see
decisions.md§ Deferred & future work.
Required capacity shape
Section titled “Required capacity shape”The locus MUST declare exactly one pool slot with an
indexed_by <fieldname> clause over a user-declared struct cell —
the same key surface as @form(hashmap) — AND the annotation arg
cap = N. lru_cache is the one form that needs BOTH a key and a
cap.
type SessionEntry { id: Int; token: String; }
@form(lru_cache, cap = 1000)locus SessionCache { capacity { pool sessions of SessionEntry indexed_by id; }}Rules verified at typecheck:
- Exactly one slot, of kind
pool. Zero slots, more than one slot, or aheapslot is rejected. - The slot MUST declare
indexed_by <fieldname>; the named field must exist on the cell struct and becomes the cache key typeK(Int or String at v1). The cell struct is the value typeS. cap = Nis required and must be a positive integer literal. v1 doesn’t const-evaluate expressions for form args.- The slot MUST NOT declare
as_parent_for— form-lowered slots own their own allocator. - The cell type MAY NOT be a locus reference — same restriction as the other forms.
Synthesized methods
Section titled “Synthesized methods”fn put(x: S) -> () # infallible; silent LRU evictfn get(k: K) -> S fallible(KeyError) # lookup + recency touchfn contains(k: K) -> Bool # membership, NO recency touchfn len() -> Int # infallible; count <= capfn remove(k: K) -> S fallible(KeyError) # take the entry outget’s and remove’s miss payload is the synthesized KeyError
(shared with @form(hashmap)); no new payload type is introduced.
The locus may declare one more method the form calls itself, the
eviction hook fn on_evict(k: K, v: S) (see below).
fn put(x: S) -> ()Insert or update by the indexed_by key extracted from x. If
the key is already present, its value is overwritten and its
recency is refreshed. If the key is new and the cache is at
capacity, the least-recently-used entry is silently evicted first.
put is infallible — over-cap is a normal, silent operation
(eviction), not a failure. This matches the “never flagged /
bounded” contract: there is no FullError, unlike
@form(ring_buffer).push which returns a Bool.
Both insert and update mark the touched entry as recently used.
fn get(k: K) -> S fallible(KeyError)Looks up the value for key k. On a hit it returns the value
and marks the entry as recently used (a recency touch) — this
is what makes the policy LRU rather than FIFO: a get on an entry
saves it from an eviction that a purely-oldest-inserted policy
would take. On a miss it raises KeyError, addressed at the
call site’s or clause.
The value returned is the reader’s: writing into it never changes the
cached entry, and a write into one of its nested parts copies that
part first unless it is part of a get-modify-set (put back), as for
a map that reads in place (@form(hashmap) § Reading a cell).
contains
Section titled “contains”fn contains(k: K) -> BoolMembership test. Returns true if k is present. Unlike get,
contains does NOT touch recency — a contains on an entry
leaves it exactly as recently-used as it was. This distinction is
observable: after contains(k), the entry k remains eligible
for eviction if it was the LRU entry.
fn len() -> IntCurrent live entry count, always <= cap. Infallible.
remove
Section titled “remove”fn remove(k: K) -> S fallible(KeyError)Takes the entry keyed k out of the cache and returns its value —
the CALLER’s own copy, allocated in the caller’s current arena, as a
@form(vec) pop hands one back. On a miss it raises KeyError. The
recency of the other entries is untouched, and on_evict is not
called: the caller now holds the value and whatever it owns (an fd it
carries, say), which is the take-semantics a pool of connections needs.
The eviction hook — on_evict
Section titled “The eviction hook — on_evict”type Conn { host: String; fd: Int; }
@form(lru_cache, cap = 4)locus Conns { capacity { pool conns of Conn indexed_by host; } fn on_evict(k: String, v: Conn) { let _ = std::io::tcp::close_fd(v.fd); }}A @form(lru_cache) locus may declare fn on_evict(k: K, v: S) — no
return value, not fallible; any other signature under that name is a
type error. When a put of a new key evicts the least-recently-used
entry, the cache calls on_evict with the evicted entry’s key and
value, then reclaims the cell — so a cell that owns a resource outside
the arena (a socket, a file) can release it. Only capacity eviction
calls it: a put over a present key replaces the value without the
hook (the caller replacing a value owns what the old one held — remove
it first to release it), and remove hands the entry to the caller.
At dissolve. When the locus dissolves, after its dissolve() body
(which still sees every entry), the cache calls on_evict once for each
entry it still holds, in LRU order: the least recently used first, the
order capacity eviction would have taken them. Each entry is taken out
of the table before the hook sees it, so len() in the hook counts the
entries still to come. This holds on every teardown path: a cache that
goes out of scope, one a locus holds as a param (dissolved with its
holder), and the main locus’s own. A cell that owns a resource outside
the arena is therefore released whether it leaves by eviction or with
the cache; the cell’s blocks go with the locus’s arena.
The hook runs inside the put that evicted, so the call graph has an
edge from every put on the locus to on_evict: an effect the hook
performs (syscall for a close_fd) is one the put’s caller reaches.
The hook must not put into its own cache: it runs inside a put and
while the cache dissolves, and a put there would evict again under it.
The check refuses a put on the cache’s own type that on_evict
reaches, directly or through the fns it calls, at that put, with the
chain: “`P.on_evict` puts into its own cache (P::on_evict -> P::keep
[P::put]): …”. A put into another cache is the hook’s to make.
Cell reclamation
Section titled “Cell reclamation”put stores the cell’s own copy of everything the value points at,
in the cache locus’s arena, as @form(hashmap)’s set does (§ Cell
reclamation there): a nested block read out of another cell is copied,
never shared. A cell the cache drops — replaced by a put of its key,
evicted over cap (after on_evict), or taken by remove — has
everything it owned retired through the retire descriptor, deferred to
the owner’s activation boundary, so a value a get handed out stays
whole for the rest of the activation; a block a replacing value still
reaches survives. A churned cache’s arena therefore stays flat. (Before,
the cache reclaimed nothing: each put of a new value grew its arena for
the locus’s lifetime.)
Lowering strategy
Section titled “Lowering strategy”@form(lru_cache) lowers the pool slot to an inline header struct
plus a single heap-allocated open-addressed table, managed by the
lotus_lru_* C runtime:
struct lotus_lru_t { size_t cap; // fixed live-entry cap (never grows) size_t len; // current live entries (<= cap) size_t key_size; size_t value_size; int key_type_tag; // Int / String key ABI (shared w/ hashmap) uint64_t tick; // monotonic access counter size_t table_cap; // power-of-two slot count (>= 2*cap) char *slots; // table_cap * (occupied + tick + key + value)}It deliberately does not reuse the lotus_hashmap_* family:
that family auto-grows and has no recency notion, both of which
break the cap invariant. The table is sized to the next power of
two >= 2*cap (load factor <= 0.5, so a probe always meets an
empty terminator). Each slot carries an access tick; eviction
removes the occupied slot with the minimum tick (the LRU entry)
via backward-shift compaction (no tombstones), keeping the table
reusable under unbounded insert/evict churn. Ticks are globally
unique per access, so the LRU entry is unambiguous.
The backing table is pre-allocated at locus birth
(lotus_lru_init) and freed at dissolve (lotus_lru_free); the
inline header lives in the locus struct and dies with the arena —
the same inline-header / heap-buffer split as @form(vec),
@form(hashmap), and @form(ring_buffer).
Forward-looking / deferred items for this area now live in the decision log — see
decisions.md§ Deferred & future work.
@form(heap)
Section titled “@form(heap)”A priority queue: a binary heap of T records, ordered by one of
their fields, smallest or largest first. The Hale analogue of
BinaryHeap<T> / std::priority_queue / Go’s container/heap
(branch-and-bound searches, Dijkstra, timer wheels; friction sweep
2026-10-12 — they hand-rolled one over @form(vec)).
Required capacity shape
Section titled “Required capacity shape”type Job { name: String; priority: Int; }
@form(heap)locus Jobs { capacity { heap jobs of Job ordered_by min priority; }}Rules verified at typecheck:
- Exactly one slot, of kind
heap(the growable contiguous discipline; apoolslot is rejected). - The slot MUST declare
ordered_by min <field>orordered_by max <field>.minputs the smallest key at the root (it comes out first),maxthe largest. There is no default: the direction is stated where the heap is declared.ordered_byis a contextual word;minandmaxare read only in that position. - The cell type MUST be a user-declared record, and the named field
one of its fields, of type
Int,Float,String,TimeorDuration. AStringkey orders by its bytes (strcmp); aFloatNaNis never ordered before anything. - No annotation arguments, no
indexed_by, noas_parent_for. The cell type may not be a locus.
Synthesized methods
Section titled “Synthesized methods”fn push(x: T) -> () # infalliblefn pop() -> T fallible(EmptyError) # the root, removedfn peek() -> T fallible(EmptyError) # the root, left in placefn len() -> Int # infalliblefn is_empty() -> Bool # infalliblefn clear() -> () # infallibleEmptyError is the payload @form(ring_buffer)’s pop fails with
(kind: "empty").
pushaddsxand restores the heap order: O(log n), amortized over the buffer’s doubling.popremoves the root — the element with the smallest key undermin, the largest undermax— and returns it: O(log n). Among equal keys the order is unspecified (a binary heap is not stable; carry a sequence number in the key’s tie-breaker field if FIFO among equals matters).peekreturns the root without removing it: O(1).cleardrops every element, as@form(vec)’scleardoes.
Like a vec’s, pop and peek return the CALLER’s own copy of the
element (in the caller’s current arena), and the heap owns what it
stores (push stores its own copy). An element pop removes, and
every element clear drops, is reclaimed exactly as @form(vec)
reclaims a popped element — its block, its String and Bytes fields,
its nested records with what they own — and the buffer keeps its
capacity, so a queue held at a steady depth keeps a flat arena.
There is no indexed access and no iteration: the elements past the
root are in heap order, which is not an order a caller can use.
Drain with pop for the sorted sequence.
Lowering strategy
Section titled “Lowering strategy”A @form(heap) slot is a @form(vec) slot — the same inline
{ cap, len, buf } struct, the same pointer-storage elements and
growth, the same dissolve — with the heap order kept over it: after
the vec’s push, lotus_heap_sift_up moves the new element up from
the end; before the vec’s pop, lotus_heap_pop_prep moves the root
to the end and sifts the old last element down from the root; peek
is the vec’s get(0). The key’s byte offset in the record, its kind
(an i64 for Int / Time / Duration, a double, a String)
and the direction are compile-time constants at each call.
Interaction with the locus tower
Section titled “Interaction with the locus tower”As for the other forms: a @form(heap) locus can carry params,
lifecycle bodies, closures, failure routing, bus membership and
perspectives, and user methods beside the synthesized set (a user
method may not take a synthesized method’s name).