Lexical structure

Reference material, synced from the compiler repo’s spec/. The guide is the gentler path in.

This document specifies the lexical layer of Hale: the set of tokens the lexer produces and feeds to the parser. The formal grammar in grammar.ebnf is defined over these tokens.

Source files are UTF-8 encoded. File extension: .hl.

Outside string literals and comments, only ASCII characters are permitted. The framework’s mathematical primitives are spelled out as ASCII names; the renderer can produce Unicode for human display, but the source is ASCII-only. This commitment serves the agent-first authorship principle (no symbol-input friction) and simplifies tooling.

• Spaces, tabs, carriage returns, and newlines are whitespace.
• Whitespace separates tokens but is otherwise insignificant.
• No off-side rule (no Python-style indentation).
• Newlines are not statement terminators; semicolons are required.

• Line comment: // ... to end of line.
• Block comment: /* ... */. Block comments do not nest in v0.
• Doc comment: /// lines directly above a declaration attach to it (decorator lines like @hot may sit between). Rendered by hale doc into the seed’s API reference (spec/testing.md). Tooling recovers doc text positionally from the source — the lexer itself skips all comments. /** */ stays reserved for a future block form; hale doc v1 reads /// only.

• Match: [a-zA-Z_][a-zA-Z0-9_]*.
• Case conventions are enforced by lints, not the lexer:
  • PascalCase for type names, locus names, perspective names.
  • snake_case for variables, fields, function names.
  • SCREAMING_SNAKE_CASE for constants.
• Reserved words (see below) are not legal identifiers.

locus           perspective     type            const
fn              import          export          module
interface

interface (F.20) declares a structural interface — a named set of method signatures. Any locus whose method set is a superset structurally satisfies the interface (no impl I for L declaration). Phase A (typecheck) and Phase B (codegen vtable dispatch) both shipped 2026-05-11; interface values are usable as fn params, fn returns, locus param/field values, and @form(vec) cell elements. Interface elements in tuples / fixed arrays remain gated on the broader composite-construction coercion design (same gap as tuple-of-LocusRef escape).

params          contract        bus             capacity
as_parent_for   indexed_by

capacity introduces an F.22 capacity { ... } block carrying zero or more pool X of T; / heap Y of T; slot declarations. as_parent_for and indexed_by are hard keywords used only in slot-clause position inside a capacity block: pool P of T as_parent_for ChildL; (v1.x-4 — share the slot’s allocator with a child locus’s same-named slot at accept time) and pool P of T indexed_by field; (v1.x-FORM-4 — names the hashmap key field on a @form(hashmap) cell type). The slot-kind words pool and heap are contextual idents — they lex as ordinary Idents and the parser recognizes them only in slot-decl head position inside a capacity block. So fn pool_alloc(...), let heap = ..., and type Heap { ... } all stay admissible outside capacity blocks. Same F.10-style narrowing the closure-keyword family uses for approx / within.

birth           accept          release         run
drain           dissolve        on_failure

bulk            harmonic        resolution

mode is a contextual keyword: recognized only at locus-member position (as the leading token of a mode_decl production in grammar.ebnf § 11). Outside that position it lexes as an ordinary Ident, so cam.mode: Int and similar param/field uses are admissible — relevant for raylib-style bindings where mode is a natural API name. Same F.10-style narrowing as bindings, birth_check, pool, heap. bulk / harmonic / resolution remain reserved (they sit in member-name position via try_keyword_as_name).

projection      rich            chunked         recognition

placement       cooperative     pinned          pool        core        cores
topology        node            l3              reserve     replicas

placement introduces the placement { } block on main locus (F.31); topology introduces the topology { } block (topology Phase 1b). cooperative / pinned are placement-spec keywords inside placement { }. pool, core, and cores are contextual idents — recognized only as kwarg names inside cooperative(pool = X) / pinned(core = N) / pinned(cores = A..B | A..=B | {a, b, c}) placement specs (topology Phase 1a). Phase 1b adds pinned(node = N) / pinned(l3 = name), plus the topology, node, l3, and reserve block keywords. Phase 1c adds replicas (a second pinned(...) kwarg, e.g. pinned(cores = A..B, replicas = K)). All are contextual idents — outside their positions they lex as ordinary Idents and can name fns / vars / fields (same F.10-style narrowing the closure / mode keyword families use). Note L3-domain names go through the identifier rule, so a hard keyword (e.g. bulk) can’t name a domain — use a plain identifier. cores range bounds and set elements are integer literals; the range reuses the expression tokens .. (exclusive) / ..= (inclusive).

The pre-F.31 schedule keyword is gone — the : schedule cooperative | pinned per-locus annotation no longer exists. Placement is a main-only deployment seam (bindings’s sibling), not a per-locus annotation.

closure         epoch           persists_through    resets_on

approx and within are contextual keywords, recognized only inside a closure { ... } block body (and only in the specific positions the closure-assertion grammar admits them). They lex as ordinary Idents elsewhere, so fn approx(...) and let within = ... are admissible outside closure bodies. Same F.10-style narrowing the mode-keyword family uses post-dot. Shipped 2026-05-11; resolves notes/hale-friction.md 2026-05-10 closure-keyword-shadows-helper-ident.

captures and inline (v1.x-VIOLATE, F.27) are contextual keywords. captures is recognized only as a clause keyword inside closure { ... } body, in the captures: f1, f2 ... ; production. inline is recognized only as an epoch_spec variant (immediately after epoch in a closure clause). Both lex as ordinary Idents elsewhere — so let captures = ... and fn inline(...) stay admissible outside closure bodies. Same F.10-style narrowing.

restart         restart_in_place    quarantine      reorganize
bubble

violate (v1.x-VIOLATE, F.27) is a contextual keyword: recognized only as the leading token of a statement inside a locus method body (the violate_stmt production in grammar.ebnf § 14). Outside that position it lexes as an ordinary Ident, so let violate = ... and fn violate(...) stay admissible. The optional trailing with in violate NAME with EXPR; is the previously-reserved keyword with (no longer reserved-for-future-use); recognized only as the separator before the payload expression in this production.

expose          consume         inferred

subscribe       publish         on              of

stable_when     serialize_as    serves          reperspective

serves (Phase 2a) is a contextual keyword — recognized only in a locus header’s post-: list, as the serves P conformance clause (locus RouterV1 : serves Router). reperspective (Phase 2b) is a contextual keyword recognized only as a statement head followed by self (reperspective self.<field> as <Impl>;). Outside those positions both lex as ordinary Idents, so fn serves(...) / let reperspective = ... stay admissible. Same F.10-style narrowing the placement / closure keyword families use.

perspective (a hard declaration keyword) doubles as a type constructor: perspective(P) in type position is a handle to the contract P. The parser recognizes it by the perspective token followed by (.

let             mut             if              else
match           for             in              while
return          break           continue        true
false           nil             tier            self

Bindings are immutable by default. let mut x = ... declares a mutable binding; reassignment via x = ... is permitted. Without mut, reassignment is a compile-time error.

self is meaningful only inside a lifecycle block, mode block, or closure block. It refers to the enclosing locus’s own params and contract-exposed state. Outside such a block, self is a parse error.

Int             Uint            Float           Decimal
String          Bool            Time            Duration
Bytes           BytesView       StringView      BytesMut

BytesView / StringView (F.30) are non-owning views over a BytesBuilder’s buffer; BytesMut (#3) is a raw {ptr, len} writable/readable window (a Topic.write ring slot or a MirrorRing window). See spec/types.md.

PascalCase per the type-name convention. The lexer emits these as Ident tokens; the parser recognizes them by name in type position only. In expression / namespace position, these names are unreserved — time::sleep is a regular path because time (lowercase) is an ordinary identifier. This eliminates the lexical collision between primitive type names and stdlib namespace names that would otherwise occur.

Shadowing a predefined type name with a user-defined type (type Int = ...) is permitted by the grammar but produces a compiler warning.

trait           impl            async           await
macro

(with is no longer in this list — v1.x-VIOLATE recognizes it as a contextual keyword inside the violate_stmt production. where is no longer in this list either — Form K recognizes it as the suffix keyword on binding_entry carrying operational constraints.)

yield is a real statement keyword (m26b) — explicit cooperative yield point; lowers to a bus-queue drain in codegen. Listed under cooperative-scheduler keywords.

terminate is a statement keyword — ends the current locus’s lifecycle from inside one of its own methods (the locus analogue of return). Only valid inside a locus method body. See spec/semantics.md § “terminate”.

yield
terminate

fallible        fail            or              raise

All four are contextual keywords, recognized only in the positions described below. Outside those positions they lex as ordinary Ident tokens, so existing code that uses let fail = ..., fn or_else(...), or let raise = ... stays admissible.

• fallible — recognized only immediately after a function declaration’s return type and before its body, in the fallible_marker = "fallible" "(" type_expr ")" production. Marks the function as one whose call sites MUST address the error. Permitted at declaration sites: free fns + stdlib- synthesized methods over @form(...) containers; rejected at typecheck on user-declared locus methods per the two-channel rule (see spec/semantics.md § “Fallible call semantics” § “Where each channel lives”).
• fail — recognized only as the leading token of a statement inside a fallible-fn body. Exits via the error path, attaching the expression as the typed payload. Symmetric to return.
• or — recognized as a postfix on any expression of fallible type (the or_clause production). Four RHS forms: or raise (propagate one frame up the call stack), or discard (swallow the error and substitute Unit — rejected at typecheck on calls whose success type is non-Unit; sugar for the old or noop(err)), or fail <expr> (B3 / G6 — diverge like raise but with a fresh payload of the enclosing fallible fn’s declared error type), or <expression> (substitute), or <handler-call> (hand off).
• raise — recognized only as the immediate RHS of or. Diverges the expression by re-entering the fallible-return shape of the enclosing fallible(E) fn, with the payload carried into the enclosing fn’s error sret slot. Past every enclosing fallible frame — at the implicit main locus’s root boundary — the runtime panics via lotus_root_panic. The value-error channel is orthogonal to the closure- violation channel; or raise does not enter the bubble / on_failure machinery by default. See spec/semantics.md § “Fallible call semantics” for the full two-channel semantics.

The implicit binding err is in scope on the RHS of an or clause and resolves to the fallible call’s payload value (typed as the fn’s fallible(T) marker). This is a typecheck rule, not a lexer rule — err lexes as an ordinary identifier everywhere; the typechecker just introduces it as a binding in the or RHS scope.

Rule of thumb for use:

• A stdlib fn whose failure carries useful diagnostic context returns fallible(T) with a named payload type T.
• A stdlib fn whose “failure” is a benign not-found case (and whose successful value has a natural default like 0 or “”) uses the sentinel-with-discriminator idiom instead (parse_int / can_parse_int). Not every stdlib fn is fallible — the marker is reserved for true error paths.

+   -   *   /   %

==  !=  <   >   <=  >=

&&  ||  !

&   |   ^   <<  >>  ~

=   +=  -=  *=  /=  %=  &=  |=  ^=

~~          equivalent to `approx`; tests value approximate-equal
            within a stated tolerance band. Used in closure tests.

<-          Send a typed message on a declared bus subject:
            `"subject" <- value;`. The left side names a subject
            declared in the locus's `bus { publish ... }`; the
            right side is the typed payload. Same Erlang-shape
            as `Pid ! Msg`; one-direction (subscribe is
            declarative, not an operator).

.   ::  (   )   [   ]

<   >   ->  =>  :   ::

(Note: < and > are overloaded between comparison and generic arguments. The parser disambiguates contextually.)

;   ,   {   }

@

@ introduces a decorator-shaped annotation that decorates the declaration immediately following it. v1 recognizes one such annotation: @form(<name>, <args>...), which sits above a locus declaration and picks an efficient lowering (see spec/forms.md).

Lexically, @ is a single-character punctuation token. The parser routes the following form ident as a contextual keyword in this position only.

Reserved for future annotation surfaces; user-defined annotations are not in v1.

#   $   ?   ??  ?:

• Decimal: 0, 42, 1_000_000 (underscores permitted as digit separators).
• Hexadecimal: 0xFF, 0x1A_2B.
• Octal: 0o755.
• Binary: 0b1010_1010.
• Optional type suffix: 42i32, 0xFFu64. Default: int.

• Decimal: 3.14, 1.0e-3, 2.5E+10.
• Optional type suffix: 3.14f32, 2.5f64. Default: float.

• Suffix d: 1.50d, 0.05d. Used for the built-in decimal type (fixed-precision, no float artifacts; same semantics as the shopspring/decimal Go library).

• Duration suffixes: ns, us, ms, s, m, h, d. Examples: 100ms, 5s, 1h30m. Compound forms permitted.
• Time literals: ISO-8601 between backticks: `2026-05-08T12:00:00Z`.

• Double-quoted: "hello". Standard escape sequences: \n, \t, \r, \\, \", \', \0, and \xNN for ASCII bytes 0x00..=0x7f (added 2026-05-16; useful for separators like "a\x01b"). High bytes (\x80+) are rejected with a message pointing at std::bytes::* — the Rust String invariant would UTF-8-encode them as two bytes and surprise the caller. No \u{NNNN} Unicode escape at v1; agents needing non-ASCII drop to byte literals.
• Raw strings (r"...") and multi-line strings ("""...""") are NOT implemented — the lexer rejects both (verified 2026-07-22, GH #228 docs-drift audit; both were previously listed here aspirationally). Byte literals + f-strings cover the practical cases; add these forms to the lexer before re-documenting them.
• F-strings (v1.x-10): f"hello {name}" — interpolates Hale expressions inside {...}, each rendered via the same formatter println uses. {{ and }} are literal braces. Plain "..." strings keep { and } as ordinary characters for back-compat (no breaking change). Interpolation accepts any expression; types are converted with to_string.

• true, false.

• nil. Represents the absent value of an option type. Distinct from numeric zero or empty string.

• b"..." — byte-string literal. Same escape table as STRING_LIT (\n, \t, \r, \\, \", \0, \xNN), but the body is raw bytes — no UTF-8 promotion, so \xNN accepts the full 0x00..0xFF range (unlike the string literal which clamps at \x7f). Embedded NULs are preserved; the value’s length is carried alongside the byte buffer, so len(b) and std::bytes::at(b, i) work over the entire literal. Codegen lowers via lotus_bytes_from_buf(arena, src, len), which copies the source bytes into the caller’s arena.

  Use cases: binary protocol headers / framing, magic bytes, embedded fixtures. Earlier code reached for std::bytes::from_string("..."), which only worked for ASCII; the b"..." form replaces that workaround end-to-end.

These identifiers have semantic meaning in the standard library and are conventionally reserved, but are not parser-reserved keywords:

B               c               sigma           phi
k_max           span_max
sum             prod            min             max
length          empty
print           println
to_string       len             abs
Int             Float

Int(x) (v1.x-11) is a built-in cast — explicit Float → Int narrowing via fptosi (truncate toward zero). Int arg is the identity; other types reject. There is no implicit Float → Int conversion; the user must commit via this constructor-shaped call. to_string(x), len(x), abs(x), min(a, b), max(a, b) are similarly bare-name builtins.

print and println are built-in functions, always in scope without an import. They write to stdout. print does not emit a trailing newline; println does. They accept any number of arguments of any displayable type and concatenate.

publish is a built-in function in scope inside any locus that declares matching bus { publish SUBJECT of type T; }. The compiler verifies the subject and type at each call site. Out of scope in loci with no publish declaration.

The framework names (B, c, σ, φ) use ASCII spellings in source: B, c, sigma, phi. The framework’s k_max is k_max or its named alias span_max.

• Longest match: == is one token, not = followed by =.
• Keywords win over identifiers: run is the lifecycle keyword, not an identifier.
• Numeric literals are recognized greedily up to the first non-numeric character (after handling 0x, 0o, 0b prefixes, decimal point, exponent, and type suffix).
• Comments are stripped before parsing (except doc comments, which are attached to the following declaration as metadata).

The following characters are not part of any token in v0 and are lexer errors if encountered outside a string or comment:

` (outside time literal)    \   (vertical bar mid-token, not || or |=)

(Backticks are reserved for time literals only; bare backslash outside a string literal is illegal.)

Annotations are @-prefixed contextual idents (never reserved words), so a program may still use these names as identifiers. They attach to the declaration that follows.

effect NAME = { A, B }; optionally DEFINES a class as the union of others. A composed class owns no bit of its own, so forbidding it forbids every member, and a fn reaching a member carries it. Members may be built-ins or other declared classes; a definition cycle is rejected.

The built-in classes are syscall, block, time, entropy, env, ffi, publish, spawn, recursion, alloc, and secret_use (GH #436 — a privileged operation over confined secret material, carried by std::secret’s methods). Every built-in name is reserved: effect secret_use; — or effect syscall; — is an error, because the built-in wins at every use site and the declaration would be a silent no-op.

effect NAME; is a top-level declaration, not an annotation: it introduces a user effect class that is: / none: / causes: then name like a built-in. effect is a contextual keyword — recognized only at the start of a top-level item followed by an identifier and ;, so an existing effect binding is unaffected. Declared classes are single-seed (see spec/verification.md).

Effect annotations stack with each other and with @hot / @budget(...); see spec/verification.md for what each class means and where its truth comes from.

group NAME = { … } [may_be_empty]; is a top-level declaration of claim vocabulary, domain NAME = { … }; declares a closed index domain for effect families (effect knowledge(wing);), and claims { … } is a main-locus member holding named bundle-level claims (GH #382). The claim verbs are forbid reaches (with via / during / avoiding modifiers), only edges (grant enumeration), bound (path budgets over user classes, plus the built-in secret_use — the one counted built-in with no @budget spelling), require subscribes|publishes (existence), require sealed(all G) and require attributed(all C) (GH #436 — universals over the closed world rather than over a path), cover, and count. All introducers — group, claims, domain, forbid, reaches, via, may_be_empty, edges, bound, require, sealed, attributed, all, cover, count, during, avoiding, seed — are contextual keywords, recognized in their positions only, so existing identifiers with those names are unaffected. @budget(<user class> = N) bounds calls to declared carriers of the class. See spec/verification.md § Claims and spec/grammar.ebnf.