Build a job queue

In about thirty minutes, you’ll build a small job queue and watch it descend the four altitudes — from a throwaway script to a service split across processes — changing almost nothing but main at the very end. The first three stages run in the browser at the playground (no install); to follow along locally, drop each program in a .hl file and hale run it.

We’ll keep the “work” trivial — squaring a number stands in for whatever a real job does — so the shape of the program stays in focus.

Start with the data and the work, as a plain script. A type is pure data; a fn does something with it.

type Job { id: Int; work: Int; }

fn process(j: Job) -> Int {
    return j.work * j.work;
}

fn main() {
    let j: Job = Job { id: 1, work: 7 };
    println("job ", j.id, " -> ", process(j));
}

job 1 -> 49

This is Hale as a small, clean scripting language — no ceremony, no runtime to think about. One job, processed.

A queue needs to hold jobs. In Hale a collection is a locus with a @form annotation — no Vec<T> to import or parameterize. @form(vec) synthesizes push, get, pop, len, and is_empty on the locus; get/pop are fallible (out of range), so you address them at the call site with or.

type Job { id: Int; work: Int; }

@form(vec)
locus Queue {
    capacity { heap jobs of Job; }
}

fn process(j: Job) -> Int { return j.work * j.work; }

fn main() {
    let q = Queue { };
    q.push(Job { id: 1, work: 7 });
    q.push(Job { id: 2, work: 3 });
    q.push(Job { id: 3, work: 9 });
    println("queued: ", q.len());

    for j in q.items {
        println("job ", j.id, " -> ", process(j));
    }
}

queued: 3
job 1 -> 49
job 2 -> 9
job 3 -> 81

This is the everyday altitude — loci as plain objects that hold state and expose behavior. Still a single program, run start to finish.

A real queue doesn’t drain itself in a loop — work arrives, and workers react. That’s the typed message bus. Declare the channels as topics, and wire loci to them: a Worker subscribes to Jobs, does the work, and publishes a Result; a Reporter subscribes to Results; a Submitter publishes jobs.

type Job    { id: Int; work: Int; }
type Result { id: Int; out: Int; }

topic Jobs    { payload: Job; }
topic Results { payload: Result; }

locus Worker {
    bus {
        subscribe Jobs as on_job;
        publish   Results;
    }
    fn on_job(j: Job) {
        let out: Int = j.work * j.work;
        Results <- Result { id: j.id, out: out };
    }
}

locus Reporter {
    bus { subscribe Results as on_result; }
    fn on_result(r: Result) { println("job ", r.id, " done -> ", r.out); }
}

locus Submitter {
    bus { publish Jobs; }
    birth() {
        Jobs <- Job { id: 1, work: 7 };
        Jobs <- Job { id: 2, work: 3 };
        Jobs <- Job { id: 3, work: 9 };
    }
}

fn main() {
    Worker { };
    Reporter { };
    Submitter { };
}

job 1 done -> 49
job 2 done -> 9
job 3 done -> 81

Run it: this exact program is live in the example gallery — no install. Or paste it into the playground and edit as you go.

Notice what you didn’t write. The Submitter never calls the Worker — it publishes to a topic, and whoever subscribes gets the message. There’s no mutex, no channel type to choose, no async/await colouring a single function. This is the concurrent-services altitude, and the cardinality is emergent: add a second Worker { }; in main and both receive jobs — the topic is many-to-many.

So far the bus has been running in-process (the default transport — an in-memory queue). The loci don’t know or care. That’s the seam we pull on next.

The loci above never mention threads or transports. You wire those in main — placement { } says where loci run, and bindings { } says how each topic travels. None of the Worker / Reporter / Submitter code changes; you give them a new main per deployment.

To run the worker as its own process — listening for jobs over a Unix socket, on its own cooperative pool — that’s a main locus:

// worker.hl — the worker as its own binary. Import the shared Job/Result
// types, the Jobs/Results topics, and the Worker/Reporter loci from §3;
// only this `main` is new.
main locus WorkerNode {
    params {
        worker:   Worker   = Worker { };
        reporter: Reporter = Reporter { };
    }
    placement {
        worker: cooperative(pool = jobs);   // its own pool / OS thread
    }
    bindings {
        Jobs: unix("/run/jobs.sock", role: listen);
    }
}

The job source becomes a second binary whose main instantiates the Submitter and binds the same topic with role: connect (Jobs: unix("/run/jobs.sock", role: connect);). Same Jobs topic, same typed payload — now crossing a process boundary instead of an in-memory queue. Swap unix(...) for udp://host:port or a broker adapter and the loci still don’t change; only main does. (Add a codec(...) on the binding to put JSON or protobuf on the wire so a non-Hale peer can read it.)

For the full multi-binary picture — sharing the loci across files, picking transports, and supervising the workers — see Across binaries and Concurrency & placement.

The same Job / Worker / topic definitions carried you from a script to a distributed service. Each altitude added exactly what it needed and nothing more:

Altitude, not scale. These four steps are how much machinery you are choosing to control. They are a different axis from the scales the README walks — value, component, service, application, environment, fleet — which measure how large the composed system is. This tutorial climbed one axis while staying near the bottom of the other, which is why the same Worker survived every stage.

That last row is the point: a Hale program is a design of loci and topics; where and how it runs is a binding you change in one place. From here, the concurrent services chapters go deeper on lifecycle, failure, and supervision — or open the playground and run the bus version in your browser.