coff
self-hosted
coff is written in c0 and compiles its own source. that is the milestone that lets the
whole stack lean on as little outside itself as it can. the compiler that first bootstrapped it, coff0 (explained below), now serves only as
a testing oracle. every self-hosted build is checked byte-for-byte against
it,a byte-for-byte match means the self-hosted compiler produces exactly the same output as the reference, which is strong evidence it did not drift as it took over from the bootstrap.
so the two must agree. moonshot itself is built through the self-hosted compiler alone.
coff0
a self-hosted compiler has a chicken-and-egg problem. coff is written in c0, so
something else had to compile it the very first
time.the naming follows the generations, coff0 is the zeroth compiler, the one that existed before c0 could compile anything, the self-hosted compiler it built is coff1.
that something is coff0, a compiler for the same c0 language, written
in plain C and built with an ordinary C compiler. it existed for exactly one
purpose, to compile the first c0-written coff, and once that build could compile
itself, coff0 was retired from the build entirely.
splitting source across files
to keep large programs manageable, c0 has one simple tool for splitting source across
files, an include directive, resolved by coff as plain textual
substitution before compilation begins.
include "jenna.c0";
include "chrone.c0";
include "punkt.c0";
this is substitution, not separate compilation.the included files are spliced in as raw text, and one flat buffer reaches the compiler, exactly as a single file always has. the parser does not even know the feature exists. it is enough to break a growing kernel into files named after its parts, without inventing a whole module system before one is needed.
inside the compiler
coff is a straight pipeline. include expansion produces one flat buffer of source, a lexer turns it into tokens, a parser builds the program from those tokens, and code generation emits x86_64 assembly. that text goes to smed, the project's own assembler and linker, which turns it into a freestanding binary. there is no optimiser, the point, for now, is output that is simple and easy to check.
the last stage is smed, a single tool that both assembles and links, written in c0 and compiled by coff, so the whole path from source text to bytes is the project's own rather than ending in the system assembler and linker. why it exists, how it is built to be easy to check and how it is checked against the tools it replaced is on the trust page.
extern
an extern int name; declaration tells coff that the symbol is defined
elsewhere (by the linker, or by another compilation unit). in expression context,
an extern name evaluates to its address (lea rax, [name]). this is
how moonshot resolves linker-defined symbols like section boundaries without
hardcoded address literals.
limits
the compiler's internal tables have grown alongside the moonshot codebase.
MAX_FUNCS, MAX_VARS, the symbol-table allocation and the
string table have each been raised more than once as the kernel outgrew them. they
are compile-time constants in both coff0.c (the C bootstrap) and
c0/coff.c0 (the self-hosted compiler), kept in lockstep and
verified by the bootstrap fixpoint check each time they change.
--raw
pass --raw to bypass the assembler and linker entirely. coff emits
x86-64 machine code directly into a flat binary, with no assembly text and no external
tools. coff --raw input.c0 output.bin. the binary starts with a
_start entry point that calls main and exits with its
return value.
the raw backend walks the same ast and uses the same code generation functions as
the text backend. every gen_* function has a parallel raw-byte path,
which encodes instructions inline and uses a label-and-patching system for forward jumps and
function calls. string literals are embedded inline in the code stream with a
short jump-over pattern and rip-relative addressing. print() emits
an inline strlen loop followed by a write syscall.
the raw mode covers numeric literals, local variables, arithmetic, comparisons,
if/else, while and for loops
with break/continue, function definitions with full
prologues and parameter shadowing, string literals with inline embedding, general
function calls with the standard calling convention (up to 6 arguments), and the
builtins load8, store8, load64,
store64, print, and exit. globals,
indirect calls, and linux i/o builtins are deferred.
--elf
pass --elf to emit a real ELF64 executable targeting
moonshot's ring-3 syscall ABI instead of linux.
coff --elf input.c0 output.elf. the output is a 64-byte ELF header, one
56-byte PT_LOAD program header (the whole file mapped at 0x80000000, RX),
and then the same raw machine code --raw produces, except with
_start issuing moonshot's SYS_EXIT (syscall 0) and
print() issuing SYS_DEBUG_PRINT (syscall 1, rdi=str,
rsi=len) instead of linux write. moonshot's elf_load maps
it page by page into a per-process address space, and
spawn3 file.elf runs it in its own kakel window. the same
builtins --raw supports work here too, plus the ring-3 syscall
builtins moonshot grew for gnista, which are
win_info, win_max, fill,
key_poll, blit, present,
readfile, alloc, and ticks.
alloc() works here because each ring-3 process gets its own
heap from the kernel, not because moonshot grew a brk.
win_info and win_max look like a pair and are
not. the first is the window's size right now, which changes on every
layout reflow, and the second is the ceiling a program budgets its frame
buffer against, which never does.
this closes the full toolchain loop, write c0 → coff --elf → ELF64 →
moonshot's ELF loader → ring 3 in a kakel window, and it is the path
gnista, the game engine, is built through.
where the oracle does not reach
the differential oracle described above is the backbone of trusting this
compiler, and it has a hole worth stating plainly. coff0 implements
only the text backend. it has no raw mode and no
--elf mode at all. so every codegen-diff test, and the whole
test suite around it, exercises the path that emits assembly, and proves
nothing whatsoever about the path that emits machine code directly. that
is the same path every ring-3 program on moonshot is built through.
the machine-code backend also has more ways to be quietly wrong than the
text backend, because it resolves things itself that an assembler would
otherwise check. labels go through a single flat numeric table rather
than distinct names, so two constructs that both need a label have to
reserve their own, and both that table and the jump-patch table are
bounds-checked and fail loudly when they overflow. a
movabs carries a full eight-byte immediate and both halves
are written, with the high word taken as a floor division rather than
c0's truncate-toward-zero so negative constants come out right. builtin
arguments follow the syscall convention, where the fourth
argument travels in r10 because the instruction itself
destroys rcx, and rex prefixes are chosen by which operand
field they extend, since 0x4C and 0x49 both
assemble cleanly with the same following byte and move the data in
opposite directions. a compiler that silently emits wrong code is worse
than one that refuses, and every one of these is a place where the
text backend cannot go wrong and the raw backend can.
giving the backend an oracle of its own
the machine-code backend has a verification story, and it is on
purpose not a differential one, because there is nothing to be
differential against. instead there is a set of small programs that
each compute a value which only comes out right if code generation is
right, and return it. the expected answer sits next to the program as a
comment. each one is compiled with --elf, baked into the
kernel, spawned in ring 3 on real moonshot, and the exit code it reports
over the serial line is compared against that comment. they stay small
and arithmetic on purpose, because the point is that a wrong answer is
a compiler bug and not a kernel one.
what they cover is the places listed above rather than a general idea
of coverage. there is chained control flow, and &&
and || with real short-circuit checks, where a side effect
proves the right-hand side was skipped. there is argument passing
through all six registers, arithmetic and comparison, recursion,
layout with sizeof and alloc()
together, readfile(), nested loops with an early return
and exit() from inside them. argument passing in
particular is a class the differential suite cannot check even in
principle, since the text backend writes
mov [rbp - N], reg as one instruction and never
stages the value through rax at all, which is the argument
for this suite existing.
source
coff and the whole test suite live at github.com/cofflang/coff. the readme walks through bootstrapping it from nothing but a C compiler and the system assembler and linker, which are needed exactly once. after that first build coff compiles itself and smed assembles and links it. neither of the system tools is touched again. contributions are welcome.