कथा · the making of वाक्
0.0.1 to 0.12.1. Vāk began as 7,161 lines of Python that could not have compiled one line of itself. It now compiles itself completely, runs on five independent engines held to identical output, and needs no Python at all. This is how that happened — including the two optimisations that turned out to be worth nothing.
0.10.0 a version that was actually cut 0.4.0 a stage, numbered here in retrospect the date it was cut; earlier stages predate the repository
Act I
Python builds a language
The first version could not have compiled a single line of itself. It was a lexer, a parser and a tree-walking interpreter — 7,161 lines of Python — and everything the language could do, Python did on its behalf.
That is the ordinary way to start, and it contains the ordinary trap: a language whose only implementation is written in another language has not really escaped it. Everything that follows is an attempt to leave.
Sanskrit does not decide meaning by word order. It marks the role each participant plays in the action.
Pāṇini analysed a sentence by kāraka: who acts, what is acted upon, by what means, from what source. Vāk lets a function's parameters declare those same 6 roles — and the analyser then enforces the grammar of roles, including Pāṇini's rule that an action has at most one agent and one patient.
Because the roles are marked, the arguments carry their own labels and the order stops mattering. That is not a convenience feature. It is the freedom case endings give a Sanskrit sentence, moved into a type system.
This is the part that exists nowhere else, and it arrived early — before any of the machinery below.
A language that only runs is little use to a student; it has to explain what
is wrong. So came the semantic analyser — undefined names, type mismatches,
assignment to a constant, विरम outside a loop, unreachable code,
and the kāraka rules.
Every message is bilingual and every diagnostic carries a Sanskrit code. That decision paid for itself much later in a way nobody planned: because the analyser already reported structured diagnostics, the editor extension needed no analysis of its own.
Act II
The language learns to build itself
This is the hinge of the whole story.
A tree-walking interpreter cannot be self-hosted in any satisfying way. So the language grew a second engine: a compiler to bytecode, and a stack machine to run it — 51 instructions, each with a Sanskrit name.
That mattered because it created something small and precise enough to be re-implemented. The Python VM became a specification. Anything that could emit the same instructions and run them to the same output was a legitimate implementation of Vāk — including one written in Vāk.
The lexer came first: 285 lines of Vāk that split Vāk source into tokens, held to the Python lexer on the kind, lexeme, value and line of every token.
Then the parser, 916 lines, held to the whole syntax tree, node for node. Then the compiler, 809 lines, held to every instruction, constant and line number.
At each step the new stage had to process its own source and agree. The parser parsing the parser. That is the point where this stopped being a toy.
The virtual machine — dispatch loop, environments, call frames, exception unwinding — written in Vāk, in 1205 lines. A machine describing a machine, in the language that machine runs.
It is still in the repository though nothing depends on it now: it is the executable specification of what the instructions mean, and the tests still run every example through it.
Act III
Python leaves
The native back end does not translate Vāk into C statement by statement. It emits the bytecode as C data and links it against a C implementation of the same machine — 3,548 lines. That is deliberate: every engine runs the same instructions, so every engine can be held to the same output.
Compile the Vāk-written toolchain that way and the result is वाक्.exe — 737 KB, self-contained, no Python anywhere in the loop. Python is used to build it once, exactly as a C compiler is used once to build a self-hosting C compiler. After that the language stands on its own.
Act IV
Making it real
The analyser was the last stage with no Vāk counterpart, and the fussiest to match. A lexer either agrees on a token or it does not; an analyser has to produce the same complaints, in the same order, with the same wording. It does — across sixty-five files, forty of them deliberately broken, one per diagnostic kind.
With that, the entire front end exists in Vāk: 4,599 lines. The language describes itself completely.
A language nobody can install is a paper. So: a browser playground — the whole toolchain compiled to WebAssembly — the language runs in the page, not on a server. An editor extension whose grammar is generated from the language's own keyword tables, so highlighting cannot drift. A typing aid, because Devanagari on a plain keyboard is otherwise a wall. POSIX branches, so Linux and macOS are reachable.
39 keywords — each spelled three ways, so none of it needs a Devanagari keyboard — 48 built-ins, 16 worked examples — and every keyword spelled two ways, so none of it requires a Devanagari keyboard.
Act V
Making it fast
The last act is measurement, and it holds the most useful mistakes.
The obvious suspect was name lookup: every variable was found by comparing Devanagari strings, and Devanagari strings all begin with the same byte. So names were resolved to places, and built-ins reached directly. Together those removed 71% of all string comparisons — and moved the wall clock 2%.
A second attempt exploited the fact that identical names share one pointer. It moved the clock not at all: most of a scope scan is spent on entries that do not match, and a pointer check only helps the one that does.
The intuition that name lookup dominates an interpreter was wrong twice.
Every real gain was elsewhere — in allocation, and in work that needed not to have happened at all. A call formatted an error message for every argument and discarded it. A string was copied four times in order to join it twice. Building a string was quadratic, because each append rebuilt the whole of it.
Act VI
Going back for what was missing
A language that compiles itself can still be missing ordinary things, and this act is the unglamorous work of finding out which. The method was to write down what C has and Python has, and go through the list.
Bit operations, as named functions rather than operators — ^ was
already exponentiation. A second output stream, so a program's warnings stop
arriving in the middle of its data. Default arguments, which turned out to be a
question about Sanskrit rather than syntax: an unexpressed kāraka is ordinary in
the language, so a default may sit anywhere in a parameter list when
every parameter names its role, not only at the end.
Then the two that had been missing longest: प्रयुज्, to call a
function with an argument list computed at run time, and लक्षणम्,
to ask a function what it declares. Without them no program could write a
wrapper around another. And कुरु — the loop that asks its question
afterwards — which cost no new instruction at all, being the two jumps a
यावत् already emits, in the other order.
Three of these shipped to four engines and not the fifth, and the test that should have caught it was reading source files instead of running them.
The tests that replaced it run every built-in on every engine and require the answers to match. They found the gap within a day — and then found a use-after-free in the C runtime that had been crashing an immediately-invoked function since long before any of this.
One fault, though, no test in this repository could have found. After
publishing, installing the package from PyPI into an empty environment and
running it there produced cc1.exe: fatal error: … yantram.c: No such file
or directory. The C runtime ships with the repository and not with the
wheel — a deliberate choice, and one the self-hosted front end had always
explained to anyone who asked for it. The native back end had never learnt to
say the same, so it failed in the compiler's words instead of its own.
A clone always has the files. Only the thing you published can tell you what you failed to publish.
Five engines, one language, and a test suite whose job is to prove they never disagree.
python bench/benchmark.py measures Vāk against C, CPython,
Node and PHP, and says plainly which of them these workloads are too small to
measure rather than printing a number that looks meaningful.