The second engagement
The Lexicon Build
The audit measures what your traffic costs you. The build delivers the
vocabulary that removes it — a private, domain-tuned lexicon, as a file you hold, measured on
your held-out text before it is delivered. The lexicon’s savings arrive once a model is adapted
to run on it.
Exclusively yours
The lexicon is a file you keep, licensed to you exclusively and permanently. We deliver it to no one else and retain no copy once the engagement closes
100%
Byte-identical round-trip on every UTF-8 text document in your corpus, including control bytes, mixed line endings and unusual scripts — not a sample
Adaptation-ready
Certified lossless and measured before any model work begins, so adaptation starts from a known vocabulary
Or no fee
If the build misses the ratio your audit measured, or misses losslessness, you do not pay for it
Why we can guarantee it
Because the measurement comes first. This is the
whole reason the two engagements are separate.
By the time a build is on the table, your audit has already measured what a tuned vocabulary
returns on your own held-out text. We are not forecasting a number — we are
committing to one we have both already seen, on your data, in a document you hold.
That is an unusual position for a vendor to be in, and it
is deliberate. The ordinary version of this sale asks you to believe a figure measured on someone
else’s corpus. The audit exists so that no part of the build rests on belief. It is also why
the audit is worth buying even if you never commission a build: the number is yours either way.
The lexiconAs a file, licensed to you exclusively and permanently, including inside the tokenizer of any model you train. We deliver it to no one else, retain no copy after the engagement closes, and it does not phone home.
A measured report on held-out textRatio against the tokenizer you run today, vocabulary utilisation, and the arithmetic in full — measured on the partition the build never saw.
Losslessness certificate, issued by Oculi DeiEvery UTF-8 text document in your corpus encodes and decodes byte-identically, including control bytes, mixed line endings and unusual scripts. Not sampled — all of it. If one does not, the engagement does not close.
An integration noteHow to load it, and a plain statement of what it does and does not change in your stack.
For teams that train their own models
If you pretrain a model, you choose a vocabulary anyway.
This is that choice, measured first.
A model learns its vocabulary when it is trained, so a team that pretrains on the lexicon needs no
adaptation step at all, and every training step and every later request carries more of its text. On public
Hindi, Ukrainian and Polish text, one vocabulary that also covers English and code needs 1.50×,
1.97× and 1.74× fewer tokens than Sarvam-105B’s, Gemma 3’s and EuroLLM’s
tokenizers (the full table).
That is a statement about token counts, not about the model you train
with it. Moving an already-trained model onto the lexicon is adaptation, which is scoped as its own
engagement. The ratio, the losslessness and the acceptance terms below are the same for builders as for
anyone: measured on your own held-out text first, agreed in writing, or there is no fee.
Acceptance, agreed in writing first
Both criteria are settled before the build
starts, not assessed afterwards.
RatioMeets or exceeds the figure your audit measured on the same held-out partition, within a tolerance agreed in advance and written down.
Losslessness100% byte-identical round-trip across every UTF-8 text document in the corpus, including control bytes, mixed line endings and unusual scripts. A single failure is a failure.
Miss either, and there is no feeNot a discount and not a credit. The commitment is only worth something if it costs us something.
Where it stops
The limits are here rather than in a footnote,
because finding them later is worse for both of us.
Your corpus sets your numberA lexicon tuned on your traffic returns its figure on your traffic. It does not generalise to a different distribution, and testing it against text from elsewhere will show a worse number. That is expected, not a defect.
Fewer tokens is compute, not an API invoiceIf you consume a third-party API, that provider counts with the provider’s own tokenizer and this changes nothing about that bill. The value is realised by a model adapted to the lexicon, as prefill work and context headroom.
Latency and compute, measuredOn identical requests and hardware, a model adapted to its lexicon cut median latency by 51.4% against the model it came from — 61.24 s to 29.75 s — and used about half the compute time, at the same accuracy. Your own workload is measured before anything is committed.
Privacy means a withheld artifactThe token stream is not interpretable without the lexicon, and the lexicon stays with you. That is the privacy of keeping something private — not encryption. No threat model has been evaluated and no confidentiality guarantee is implied against an adversary who obtains the lexicon.
Next step
Start with the measurement
Domain, approximate volume, and whether you are
self-hosted or on an API. If you have already audited, tell us and we will scope the build
against the figure we measured.
Enquire about a build →