Merge pull request #176 from interscript/docs-libraries-measure

docs: Libraries + How-we-measure

Author: ronaldtse

src/content/docs/How_we_measure.adoc

@@ -0,0 +1,70 @@
+= How we measure
+
+Every quality number we publish is measured, recorded, and can be
+re-derived by anyone. This page explains how.
+
+== Where numbers come from
+
+Each model's accuracy figure comes from a fixed evaluation protocol,
+run on the complete test set, and is recorded in a public results log:
+https://github.com/interscript/interscript-ml/blob/main/docs/RESULTS.md[docs/RESULTS.md].
+The log records the number, a confidence interval where the protocol
+supports one, and a checksum of the exact training labels used.
+
+When a number turned out to be wrong, we corrected it in public. One
+example: the Arabic model ara-diac-small-2.0 was first published with a
+score of 4.82. Two independent re-measurements of the released files
+agreed on a different value, 5.08, and the original number was
+withdrawn (the correction entry in the results log explains what
+changed and what it affected).
+
+== Re-deriving our results
+
+The pieces are public:
+
+* The benchmark: https://huggingface.co/datasets/Misraj/SadeedDiac-25[SadeedDiac-25]
+* The scoring tool: `pip install interscript-ml-tools[sadeed]`
+* Our raw predictions for every published run: the
+  https://github.com/interscript/interscript-ml/releases/tag/frontier-predictions-v1[frontier-predictions-v1]
+  release
+* The teacher checkpoints behind the training results: the
+  https://github.com/interscript/interscript-ml/releases/tag/teachers-arabic-v1[teachers-arabic-v1]
+  release
+
+To re-score one of our runs:
+
+```
+pip install interscript-ml-tools[sadeed]
+interscript-sadeed-eval score \
+  --preds <predictions file> \
+  --data Misraj/SadeedDiac-25
+```
+
+The tool reproduces the published numbers exactly; that is how the
+incorrect 4.82 above was found.
+
+== Cross-runtime correctness
+
+The same model file must produce the same output in TypeScript,
+Python, and Ruby. We check this with a shared corpus of reference
+inputs and outputs (the
+https://github.com/interscript/interscript-ml/releases/tag/golden-v1[golden-v1]
+release), generated from the released artifacts themselves.
+
+The guarantee has a measured boundary. Models stored at full precision
+(fp32, fp16) produce byte-identical output on every machine we tested.
+Quantized models (int8, int4) run faster and smaller, but on different
+CPU types their outputs can differ slightly: tiny numerical
+differences flip decisions that the model is nearly undecided on. For
+those models the guarantee is quality-level, not byte-level, and each
+artifact carries its measured quality difference from the full
+precision version in its metadata.
+
+== Negative results
+
+Not every approach worked. Attempts to close the quality gap between
+the small student models and their teachers by adding more classical
+training text, changing the text mixture, or training on the model's
+own mistakes all failed, with measurements recorded in the results
+log. We publish these alongside the successes because they define what
+the current models are and are not.

src/content/docs/Libraries.adoc

@@ -0,0 +1,110 @@
+= Libraries
+
+Interscript ships as libraries for three languages, a command-line
+tool, and a web API. All of them run the same transliteration systems
+and the same neural models, and all model artifacts are
+checksum-verified when they load.
+
+== TypeScript (Node and browsers)
+
+Install:
+
+```
+npm install interscript
+```
+
+Transliterate with one of the 289 authority systems:
+
+```ts
+import { transliterateAsync } from "interscript"
+
+const out = await transliterateAsync(
+  "bgnpcgn-ukr-Cyrl-Latn-2019",
+  "Антон",
+)
+```
+
+Use a neural model (for example, Arabic diacritization). The model
+downloads once, is verified against its checksum, and is cached:
+
+```ts
+import { imf } from "interscript/ml"
+
+const resolved = await imf.resolve("ara-diac-layerdrop-1.0-int4")
+const model = await imf.IMFModel.fromZipBytes(resolved.bytes)
+const vocalized = await model.translate("السلام عليكم")
+```
+
+== Python
+
+Install:
+
+```
+pip install interscript-ml
+```
+
+Load a model and run it:
+
+```python
+from interscript_ml import Model
+
+model = Model.load("ara-diac-small-2.1-int8")
+print(model.translate("قوله فحكمها في الوفاة"))
+```
+
+== Ruby
+
+Install:
+
+```
+gem install secryst
+```
+
+Load and translate:
+
+```ruby
+require "secryst"
+
+model = Secryst::Model.load("ara-diac-small-2.1-int8")
+model.translate("قوله فحكمها في الوفاة")
+```
+
+== Command line
+
+The deterministic systems:
+
+```
+echo "Антон" | npx interscript t bgnpcgn-ukr-Cyrl-Latn-2019
+```
+
+The neural layer:
+
+```
+echo "السلام عليكم" | npx interscript m ara-diac-layerdrop-1.0-int4
+```
+
+Run `npx interscript --help` for the full command list, including bulk
+processing and script detection.
+
+== Web API
+
+A hosted REST API serves both layers:
+https://api.interscript.org[api.interscript.org]. Its documentation,
+including batch endpoints and asset serving, lives at
+https://api.interscript.org/docs[/docs].
+
+== Try it in the browser
+
+The link:/playground[playground] runs the full stack in your browser:
+pick any system, optionally vocalize Arabic text first, and copy the
+exact code for the configuration you selected. Nothing is sent to a
+server.
+
+== Which models exist
+
+The model catalog with measured quality numbers for every artifact is
+on the link:/ml[neural layer page]. The machine-readable index that the
+libraries resolve against is
+https://github.com/interscript/interscript-ml/blob/main/models.yaml[models.yaml]
+in the https://github.com/interscript/interscript-ml[interscript-ml]
+repository.