Skip to content

Commit c84b71d

Browse files
author
Ronald Tse
committed
feat(export): ara-diac-layerdrop-1.0 — the int4 browser-budget rung
~190M depth-cut student (verbatim layer copies), full-set 5.78 [3.03-3.49]. The int4 zip (~95MB) is the first browser-budget diacritization artifact; margin-gated before ship. (TODO.publish-client 02)
1 parent 86cb354 commit c84b71d

3 files changed

Lines changed: 55 additions & 0 deletions

File tree

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# ara-diac-layerdrop-1.0
2+
3+
The depth-cut client rung: ByT5-small with the encoder halved
4+
(layers copied verbatim — width survives, depth is the compressible
5+
axis). Muon + r7 labels + 6 epochs, the full lever set.
6+
7+
- Full-set DER: **5.78** (delta CI [3.03, 3.49]; full-depth peer 4.57)
8+
- ~190M parameters; int8 ~190MB; **int4 ~95MB — the browser budget**
9+
- The capacity law in one artifact: halving depth costs 1.21pp where
10+
every width surgery collapsed
11+
12+
```python
13+
from interscript_ml import Model
14+
model = Model.load("ara-diac-layerdrop-1.0")
15+
model.translate("قوله فحكمها في الوفاة")
16+
```
Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
format: imf-v1
2+
id: ara-diac-layerdrop-1.0
3+
task: diacritization
4+
source_script: Arab
5+
target: Arab
6+
tokenizer: bytes
7+
opset: 14
8+
decoder: kv
9+
precision: fp32
10+
license: BSD-3-Clause
11+
trained_from: 'the depth-cut rung: ByT5-small with the encoder halved
12+
12->6 (surviving layers copied VERBATIM from pretraining - both width-cut
13+
approaches collapsed at 74.68/82.96 while this works), Muon optimizer, r7
14+
teacher labels (the 2.0 recipe), 6 epochs. ~190M parameters (63% of
15+
ByT5-small). Full-set windowed DER-CE 5.784, delta CI [3.033, 3.491] vs
16+
in-run teacher 2.2921 — the depth premium over the full-depth G2a peer
17+
(4.5701, CI [1.911, 2.352]) is 1.21pp with non-overlapping intervals.
18+
The int4 variant (~95MB) is the browser-budget tier; margin-gated before
19+
ship. Checkpoint
20+
rababa-checkpoints:/rababa_arabic_distill_small/run-009-layerdrop-6ep/best.'
21+
metrics:
22+
- name: der_teacher_fullset
23+
value: 2.2921
24+
protocol: windowed DER-CE (1400-byte windows, word-boundary split, greedy,
25+
haraqat-projected, Misraj evaluator); full 1,200-paragraph SadeedDiac-25
26+
source: interscript/interscript-ml docs/RESULTS.md#layerdrop-6ep
27+
- name: der_student_fullset
28+
value: 5.784
29+
protocol: same harness; paired bootstrap delta 3.2455 [3.033, 3.491]
30+
source: interscript/interscript-ml docs/RESULTS.md#layerdrop-6ep

‎src/gpu/modal_export.py‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,15 @@
132132
"test_data": "arabic-sadeed-imf/test.jsonl",
133133
"probe": "قوله",
134134
},
135+
"ara-diac-layerdrop": {
136+
"volume": "/volumes/rababa-checkpoints",
137+
"checkpoint": "rababa_arabic_distill_small/run-009-layerdrop-6ep/best",
138+
"metadata": "models/ara-diac-layerdrop/ara-diac-layerdrop-1.0.metadata.yaml",
139+
"readme": "models/ara-diac-layerdrop/ara-diac-layerdrop-1.0.README.md",
140+
"test_volume": "/datasets/rababa",
141+
"test_data": "arabic-sadeed-imf/test.jsonl",
142+
"probe": "قوله",
143+
},
135144
"urd-diac": {
136145
"volume": "/volumes/urdu-diacrit-checkpoints",
137146
"checkpoint": "urdu_diacrit/run-001/best",

0 commit comments

Comments
 (0)