Skip to content

Commit e1e7459

Browse files
Ronald Tseronaldtse
authored andcommitted
fix: explicit d_kv 64 in the layerdrop spec
1 parent 32ccb09 commit e1e7459

1 file changed

Lines changed: 1 addition & 0 deletions

File tree

‎src/gpu/distill_specs.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,7 @@ ara-diac-small-layerdrop:
142142
layer_drop: 'true'
143143
student_config:
144144
d_model: 1472
145+
d_kv: 64
145146
d_ff: 3584
146147
num_heads: 6
147148
enc_layers: 6

0 commit comments

Comments
 (0)