|
| 1 | +# 01 — Fix lutaml-model YAML deserialization for large collections |
| 2 | + |
| 3 | +## Problem |
| 4 | +The YAML round-trip works for small maps (<100 rules per parallel block) |
| 5 | +but fails for large maps. After YAML → model → hash, some `to` items have |
| 6 | +nil values. This affects 17/20 maps in the round-trip test. |
| 7 | + |
| 8 | +## Root Cause |
| 9 | +lutaml-model's YAML deserialization (`from_yaml`) doesn't properly |
| 10 | +reconstruct nested `Item` attributes when processing large collections. |
| 11 | +The `Item` model has 9 optional attributes (type, value, name, index, |
| 12 | +lo, hi, chars, parts, inner) — lutaml-model may not correctly set all |
| 13 | +of them during deserialization of deeply nested structures. |
| 14 | + |
| 15 | +## Investigation Steps |
| 16 | +1. Check if lutaml-model v0.8.19 has a known issue with nested Serializable types in collections |
| 17 | +2. Test with a minimal 200-item parallel block to reproduce |
| 18 | +3. Check if the issue is in YAML parsing (Psych) or in lutaml-model's attribute mapping |
| 19 | +4. Consider using a custom `from_yaml` override in `Model::Item` that handles the discriminator |
| 20 | + |
| 21 | +## Potential Fixes |
| 22 | +### Option A: Custom deserialization for Item |
| 23 | +Override `Item.from_yaml` to manually parse the hash and construct |
| 24 | +the correct object based on the `type` field. |
| 25 | + |
| 26 | +### Option B: Flatten Item into Rule |
| 27 | +Instead of a polymorphic Item class, flatten all item attributes |
| 28 | +into Rule (from_type, from_value, from_name, etc.). Less elegant |
| 29 | +but avoids lutaml-model's collection deserialization issues. |
| 30 | + |
| 31 | +### Option C: Use JSON instead of YAML |
| 32 | +lutaml-model's JSON serialization might not have the same bug. |
| 33 | +Test if JSON round-trip works for large collections. |
0 commit comments