diff --git a/crates/shift-backends/docs/DOCS.md b/crates/shift-backends/docs/DOCS.md index 3d5ff54a..35ea8740 100644 --- a/crates/shift-backends/docs/DOCS.md +++ b/crates/shift-backends/docs/DOCS.md @@ -8,7 +8,7 @@ Font format backends that convert between on-disk font files and the `Font` IR u **Architecture Invariant:** `FontReader` and `FontWriter` require `Send + Sync`. WHY: Backends are stored in `FontLoader` which lives inside the editor's shared state; they must be safe to use from multiple threads. -**Architecture Invariant:** Eager reader/writer backends are stateless unit structs. A `FontImport` owns only the foreign bytes or GLIF directory records and its current cursor. WHY: ordinary conversion stays pure, while bounded imports retain only the state needed to produce the next batch. +**Architecture Invariant:** Eager reader/writer backends are stateless unit structs. A `FontImport` owns the foreign bytes, GLIF directory records, or one upstream-parsed Glyphs source model plus its current cursor; it never owns a complete geometry-resident Shift `Font`. WHY: ordinary conversion stays pure, while bounded Shift conversion retains only format state needed to produce the next batch. **Architecture Invariant:** `UfoWriter` stages a complete UFO beside the destination and swaps it into place only after the staged tree is durable. WHY: a failed save must preserve the previous source rather than leave a partial directory. @@ -20,7 +20,7 @@ Font format backends that convert between on-disk font files and the `Font` IR u **Architecture Invariant:** TrueType export compiles an owned snapshot of the Shift `Font` IR directly through fontir/fontc. It must not serialize a temporary UFO or fall back to another authoring format. WHY: `.shift` is the canonical authoring source, and an intermediate format would discard or reinterpret Shift concepts before compilation. -**Architecture Invariant:** TTF/OTF, UFO, and Designspace streaming imports parse glyphs in bounded Rayon batches and preserve input order when publishing each batch. Eager readers drain those same canonical streams rather than maintaining a second parser. UFO and Designspace share `GlifGlyphStream`; only source discovery differs. SQLite remains outside this crate and is written by one workspace-owned sink. WHY: one conversion path prevents eager/streaming semantic drift, while concurrent SQLite authors would add contention and weaken transaction ownership. +**Architecture Invariant:** TTF/OTF, UFO, Designspace, and Glyphs streaming imports convert glyphs in bounded Rayon batches and preserve input order when publishing each batch. Eager readers drain those same canonical streams rather than maintaining a second conversion path. UFO and Designspace share `GlifGlyphStream`; Glyphs parses its source model once, publishes stable glyph identities, then releases owned Shift batches through `GlyphsGlyphStream`. SQLite remains outside this crate and is written by one workspace-owned sink. WHY: one conversion path prevents eager/streaming semantic drift, while concurrent SQLite authors would add contention and weaken transaction ownership. **Architecture Invariant:** Compiled-font streaming enumerates `maxp` glyph IDs, not only `cmap` mappings. Unencoded glyphs receive their `post`/CFF name or a synthesized `gidN` name, and all Unicode mappings for a glyph share one authored glyph identity. WHY: `cmap` is character lookup, not the complete glyph directory. @@ -43,8 +43,10 @@ src/ reader.rs -- UfoReader eagerly drains the canonical UFO stream writer.rs -- UfoWriter: shift_font::Font -> atomically written norad::Font glyphs/ - mod.rs -- GlyphsReader re-export; fixture-based integration tests - reader.rs -- GlyphsReader: glyphs_reader::Font -> shift_font::Font (read-only) + mod.rs -- GlyphsReader and bounded stream exports; fixture-based integration tests + conversion.rs -- Glyphs header, glyph geometry, features, and kerning conversion + import.rs -- parsed Glyphs directory plus bounded parallel `GlyphsGlyphStream` + reader.rs -- eager compatibility reader that drains the canonical Glyphs stream designspace/ import.rs -- Designspace source discovery configured into the shared GLIF stream binary/ @@ -71,12 +73,13 @@ src/ - `UfoWriter` -- atomically writes `.ufo` bundles via `norad` - `DesignspaceReader` / `DesignspaceWriter` -- read and atomically write `.designspace` projects plus companion UFOs, including continuous/discrete axes, axis value labels, per-axis maps, and cross-axis mappings - `UfoBackend` -- unit struct implementing `FontBackend` by delegating to `UfoReader`/`UfoWriter` -- `GlyphsReader` -- loads `.glyphs` and `.glyphspackage` files via `glyphs-reader`; read-only (no writer) +- `GlyphsReader` -- eagerly drains the canonical `.glyphs` / `.glyphspackage` stream for compatibility callers; read-only (no writer) +- `GlyphsGlyphStream` -- owns one upstream-parsed Glyphs model and converts bounded, layer-aware Shift glyph batches in directory order - `FontExporter` -- compiles a `FontView` directly to TTF via `ShiftIrSource` and fontc ## How it works -**Loading a font:** `FontLoader::read_font` retains the eager API but the TTF/OTF, UFO, and Designspace readers implement it by draining their bounded streams. `FontLoader::stream_font` dispatches those sources to the same importers. It first returns complete top-level metadata and a cheap glyph/source directory, then materializes at most the requested batch of `Glyph` values. UFO and Designspace both feed shared GLIF work records into `GlifGlyphStream`; Designspace only adds stable multi-source discovery. Rayon converts geometry records in parallel; indexed collection preserves glyph order. The workspace writes and releases each batch before requesting another. +**Loading a font:** `FontLoader::read_font` retains the eager API but the TTF/OTF, UFO, Designspace, and Glyphs readers implement it by draining their bounded streams. `FontLoader::stream_font` dispatches those sources to the same importers. It first returns complete top-level metadata and a cheap glyph/source directory, then materializes at most the requested batch of `Glyph` values. UFO and Designspace both feed shared GLIF work records into `GlifGlyphStream`; Designspace only adds stable multi-source discovery. Glyphs syntax is parsed once by `glyphs-reader`; `GlyphsGlyphStream` preassigns every glyph identity so component references resolve before their bases are converted. Rayon converts geometry records in parallel; indexed collection preserves glyph order. The workspace writes and releases each Shift batch before requesting another. **Point type mapping (read):** norad uses separate `Move`, `Line`, `Curve`, `OffCurve`, `QCurve` types. The IR collapses `Move`/`Line`/`Curve` into `OnCurve` and keeps `OffCurve` and `QCurve` distinct. On write, context (position in contour, open/closed, preceding point type) is used to reconstruct the correct norad variant. @@ -84,7 +87,7 @@ src/ **Binary variation metadata:** The TTF/OTF reader imports `fvar` axis definitions, hidden flags, and named instances into the Shift IR. The bounded path enumerates every `maxp` glyph ID, groups all `cmap` values by glyph, deterministically derives contour/point identities from emitted positions, and preserves TrueType quadratics instead of expanding them to cubic control pairs. Binary glyph geometry is still materialized only at the default variation location; recovering editable `gvar` sources is separate work. -**Glyphs-format specifics:** `GlyphsReader` also extracts axes, sources, and per-master locations -- data that UFO does not natively represent. Kerning group membership is derived from per-glyph `right_kern`/`left_kern` fields and normalized to `public.kern1.*`/`public.kern2.*` conventions. +**Glyphs-format specifics:** `GlyphsReader` also extracts axes, sources, and per-master locations -- data that UFO does not natively represent. Kerning group membership is derived from per-glyph `right_kern`/`left_kern` fields and normalized to `public.kern1.*`/`public.kern2.*` conventions. The upstream parser currently materializes its complete normalized Glyphs source model before the bounded cursor begins; batching bounds Shift glyph conversion and persistence, not source-syntax parsing. **Designspace mapping:** Per-axis `` entries become independent `AxisMapping` values. Designspace 5.1+ `` entries become the font's single cross-axis mapping group. Axis value labels use the standard Designspace 5.0 `` representation; imported labels receive newly minted Shift identity because Designspace has no equivalent stable label ID. @@ -123,6 +126,7 @@ src/ - **Cross-platform UFO replacement:** macOS and Linux use an atomic directory exchange when supported. The fallback moves the old tree aside first and restores it if installing the staged tree fails. - **OnCurve ambiguity on write:** The IR's `OnCurve` type is context-dependent when writing. The first point of an open contour becomes `Move`, a point after `OffCurve` becomes `Curve`, everything else becomes `Line`. If contour structure is malformed, this heuristic may produce wrong results. +- **Glyphs source parsing is eager:** `glyphs-reader` materializes one normalized source model before `GlyphsGlyphStream` starts. Shift geometry conversion, packing, compression, and SQLite writes remain bounded. - **Glyphs kerning is default-master only:** Multi-master kerning is silently dropped to a single master's values. - **Cross-axis mappings:** Direct TTF compilation rejects cross-axis mappings until the compiler stack supports `avar` version 2. It never flattens the mapping or falls back to temporary UFO compilation. - **Authored STAT tables:** When Shift axis labels exist, export appends a generated `STAT` feature block. If authored feature text also declares `STAT`, the feature compiler reports the conflict. diff --git a/crates/shift-backends/src/font_loader.rs b/crates/shift-backends/src/font_loader.rs index 6410875f..3124d124 100644 --- a/crates/shift-backends/src/font_loader.rs +++ b/crates/shift-backends/src/font_loader.rs @@ -63,6 +63,11 @@ impl FontAdaptor for GlyphsFontAdaptor { fn write_font(&self, _font: &Font, _path: &str) -> FormatBackendResult<()> { Err(FormatBackendError::WriteUnsupported) } + + fn stream(&self, path: &str) -> FormatBackendResult)>> { + let (header, stream) = crate::glyphs::stream_font(path)?; + Ok(Some((header, Box::new(stream)))) + } } impl FontAdaptor for DesignspaceFontAdaptor { diff --git a/crates/shift-backends/src/glyphs/conversion.rs b/crates/shift-backends/src/glyphs/conversion.rs new file mode 100644 index 00000000..3e51209e --- /dev/null +++ b/crates/shift-backends/src/glyphs/conversion.rs @@ -0,0 +1,291 @@ +use std::collections::HashMap; + +use glyphs_reader::{FeatureSnippet, Font as GlyphsFont, Glyph as GlyphsGlyph, NodeType, Shape}; +use shift_font::{ + Anchor, Axis, Component, Contour, FeatureData, Font, Glyph, GlyphId, GlyphLayer, KerningData, + KerningPair, KerningSide, LayerId, Location, MetricKind, Source, SourceId, Transform, +}; + +use crate::{metrics::set_metric_position, FormatBackendError, FormatBackendResult}; + +const GLYPHS_SIDE1_PREFIX: &str = "@MMK_L_"; +const GLYPHS_SIDE2_PREFIX: &str = "@MMK_R_"; +const UFO_SIDE1_PREFIX: &str = "public.kern1."; +const UFO_SIDE2_PREFIX: &str = "public.kern2."; + +pub(super) fn font_header( + glyphs_font: &GlyphsFont, +) -> FormatBackendResult<(Font, HashMap)> { + let mut font = Font::empty(); + + if let Some(family_name) = glyphs_font.get_default_name("familyNames") { + font.metadata_mut().family_name = Some(family_name.to_string()); + } + if let Some(default_master) = glyphs_font.masters.get(glyphs_font.default_master_idx) { + font.metadata_mut().style_name = Some(default_master.name.clone()); + } + font.metadata_mut().version_major = Some(glyphs_font.version_major); + font.metadata_mut().version_minor = Some(glyphs_font.version_minor as i32); + font.metrics_mut().units_per_em = glyphs_font.units_per_em as f64; + + let mut axis_ids_by_index = Vec::new(); + for (index, glyphs_axis) in glyphs_font.axes.iter().enumerate() { + let axis_values = glyphs_font + .masters + .iter() + .filter_map(|master| { + master + .axes_values + .get(index) + .map(|value| value.into_inner()) + }) + .collect::>(); + if axis_values.is_empty() { + continue; + } + + let default = glyphs_font + .masters + .get(glyphs_font.default_master_idx) + .and_then(|master| master.axes_values.get(index)) + .map(|value| value.into_inner()) + .unwrap_or(axis_values[0]); + let minimum = axis_values.iter().copied().fold(f64::INFINITY, f64::min); + let maximum = axis_values + .iter() + .copied() + .fold(f64::NEG_INFINITY, f64::max); + let mut axis = Axis::new( + glyphs_axis.tag.clone(), + glyphs_axis.name.clone(), + minimum, + default, + maximum, + ); + axis.set_hidden(glyphs_axis.hidden.unwrap_or(false)); + axis_ids_by_index.push(axis.id()); + font.add_axis(axis)?; + } + + let mut source_ids_by_master_id = HashMap::new(); + for (master_index, master) in glyphs_font.masters.iter().enumerate() { + let mut location = Location::new(); + for (axis_index, axis_id) in axis_ids_by_index.iter().enumerate() { + if let Some(value) = master.axes_values.get(axis_index) { + location.set(axis_id.clone(), value.into_inner()); + } + } + let source_id = font.add_source(Source::new(master.name.clone(), location)); + let metric_definitions = font.metric_definitions().to_vec(); + let source = font + .source_mut(source_id.clone()) + .expect("a newly added source should exist"); + set_metric_position( + &metric_definitions, + source, + MetricKind::Ascender, + master.ascender(), + ); + set_metric_position( + &metric_definitions, + source, + MetricKind::Descender, + master.descender(), + ); + set_metric_position( + &metric_definitions, + source, + MetricKind::CapHeight, + master.cap_height(), + ); + set_metric_position( + &metric_definitions, + source, + MetricKind::XHeight, + master.x_height(), + ); + source.set_italic_angle(master.italic_angle()); + source_ids_by_master_id.insert(master.id.clone(), source_id.clone()); + if master_index == glyphs_font.default_master_idx { + font.set_default_source_id(source_id); + } + } + + *font.features_mut() = convert_features(glyphs_font); + *font.kerning_mut() = convert_kerning(glyphs_font); + Ok((font, source_ids_by_master_id)) +} + +pub(super) fn imported_layer_count( + glyph: &GlyphsGlyph, + source_ids_by_master_id: &HashMap, +) -> usize { + glyph + .layers + .iter() + .filter(|layer| source_ids_by_master_id.contains_key(layer.master_id())) + .count() +} + +pub(super) fn convert_glyph( + glyph: &GlyphsGlyph, + glyph_ids: &HashMap, + source_ids_by_master_id: &HashMap, +) -> FormatBackendResult { + let glyph_id = glyph_ids + .get(glyph.name.as_str()) + .expect("every Glyphs glyph should have a published identity") + .clone(); + let mut result = Glyph::with_id(glyph_id, glyph.name.to_string()); + result.set_unicodes(glyph.unicode.iter().copied().collect()); + + for layer in &glyph.layers { + let Some(source_id) = source_ids_by_master_id.get(layer.master_id()).cloned() else { + continue; + }; + + let mut result_layer = + GlyphLayer::with_width(LayerId::new(), source_id, layer.width.into_inner()); + for shape in &layer.shapes { + match shape { + Shape::Path(path) => { + let mut contour = Contour::new(); + for node in &path.nodes { + let (point_type, smooth) = convert_node_type(node.node_type); + contour.add_point(node.pt.x, node.pt.y, point_type, smooth); + } + if path.closed { + contour.close(); + } + result_layer.add_contour(contour); + } + Shape::Component(component) => { + let base_glyph_id = + glyph_ids.get(component.name.as_str()).ok_or_else(|| { + FormatBackendError::Glyphs(format!( + "component base glyph {:?} does not exist", + component.name + )) + })?; + let coeffs = component.transform.as_coeffs(); + result_layer.add_component(Component::with_matrix( + base_glyph_id.clone(), + component.name.to_string(), + &Transform { + xx: coeffs[0], + xy: coeffs[1], + yx: coeffs[2], + yy: coeffs[3], + dx: coeffs[4], + dy: coeffs[5], + }, + )); + } + } + } + + for anchor in &layer.anchors { + let name = if anchor.name.is_empty() { + None + } else { + Some(anchor.name.to_string()) + }; + result_layer.add_anchor(Anchor::new(name, anchor.pos.x, anchor.pos.y)); + } + + result.set_layer(result_layer); + } + + Ok(result) +} + +fn convert_node_type(node_type: NodeType) -> (shift_font::PointType, bool) { + match node_type { + NodeType::Line => (shift_font::PointType::OnCurve, false), + NodeType::LineSmooth => (shift_font::PointType::OnCurve, true), + NodeType::OffCurve => (shift_font::PointType::OffCurve, false), + NodeType::Curve => (shift_font::PointType::OnCurve, false), + NodeType::CurveSmooth => (shift_font::PointType::OnCurve, true), + NodeType::QCurve => (shift_font::PointType::QCurve, false), + NodeType::QCurveSmooth => (shift_font::PointType::QCurve, true), + } +} + +fn convert_features(font: &GlyphsFont) -> FeatureData { + let source = font + .features + .iter() + .filter_map(FeatureSnippet::str_if_enabled) + .collect::>() + .join("\n\n"); + if source.trim().is_empty() { + FeatureData::new() + } else { + FeatureData::from_fea(source) + } +} + +fn convert_kerning(font: &GlyphsFont) -> KerningData { + let mut kerning = KerningData::new(); + + for glyph in font.glyphs.values() { + if let Some(group) = glyph.right_kern.as_deref() { + let group_name = format!("{UFO_SIDE1_PREFIX}{group}"); + let mut members = kerning + .groups1() + .get(&group_name) + .cloned() + .unwrap_or_default(); + members.push(glyph.name.to_string().into()); + members.sort(); + members.dedup(); + kerning.set_group1(group_name, members); + } + + if let Some(group) = glyph.left_kern.as_deref() { + let group_name = format!("{UFO_SIDE2_PREFIX}{group}"); + let mut members = kerning + .groups2() + .get(&group_name) + .cloned() + .unwrap_or_default(); + members.push(glyph.name.to_string().into()); + members.sort(); + members.dedup(); + kerning.set_group2(group_name, members); + } + } + + let Some(default_master) = font.masters.get(font.default_master_idx) else { + return kerning; + }; + let Some(pairs) = font.kerning_ltr.get(&default_master.id) else { + return kerning; + }; + + for ((first, second), value) in pairs { + let first_side = if let Some(group) = first + .strip_prefix(GLYPHS_SIDE1_PREFIX) + .or_else(|| first.strip_prefix(GLYPHS_SIDE2_PREFIX)) + { + KerningSide::Group(format!("{UFO_SIDE1_PREFIX}{group}")) + } else { + KerningSide::Glyph(first.to_string().into()) + }; + let second_side = if let Some(group) = second + .strip_prefix(GLYPHS_SIDE2_PREFIX) + .or_else(|| second.strip_prefix(GLYPHS_SIDE1_PREFIX)) + { + KerningSide::Group(format!("{UFO_SIDE2_PREFIX}{group}")) + } else { + KerningSide::Glyph(second.to_string().into()) + }; + kerning.add_pair(KerningPair::new( + first_side, + second_side, + value.into_inner(), + )); + } + + kerning +} diff --git a/crates/shift-backends/src/glyphs/import.rs b/crates/shift-backends/src/glyphs/import.rs new file mode 100644 index 00000000..d3fe0c53 --- /dev/null +++ b/crates/shift-backends/src/glyphs/import.rs @@ -0,0 +1,85 @@ +use std::{collections::HashMap, path::Path}; + +use glyphs_reader::{Font as GlyphsFont, Glyph as GlyphsGlyph}; +use rayon::prelude::*; +use shift_font::{Font, Glyph, GlyphId, SourceId}; + +use super::conversion::{convert_glyph, font_header, imported_layer_count}; +use crate::{ + import::{GlyphDirectoryEntry, GlyphStream, ImportBatchLimit}, + FormatBackendError, FormatBackendResult, +}; + +/// Bounded Shift conversion over one parsed Glyphs source. +pub(crate) struct GlyphsGlyphStream { + glyph_ids: HashMap, + glyphs: Vec, + source_ids_by_master_id: HashMap, + next_glyph: usize, +} + +impl GlyphStream for GlyphsGlyphStream { + fn directory(&self) -> Vec { + self.glyphs + .iter() + .map(|glyph| GlyphDirectoryEntry { + glyph_id: self.glyph_ids[glyph.name.as_str()].clone(), + name: glyph.name.to_string().into(), + }) + .collect() + } + + fn glyph_count(&self) -> usize { + self.glyphs.len() + } + + fn next_batch(&mut self, limit: ImportBatchLimit) -> FormatBackendResult> { + if self.next_glyph == self.glyphs.len() { + return Ok(Vec::new()); + } + + let mut end = self.next_glyph; + let mut layer_count = 0; + while end < self.glyphs.len() && end - self.next_glyph < limit.max_glyphs() { + let next_layers = + imported_layer_count(&self.glyphs[end], &self.source_ids_by_master_id); + if end > self.next_glyph && layer_count + next_layers > limit.max_layers() { + break; + } + + layer_count += next_layers; + end += 1; + } + + let glyphs = self.glyphs[self.next_glyph..end] + .par_iter() + .map(|glyph| convert_glyph(glyph, &self.glyph_ids, &self.source_ids_by_master_id)) + .collect::>>()?; + self.next_glyph = end; + Ok(glyphs) + } +} + +pub(crate) fn stream_font(path: &str) -> FormatBackendResult<(Font, GlyphsGlyphStream)> { + let mut glyphs_font = GlyphsFont::load(Path::new(path)) + .map_err(|error| FormatBackendError::Glyphs(error.to_string()))?; + let (header, source_ids_by_master_id) = font_header(&glyphs_font)?; + let glyph_ids = glyphs_font + .glyphs + .values() + .map(|glyph| (glyph.name.to_string(), GlyphId::new())) + .collect(); + let glyphs = std::mem::take(&mut glyphs_font.glyphs) + .into_values() + .collect(); + + Ok(( + header, + GlyphsGlyphStream { + glyph_ids, + glyphs, + source_ids_by_master_id, + next_glyph: 0, + }, + )) +} diff --git a/crates/shift-backends/src/glyphs/mod.rs b/crates/shift-backends/src/glyphs/mod.rs index 1b402b35..90aff51f 100644 --- a/crates/shift-backends/src/glyphs/mod.rs +++ b/crates/shift-backends/src/glyphs/mod.rs @@ -1,5 +1,8 @@ +mod conversion; +mod import; mod reader; +pub(crate) use import::stream_font; pub use reader::GlyphsReader; #[cfg(test)] diff --git a/crates/shift-backends/src/glyphs/reader.rs b/crates/shift-backends/src/glyphs/reader.rs index 00e06b3d..73b532c7 100644 --- a/crates/shift-backends/src/glyphs/reader.rs +++ b/crates/shift-backends/src/glyphs/reader.rs @@ -1,157 +1,13 @@ -use glyphs_reader::{FeatureSnippet, Font as GlyphsFont, NodeType, Shape}; -use shift_font::{ - Anchor, Axis, Component, Contour, FeatureData, Font, Glyph, GlyphLayer, KerningData, - KerningPair, KerningSide, LayerId, Location, MetricKind, Source, Transform, -}; -use std::collections::HashMap; -use std::path::Path; +use shift_font::Font; -use crate::errors::{FormatBackendError, FormatBackendResult}; -use crate::metrics::set_metric_position; -use crate::traits::FontReader; - -const GLYPHS_SIDE1_PREFIX: &str = "@MMK_L_"; -const GLYPHS_SIDE2_PREFIX: &str = "@MMK_R_"; -const UFO_SIDE1_PREFIX: &str = "public.kern1."; -const UFO_SIDE2_PREFIX: &str = "public.kern2."; +use crate::{errors::FormatBackendResult, import::collect_streamed_font, traits::FontReader}; pub struct GlyphsReader; -struct PendingComponent { - layer_id: LayerId, - base_glyph_name: String, - matrix: Transform, -} - impl GlyphsReader { pub fn new() -> Self { Self } - - fn convert_node_type(node_type: NodeType) -> (shift_font::PointType, bool) { - match node_type { - NodeType::Line => (shift_font::PointType::OnCurve, false), - NodeType::LineSmooth => (shift_font::PointType::OnCurve, true), - NodeType::OffCurve => (shift_font::PointType::OffCurve, false), - NodeType::Curve => (shift_font::PointType::OnCurve, false), - NodeType::CurveSmooth => (shift_font::PointType::OnCurve, true), - NodeType::QCurve => (shift_font::PointType::QCurve, false), - NodeType::QCurveSmooth => (shift_font::PointType::QCurve, true), - } - } - - fn convert_features(font: &GlyphsFont) -> FeatureData { - let source = font - .features - .iter() - .filter_map(FeatureSnippet::str_if_enabled) - .collect::>() - .join("\n\n"); - if source.trim().is_empty() { - FeatureData::new() - } else { - FeatureData::from_fea(source) - } - } - - fn convert_kerning(font: &GlyphsFont) -> KerningData { - let mut kerning = KerningData::new(); - - // Build group membership from glyph-level kerning groups. - for glyph in font.glyphs.values() { - if let Some(group) = glyph.right_kern.as_deref() { - let group_name = format!("{UFO_SIDE1_PREFIX}{group}"); - let mut members = kerning - .groups1() - .get(&group_name) - .cloned() - .unwrap_or_default(); - members.push(glyph.name.to_string().into()); - members.sort(); - members.dedup(); - kerning.set_group1(group_name, members); - } - - if let Some(group) = glyph.left_kern.as_deref() { - let group_name = format!("{UFO_SIDE2_PREFIX}{group}"); - let mut members = kerning - .groups2() - .get(&group_name) - .cloned() - .unwrap_or_default(); - members.push(glyph.name.to_string().into()); - members.sort(); - members.dedup(); - kerning.set_group2(group_name, members); - } - } - - // shift-font currently stores static kerning, so we load kerning for default master. - let Some(default_master) = font.masters.get(font.default_master_idx) else { - return kerning; - }; - - let Some(pairs) = font.kerning_ltr.get(&default_master.id) else { - return kerning; - }; - - for ((first, second), value) in pairs { - let first_side = if let Some(group) = first - .strip_prefix(GLYPHS_SIDE1_PREFIX) - .or_else(|| first.strip_prefix(GLYPHS_SIDE2_PREFIX)) - { - KerningSide::Group(format!("{UFO_SIDE1_PREFIX}{group}")) - } else { - KerningSide::Glyph(first.to_string().into()) - }; - - let second_side = if let Some(group) = second - .strip_prefix(GLYPHS_SIDE2_PREFIX) - .or_else(|| second.strip_prefix(GLYPHS_SIDE1_PREFIX)) - { - KerningSide::Group(format!("{UFO_SIDE2_PREFIX}{group}")) - } else { - KerningSide::Glyph(second.to_string().into()) - }; - - kerning.add_pair(KerningPair::new( - first_side, - second_side, - value.into_inner(), - )); - } - - kerning - } - - fn resolve_components( - font: &mut Font, - pending_components: Vec, - ) -> FormatBackendResult<()> { - for pending in pending_components { - let base_glyph_id = - font.glyph_id_by_name(&pending.base_glyph_name) - .ok_or_else(|| { - FormatBackendError::Glyphs(format!( - "component base glyph {:?} does not exist", - pending.base_glyph_name - )) - })?; - let layer = font.layer_mut(pending.layer_id.clone()).ok_or_else(|| { - FormatBackendError::Glyphs(format!( - "component target layer {} does not exist", - pending.layer_id - )) - })?; - layer.add_component(Component::with_matrix( - base_glyph_id, - pending.base_glyph_name, - &pending.matrix, - )); - } - - Ok(()) - } } impl Default for GlyphsReader { @@ -162,168 +18,7 @@ impl Default for GlyphsReader { impl FontReader for GlyphsReader { fn load(&self, path: &str) -> FormatBackendResult { - let glyphs_font = GlyphsFont::load(Path::new(path)) - .map_err(|e| FormatBackendError::Glyphs(e.to_string()))?; - - let mut font = Font::empty(); - - // Metadata and metrics from default master. - if let Some(family_name) = glyphs_font.get_default_name("familyNames") { - font.metadata_mut().family_name = Some(family_name.to_string()); - } - if let Some(default_master) = glyphs_font.masters.get(glyphs_font.default_master_idx) { - font.metadata_mut().style_name = Some(default_master.name.clone()); - } - font.metadata_mut().version_major = Some(glyphs_font.version_major); - font.metadata_mut().version_minor = Some(glyphs_font.version_minor as i32); - font.metrics_mut().units_per_em = glyphs_font.units_per_em as f64; - - // Axes and source locations derived from masters. - let mut axis_ids_by_index = Vec::new(); - for (idx, glyphs_axis) in glyphs_font.axes.iter().enumerate() { - let axis_values: Vec = glyphs_font - .masters - .iter() - .filter_map(|m| m.axes_values.get(idx).map(|v| v.into_inner())) - .collect(); - if axis_values.is_empty() { - continue; - } - - let default = glyphs_font - .masters - .get(glyphs_font.default_master_idx) - .and_then(|m| m.axes_values.get(idx)) - .map(|v| v.into_inner()) - .unwrap_or(axis_values[0]); - let minimum = axis_values.iter().copied().fold(f64::INFINITY, f64::min); - let maximum = axis_values - .iter() - .copied() - .fold(f64::NEG_INFINITY, f64::max); - - let mut axis = Axis::new( - glyphs_axis.tag.clone(), - glyphs_axis.name.clone(), - minimum, - default, - maximum, - ); - axis.set_hidden(glyphs_axis.hidden.unwrap_or(false)); - axis_ids_by_index.push(axis.id()); - font.add_axis(axis)?; - } - - let mut source_by_master_id = HashMap::new(); - for (master_idx, master) in glyphs_font.masters.iter().enumerate() { - let mut location = Location::new(); - for (axis_idx, axis_id) in axis_ids_by_index.iter().enumerate() { - if let Some(value) = master.axes_values.get(axis_idx) { - location.set(axis_id.clone(), value.into_inner()); - } - } - let source_id = font.add_source(Source::new(master.name.clone(), location)); - let metric_definitions = font.metric_definitions().to_vec(); - let source = font - .source_mut(source_id.clone()) - .expect("a newly added source should exist"); - set_metric_position( - &metric_definitions, - source, - MetricKind::Ascender, - master.ascender(), - ); - set_metric_position( - &metric_definitions, - source, - MetricKind::Descender, - master.descender(), - ); - set_metric_position( - &metric_definitions, - source, - MetricKind::CapHeight, - master.cap_height(), - ); - set_metric_position( - &metric_definitions, - source, - MetricKind::XHeight, - master.x_height(), - ); - source.set_italic_angle(master.italic_angle()); - source_by_master_id.insert(master.id.clone(), source_id.clone()); - if master_idx == glyphs_font.default_master_idx { - font.set_default_source_id(source_id); - } - } - - let mut pending_components = Vec::new(); - for glyph in glyphs_font.glyphs.values() { - let mut ir_glyph = Glyph::new(glyph.name.to_string()); - for unicode in glyph.unicode.iter() { - ir_glyph.add_unicode(*unicode); - } - - for layer in &glyph.layers { - let Some(source_id) = source_by_master_id.get(layer.master_id()).cloned() else { - continue; - }; - - let mut ir_layer = - GlyphLayer::with_width(LayerId::new(), source_id, layer.width.into_inner()); - let layer_id = ir_layer.id(); - - for shape in &layer.shapes { - match shape { - Shape::Path(path) => { - let mut contour = Contour::new(); - for node in &path.nodes { - let (point_type, smooth) = Self::convert_node_type(node.node_type); - contour.add_point(node.pt.x, node.pt.y, point_type, smooth); - } - if path.closed { - contour.close(); - } - ir_layer.add_contour(contour); - } - Shape::Component(component) => { - let coeffs = component.transform.as_coeffs(); - pending_components.push(PendingComponent { - layer_id: layer_id.clone(), - base_glyph_name: component.name.to_string(), - matrix: Transform { - xx: coeffs[0], - xy: coeffs[1], - yx: coeffs[2], - yy: coeffs[3], - dx: coeffs[4], - dy: coeffs[5], - }, - }); - } - } - } - - for anchor in &layer.anchors { - let name = if anchor.name.is_empty() { - None - } else { - Some(anchor.name.to_string()) - }; - ir_layer.add_anchor(Anchor::new(name, anchor.pos.x, anchor.pos.y)); - } - - ir_glyph.set_layer(ir_layer); - } - - font.insert_glyph(ir_glyph)?; - } - Self::resolve_components(&mut font, pending_components)?; - - *font.features_mut() = Self::convert_features(&glyphs_font); - *font.kerning_mut() = Self::convert_kerning(&glyphs_font); - - Ok(font) + let (header, mut stream) = super::stream_font(path)?; + collect_streamed_font(header, &mut stream) } } diff --git a/crates/shift-backends/tests/loading.rs b/crates/shift-backends/tests/loading.rs index 58240c71..0956607f 100644 --- a/crates/shift-backends/tests/loading.rs +++ b/crates/shift-backends/tests/loading.rs @@ -212,7 +212,7 @@ fn loads_binary_fonts_with_contours() { } #[test] -fn streams_binary_ufo_and_designspace_without_eager_glyphs() { +fn streams_binary_ufo_designspace_and_glyphs_without_eager_shift_geometry() { let binary_path = mutatorsans_ttf_path(); let binary_bytes = std::fs::read(&binary_path).unwrap(); let binary = skrifa::FontRef::new(&binary_bytes).unwrap(); @@ -222,7 +222,7 @@ fn streams_binary_ufo_and_designspace_without_eager_glyphs() { let streamed_binary = stream_font(&binary_path); assert_eq!(streamed_binary.glyph_count(), expected_binary_glyphs); - for path in [binary_path, mutatorsans_ufo_path()] { + for path in [binary_path, mutatorsans_ufo_path(), homenaje_glyphs_path()] { let eager = load_font(&path); let streamed = stream_font(&path); assert_eq!(streamed.glyph_count(), eager.glyph_count()); @@ -282,6 +282,8 @@ fn streaming_batches_preserve_published_directory_order() { mutatorsans_ttf_path(), mutatorsans_ufo_path(), mutatorsans_designspace_path(), + homenaje_glyphs_path(), + mutatorsans_variable_glyphs_path(), ] { let mut import = FontLoader::new() .stream_font(path.to_str().unwrap()) @@ -317,21 +319,25 @@ fn streaming_batches_preserve_published_directory_order() { #[test] fn streaming_batches_bound_authored_layers_across_sources() { - let path = mutatorsans_designspace_path(); - let mut import = FontLoader::new() - .stream_font(path.to_str().unwrap()) - .unwrap(); - let glyphs = import - .next_batch(shift_backends::ImportBatchLimit::new(512, 4)) - .unwrap(); - let layer_count = glyphs - .iter() - .map(|glyph| glyph.layers().len()) - .sum::(); + for path in [ + mutatorsans_designspace_path(), + mutatorsans_variable_glyphs_path(), + ] { + let mut import = FontLoader::new() + .stream_font(path.to_str().unwrap()) + .unwrap(); + let glyphs = import + .next_batch(shift_backends::ImportBatchLimit::new(512, 4)) + .unwrap(); + let layer_count = glyphs + .iter() + .map(|glyph| glyph.layers().len()) + .sum::(); - assert!(!glyphs.is_empty()); - assert!(glyphs.len() < import.glyph_count()); - assert!(layer_count <= 4 || glyphs.len() == 1); + assert!(!glyphs.is_empty()); + assert!(glyphs.len() < import.glyph_count()); + assert!(layer_count <= 4 || glyphs.len() == 1); + } } #[test] @@ -489,8 +495,8 @@ fn truncated_binary_font_returns_error_instead_of_panicking() { } #[test] -fn loads_glyphs_file_features_kerning_components_and_anchors() { - let font = load_font(&homenaje_glyphs_path()); +fn streams_glyphs_file_features_kerning_components_and_anchors() { + let font = stream_font(&homenaje_glyphs_path()); assert_eq!(font.metadata().family_name.as_deref(), Some("Homenaje")); assert_eq!(font.metrics().units_per_em, 1000.0); @@ -542,8 +548,8 @@ fn loads_glyphs_file_features_kerning_components_and_anchors() { } #[test] -fn loads_variable_glyphs_sources_and_compatible_layers() { - let font = load_font(&mutatorsans_variable_glyphs_path()); +fn streams_variable_glyphs_sources_and_compatible_layers() { + let font = stream_font(&mutatorsans_variable_glyphs_path()); assert!(font.is_variable()); assert_eq!(font.axes().len(), 1); diff --git a/crates/shift-workspace/docs/DOCS.md b/crates/shift-workspace/docs/DOCS.md index 98118897..ce561a20 100644 --- a/crates/shift-workspace/docs/DOCS.md +++ b/crates/shift-workspace/docs/DOCS.md @@ -6,7 +6,7 @@ Backend runtime object for an open Shift font workspace. - **Architecture Invariant:** `FontWorkspace` composes a directory-complete, payload-lazy `shift-font::Font`, the user-selected `shift-source` package, and the working `shift-store` database. - **Architecture Invariant:** Resuming SQLite loads metadata and glyph/layer directory facts only. `acquire_glyphs` performs explicit bounded payload I/O; synchronous `font()` reads never initiate I/O. -- **Architecture Invariant:** TTF/OTF, UFO, and Designspace imports consume bounded `FontImport` batches and write through one `FontImportWriter`; they never construct a complete geometry-resident `Font`. Format readers, MessagePack encoding, BLAKE3 hashing, and independent per-layer compression use Rayon, while SQLite has one transaction owner. +- **Architecture Invariant:** TTF/OTF, UFO, Designspace, and Glyphs imports consume bounded `FontImport` batches and write through one `FontImportWriter`; they never construct a complete geometry-resident Shift `Font`. Glyph conversion, MessagePack encoding, BLAKE3 hashing, and independent per-layer compression use Rayon, while SQLite has one transaction owner. The upstream Glyphs parser still materializes its normalized source model before bounded Shift conversion begins. - **Architecture Invariant:** `LayerResidency` is the sole owner of loaded-layer membership and placeholder replacement. A loaded layer is only a cache of already-committed authored state; apply, undo, and redo reacquire their complete layer read sets before mutation, and `evict_glyphs` replaces only committed layers with directory placeholders. - **Architecture Invariant:** The `.shift` source package path and SQLite working store path are separate inputs. - **Architecture Invariant:** Package recovery policy is not ranked in Rust. `FontWorkspace` exposes package and working-store inspection primitives; the utility process owns binding and lifecycle decisions. @@ -46,7 +46,7 @@ crates/shift-workspace/examples/ `FontWorkspace::create(source_path, store_path, options)` creates a placeholder `.shift` package, opens the working SQLite store, writes initial font metadata, and starts with an empty `shift-font::Font`. -`FontWorkspace::open(path, store_path)` detects `.shift` paths as source packages. TTF/OTF, UFO, and Designspace paths use a metadata/directory-first backend cursor, parse batches of at most 512 glyphs and 1,024 authored layers, parallel-pack/hash/compress those layers, and insert them into a disposable sibling staging database. The legal import transitions are **Staging** (foreign source remains authoritative), **Durable** (stream committed, indexes restored, workspace state written, database synced), then **Published** (closed staging file atomically installed and parent directory synced). Failure before Published removes staging and leaves the previous destination untouched. The returned workspace contains directory placeholders and zero loaded layer payloads. This synchronous API still returns only after finalization; publishing the directory and binary packed-outline grid while import continues requires the separate app import-session boundary. Other supported foreign formats retain the eager compatibility path until they gain a bounded reader. +`FontWorkspace::open(path, store_path)` detects `.shift` paths as source packages. TTF/OTF, UFO, Designspace, Glyphs, and Glyphs package paths use a metadata/directory-first backend cursor, convert batches of at most 512 glyphs and 1,024 authored layers, parallel-pack/hash/compress those layers, and insert them into a disposable sibling staging database. The legal import transitions are **Staging** (foreign source remains authoritative), **Durable** (stream committed, indexes restored, workspace state written, database synced), then **Published** (closed staging file atomically installed and parent directory synced). Failure before Published removes staging and leaves the previous destination untouched. The returned workspace contains directory placeholders and zero loaded layer payloads. Glyphs source syntax is parsed once into the upstream normalized model before its cursor publishes the header and directory; subsequent Shift conversion and persistence remain bounded. This synchronous API still returns only after finalization; publishing the directory and binary packed-outline grid while import continues requires the separate app import-session boundary. Other supported foreign formats retain the eager compatibility path until they gain a bounded reader. `FontWorkspace::save()` succeeds for saved `.shift` workspaces and returns `NeedsSaveAs` for imported workspaces. `save_as(path)` creates a `.shift` package and makes it the save target. diff --git a/crates/shift-workspace/tests/workspace_test.rs b/crates/shift-workspace/tests/workspace_test.rs index 509cdc30..a154ded7 100644 --- a/crates/shift-workspace/tests/workspace_test.rs +++ b/crates/shift-workspace/tests/workspace_test.rs @@ -347,6 +347,81 @@ fn designspace_and_ufo_sources_roundtrip_through_lazy_component_acquisition() { } } +#[test] +fn glyphs_source_imports_directory_first_and_acquires_component_closure() { + let temp = tempfile::tempdir().unwrap(); + let source_path = fixture("fixtures/fonts/Homenaje.glyphs"); + let store_path = temp.path().join("homenaje.sqlite"); + + let mut workspace = FontWorkspace::open(&source_path, &store_path).unwrap(); + let root = workspace + .font() + .glyph_id_by_name("Aacute") + .expect("Homenaje should contain Aacute"); + let closure = workspace + .store() + .referenced_glyph_closure([root.clone()]) + .unwrap(); + let closure_layer_count = closure + .iter() + .map(|glyph_id| { + workspace + .font() + .glyph(glyph_id.clone()) + .unwrap() + .layers() + .len() + }) + .sum::(); + + assert!(closure.len() > 1); + assert_eq!(workspace.loaded_layer_count(), 0); + assert!( + workspace + .font() + .glyph(root.clone()) + .unwrap() + .layers() + .values() + .all(|layer| layer.is_empty()) + ); + + workspace + .acquire_glyphs(std::slice::from_ref(&root), AcquireScope::ComponentClosure) + .unwrap(); + assert_eq!(workspace.loaded_layer_count(), closure_layer_count); + assert_eq!( + workspace + .font() + .glyph(root.clone()) + .unwrap() + .layers() + .values() + .flat_map(|layer| layer.components_iter()) + .count(), + 2 + ); + drop(workspace); + + let mut resumed = FontWorkspace::resume(&store_path).unwrap(); + assert_eq!(resumed.loaded_layer_count(), 0); + resumed + .acquire_glyphs(std::slice::from_ref(&root), AcquireScope::ComponentClosure) + .unwrap(); + assert_eq!(resumed.loaded_layer_count(), closure_layer_count); + assert_eq!( + resumed + .font() + .glyph(root) + .unwrap() + .layers() + .values() + .flat_map(|layer| layer.components_iter()) + .count(), + 2 + ); +} + #[test] fn ufo_source_roundtrips_through_lazy_acquisition() { let temp = tempfile::tempdir().unwrap();