Describe the feature
Issue
Currently, faces.py stores face embeddings as JSON strings (TEXT), whereas image_embeddings.py and video_frames.py already store embeddings as raw float32 BLOBs .
Storing embeddings as JSON strings introduces unnecessary overhead in storage size, CPU deserialization cycles, and numeric formatting precision.
Impact
- Storage Reduction: 512-dim
float32 vectors take 2 KB as binary BLOB vs 8–10 KB as JSON text which will be crucial in case of large data.
- Fast Deserialization:
np.frombuffer() eliminates the CPU overhead of json.loads() when querying thousands of faces for clustering and search.
Backward Compatibility
- Startup Data Migration: In
db_create_faces_table(), migrate existing typeof(embeddings) = 'text' rows to BLOB in-place.
- Dual-mode Deserializer: Add a fallback helper supporting both
bytes (np.frombuffer) and legacy str (json.loads).
Proposed Changes
- faces.py: Update insertion and reader queries to write/read BLOB, plus add one-time migration logic.
- test_faces_db.py: Add tests for BLOB storage and legacy JSON backward compatibility.
Add ScreenShots
@maintainer I would like to solve this.
Record
Describe the feature
Issue
Currently,
faces.pystores face embeddings as JSON strings (TEXT), whereasimage_embeddings.pyandvideo_frames.pyalready store embeddings as rawfloat32BLOBs .Storing embeddings as JSON strings introduces unnecessary overhead in storage size, CPU deserialization cycles, and numeric formatting precision.
Impact
float32vectors take 2 KB as binary BLOB vs 8–10 KB as JSON text which will be crucial in case of large data.np.frombuffer()eliminates the CPU overhead ofjson.loads()when querying thousands of faces for clustering and search.Backward Compatibility
db_create_faces_table(), migrate existingtypeof(embeddings) = 'text'rows to BLOB in-place.bytes(np.frombuffer) and legacystr(json.loads).Proposed Changes
Add ScreenShots
@maintainer I would like to solve this.
Record