Add multimodal perception, quaternary encoding, and Qalb-aware analysis - #28
Merged
Merged
Conversation
…ding Close all actionable gaps from the MIZAN × QALB-7 gap analysis: Phase 1 — Memory System Fixes: - Add missing search_entities() to KnowledgeGraph (fixes MemoryPyramid layer 4) - Integrate VectorStore into LivingMemory novelty gate for hybrid text+semantic similarity with graceful ChromaDB fallback Phase 2 — Perception Layer Integration: - Flesh out BasirahEngine: JSON-structured LLM output parsing for extracted_text, key_elements, confidence; fix category detection bug - Flesh out NutqEngine: expanded intent detection (~8 types), tone adjustment, language detection (Arabic/Urdu/English) - Wire perception into QCA Engine via process_multimodal() with auditory-first priority (Sam' before Basar per Quran 16:78) - Update perception __init__.py exports - Add /api/perception/analyze endpoint and WebSocket multimodal handler Phase 3 — Perception-Qalb Integration: - Both BasirahEngine and NutqEngine accept qalb_state parameter - Emotional context modulates perception focus and tone calibration Phase 4 — DNA-Inspired Quaternary Encoding: - New quaternary.py module: ACGT encoding, codon chunking, parity-based error detection, Hamming distance verification - Integrate as 4th integrity layer in LawhMahfuz alongside SHA-256/CRC-32 - Database migration for quaternary_checksum column All changes are backward-compatible. Zero test regressions (307/307 passing tests remain passing; 33 pre-existing failures from missing deps unchanged). https://claude.ai/code/session_01JGamoWGB9PN39TCpi3F6J1
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces comprehensive multimodal perception capabilities to the QCA system, including DNA-inspired quaternary encoding for data integrity, Qalb-aware (emotional state) modulation of perception, and enhanced language/intent detection. The changes follow Quranic ordering principles (Sam'/hearing before Basar/sight) and integrate audio, image, and text processing through a unified perception pipeline.
Changes
backend/memory/quaternary.py): DNA-inspired data encoding using ACGT alphabet with per-codon XOR parity, Hamming distance error detection, and quaternary checksums for integrity verificationprocess_multimodal()to QCA engine and ISM layer supporting simultaneous audio transcription, image analysis, and text processingqalb_state) parameter that modulates:NutqEngine._detect_language()): Distinguishes Arabic, Urdu, and English using Unicode character rangesNutqEngine._detect_intent()): Expanded patterns for questions, greetings, farewells, confirmations, commands, and requestsBasirahEngine): JSON-structured parsing of vision results with confidence scores and category validation/api/perception/analyzePOST endpoint and WebSocketmultimodalmessage type for real-time multimodal processingsearch_entities()for LIKE-based entity lookupTest Plan
https://claude.ai/code/session_01JGamoWGB9PN39TCpi3F6J1