diff --git a/spp_data_classification/README.rst b/spp_data_classification/README.rst new file mode 100644 index 00000000..07110f3a --- /dev/null +++ b/spp_data_classification/README.rst @@ -0,0 +1,172 @@ +=========================== +OpenSPP Data Classification +=========================== + +.. + !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! + !! This file is generated by oca-gen-addon-readme !! + !! changes will be overwritten. !! + !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! + !! source digest: sha256:4ef670165bef57cbeae9b91f0e6599471121b01cc624df1e6da3bf3c1a6c327d + !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! + +.. |badge1| image:: https://img.shields.io/badge/maturity-Alpha-red.png + :target: https://odoo-community.org/page/development-status + :alt: Alpha +.. |badge2| image:: https://img.shields.io/badge/license-LGPL--3-blue.png + :target: http://www.gnu.org/licenses/lgpl-3.0-standalone.html + :alt: License: LGPL-3 +.. |badge3| image:: https://img.shields.io/badge/github-OpenSPP%2FOpenSPP2-lightgray.png?logo=github + :target: https://github.com/OpenSPP/OpenSPP2/tree/19.0/spp_data_classification + :alt: OpenSPP/OpenSPP2 + +|badge1| |badge2| |badge3| + +Data sensitivity classification registry for OpenSPP. Defines +classification levels, maps model fields to those levels with PII +categorization, and provides regex/CEL auto-detection patterns. This is +the foundation other modules consume to decide what to encrypt, mask, or +audit. + +Key Capabilities +~~~~~~~~~~~~~~~~ + +- Define sensitivity levels (PUBLIC, INTERNAL, CONFIDENTIAL, RESTRICTED) + with encryption, masking, audit, consent, and retention policy flags +- Map model fields to classification levels with PII categorization + (direct identifiers, quasi-identifiers, biometric, health, financial, + etc.) +- Flag classified fields as PII (``is_pii``, derived from the assigned + PII category) so consumers can query them +- Auto-classify fields using regex (or CEL) patterns that match field + names and metadata, via a manually-invoked scan + +Key Models +~~~~~~~~~~ + ++-----------------------------------+----------------------------------+ +| Model | Description | ++===================================+==================================+ +| ``spp.data.classification.level`` | Sensitivity levels with | +| | encryption, masking, and | +| | retention policies | ++-----------------------------------+----------------------------------+ +| ``spp.field.classification`` | Maps specific model fields to | +| | classification levels and PII | +| | category | ++-----------------------------------+----------------------------------+ +| ``spp.classification.pattern`` | Auto-detection patterns using | +| | regex or CEL expressions | ++-----------------------------------+----------------------------------+ + +Configuration +~~~~~~~~~~~~~ + +After installing: + +1. Navigate to **Settings > Data Classification > Classification + Levels** to review pre-loaded sensitivity levels (PUBLIC, INTERNAL, + CONFIDENTIAL, RESTRICTED) +2. Review **Detection Patterns** to customize auto-classification rules + for your organization +3. Map fields under **Field Classifications** + +UI Location +~~~~~~~~~~~ + +- **Menu**: Settings > Data Classification +- **Classification Levels**: Settings > Data Classification > + Classification Levels +- **Field Classifications**: Settings > Data Classification > Field + Classifications +- **Detection Patterns**: Settings > Data Classification > Detection + Patterns + +Security +~~~~~~~~ + ++----------------------------------------------------------+----------------------------------+ +| Group | Access | ++==========================================================+==================================+ +| ``spp_data_classification.group_classification_admin`` | Full CRUD on levels, patterns, | +| | and field classifications | ++----------------------------------------------------------+----------------------------------+ +| ``spp_data_classification.group_classification_manager`` | Read/Write/Create field | +| | classifications (no delete); | +| | read-only on levels and patterns | ++----------------------------------------------------------+----------------------------------+ +| ``base.group_user`` | Read-only access to | +| | classifications | ++----------------------------------------------------------+----------------------------------+ + +The ``pii_full_access`` and ``pii_confidential_access`` groups are +defined for downstream masking/access-control consumers. + +Extension Points +~~~~~~~~~~~~~~~~ + +- Query ``spp.field.classification`` to discover classified/PII fields + per model (e.g. ``get_fields_requiring_encryption()``) +- Override ``spp.classification.pattern.matches_field()`` to implement + custom detection logic +- Add custom PII categories by extending the ``pii_category`` selection + field + +Deferred subsystems +~~~~~~~~~~~~~~~~~~~ + +PII-aware enforcement (automatic read-time masking + the +``pii=``/``classification=`` field attributes), GDPR DSAR handling, +data-retention scheduling, and consent integration are part of this +module's design but are delivered in a separate governance change. This +module ships the classification registry only. + +Dependencies +~~~~~~~~~~~~ + +``base``, ``spp_security`` + +.. IMPORTANT:: + This is an alpha version, the data model and design can change at any time without warning. + Only for development or testing purpose, do not use in production. + +**Table of contents** + +.. contents:: + :local: + +Bug Tracker +=========== + +Bugs are tracked on `GitHub Issues `_. +In case of trouble, please check there if your issue has already been reported. +If you spotted it first, help us to smash it by providing a detailed and welcomed +`feedback `_. + +Do not contact contributors directly about support or help with technical issues. + +Credits +======= + +Authors +------- + +* OpenSPP.org + +Maintainers +----------- + +.. |maintainer-jeremi| image:: https://github.com/jeremi.png?size=40px + :target: https://github.com/jeremi + :alt: jeremi +.. |maintainer-gonzalesedwin1123| image:: https://github.com/gonzalesedwin1123.png?size=40px + :target: https://github.com/gonzalesedwin1123 + :alt: gonzalesedwin1123 + +Current maintainers: + +|maintainer-jeremi| |maintainer-gonzalesedwin1123| + +This module is part of the `OpenSPP/OpenSPP2 `_ project on GitHub. + +You are welcome to contribute. \ No newline at end of file diff --git a/spp_data_classification/__init__.py b/spp_data_classification/__init__.py new file mode 100644 index 00000000..d3361032 --- /dev/null +++ b/spp_data_classification/__init__.py @@ -0,0 +1,2 @@ +# Part of OpenSPP. See LICENSE file for full copyright and licensing details. +from . import models diff --git a/spp_data_classification/__manifest__.py b/spp_data_classification/__manifest__.py new file mode 100644 index 00000000..1c774a2b --- /dev/null +++ b/spp_data_classification/__manifest__.py @@ -0,0 +1,37 @@ +# Part of OpenSPP. See LICENSE file for full copyright and licensing details. +# pylint: disable=pointless-statement +{ + "name": "OpenSPP Data Classification", + "summary": "Data sensitivity classification and PII protection for OpenSPP", + "category": "OpenSPP/Configuration", + "version": "19.0.1.0.0", + "sequence": 1, + "author": "OpenSPP.org", + "website": "https://github.com/OpenSPP/OpenSPP2", + "license": "LGPL-3", + "development_status": "Alpha", + "maintainers": ["jeremi", "gonzalesedwin1123"], + "depends": [ + "base", + "spp_security", + ], + "external_dependencies": { + "python": [], + }, + "data": [ + "security/security_groups.xml", + "security/ir.model.access.csv", + "data/classification_levels.xml", + "data/detection_patterns.xml", + "views/field_classification_views.xml", + "views/classification_level_views.xml", + "views/classification_pattern_views.xml", + "views/menu.xml", + ], + "assets": {}, + "demo": [], + "images": [], + "application": False, + "installable": True, + "auto_install": False, +} diff --git a/spp_data_classification/data/classification_levels.xml b/spp_data_classification/data/classification_levels.xml new file mode 100644 index 00000000..41f09f5f --- /dev/null +++ b/spp_data_classification/data/classification_levels.xml @@ -0,0 +1,71 @@ + + + + + + Public + PUBLIC + 10 + 10 + Data that can be freely shared. No restrictions on access or distribution. + False + False + False + False + 0 + none + + + + Internal + INTERNAL + 20 + 4 + Data for internal use only. Should not be shared externally without authorization. + False + False + True + False + 0 + none + + + + Confidential + CONFIDENTIAL + 30 + 3 + Sensitive personal data. Access restricted to authorized personnel. Recommended encryption. + False + True + True + True + + 2555 + anonymize + + + + Restricted + RESTRICTED + 40 + 1 + Highly sensitive data (IDs, financial info). Mandatory encryption. Strict access control. + True + True + True + True + True + + 2555 + delete + + diff --git a/spp_data_classification/data/detection_patterns.xml b/spp_data_classification/data/detection_patterns.xml new file mode 100644 index 00000000..2b26c1c5 --- /dev/null +++ b/spp_data_classification/data/detection_patterns.xml @@ -0,0 +1,190 @@ + + + + + + + + + National/Government ID + (national|passport|ssn|social.?security|tax|identity|citizen).*id + + direct_id + 95 + ****-****-#### + blind_index + Matches national_id, passport_id, ssn_id, tax_id, etc. + + + + Bank Account Numbers + (account|iban|bank|routing|swift).*n?(o|um|umber)? + + financial + 95 + ****-****-#### + blind_index + Matches account_number, iban, bank_no, routing_number, etc. + + + + Biometric Data + (fingerprint|biometric|face.?id|iris|retina|voice.?print) + + biometric + 95 + none + Matches fingerprint, biometric_data, face_id, etc. + + + + Health Information + (health|medical|diagnosis|disability|condition|treatment|medication) + + health + 90 + none + Matches health_status, medical_condition, disability_type, etc. + + + + Political/Religious + (religion|religious|ethnic|ethnicity|race|racial|political|union.?member) + + political + 90 + none + GDPR Article 9 special categories + + + + + + Personal Names + (family|given|first|last|middle|maiden|sur).*name|^name$ + + direct_id + 80 + phonetic + Matches family_name, given_name, first_name, surname, etc. + + + + Birth Date + (birth|dob|date.?of.?birth|born) + + quasi_id + 75 + range + Matches birthdate, dob, date_of_birth, birth_date, etc. + + + + Phone Numbers + (phone|mobile|cell|tel|fax|whatsapp) + + contact + 70 + ***-***-#### + partial_index + Matches phone, mobile_phone, telephone, cell_number, etc. + + + + Physical Address + (address|street|city|postal|zip|province|district|village|barangay) + + contact + 70 + blind_index + Matches address, street_address, postal_code, city, etc. + + + + Income/Salary + (income|salary|wage|earning|revenue) + + financial + 70 + range + Matches income, salary, monthly_wage, earnings, etc. + + + + GPS/Location + (gps|latitude|longitude|geo.?location|coordinates) + + location + 65 + range + Matches gps_coordinates, latitude, longitude, etc. + + + + + + Email Address + e?.?mail + + contact + 50 + blind_index + Matches email, e_mail, email_address, etc. + + + + Gender + (gender|sex) + + quasi_id + 40 + full + Matches gender, sex, gender_id, etc. + + + + Marital Status + (marital|civil).?(status|state) + + quasi_id + 40 + full + Matches marital_status, civil_status, etc. + + + + Occupation + (occupation|profession|job|employment) + + quasi_id + 40 + full + Matches occupation, profession, job_title, etc. + + diff --git a/spp_data_classification/models/__init__.py b/spp_data_classification/models/__init__.py new file mode 100644 index 00000000..9bb81ced --- /dev/null +++ b/spp_data_classification/models/__init__.py @@ -0,0 +1,4 @@ +# Part of OpenSPP. See LICENSE file for full copyright and licensing details. +from . import classification_level +from . import field_classification +from . import classification_pattern diff --git a/spp_data_classification/models/classification_level.py b/spp_data_classification/models/classification_level.py new file mode 100644 index 00000000..55565d21 --- /dev/null +++ b/spp_data_classification/models/classification_level.py @@ -0,0 +1,144 @@ +# Part of OpenSPP. See LICENSE file for full copyright and licensing details. +import logging + +from odoo import _, api, fields, models +from odoo.exceptions import UserError + +_logger = logging.getLogger(__name__) + + +class DataClassificationLevel(models.Model): + """Data sensitivity classification levels. + + Defines the sensitivity levels for PII fields (e.g., PUBLIC, INTERNAL, + CONFIDENTIAL, RESTRICTED) along with policies that apply at each level. + + Usage: + Levels are typically defined via XML data and referenced by + field classifications. Each level carries policy flags that + determine how data should be handled. + """ + + _name = "spp.data.classification.level" + _description = "Data Classification Level" + _order = "sequence, id" + + name = fields.Char( + required=True, + translate=True, + help="Human-readable name for this classification level", + ) + code = fields.Char( + required=True, + help="Technical code (e.g., PUBLIC, INTERNAL, CONFIDENTIAL, RESTRICTED)", + ) + sequence = fields.Integer( + default=10, + help="Higher sequence = more sensitive. Used for access control comparison.", + ) + color = fields.Integer( + default=0, + help="Color index for UI display", + ) + description = fields.Text( + translate=True, + help="Detailed description of when to use this classification", + ) + active = fields.Boolean(default=True) + + # === Policy Flags === + is_requires_encryption = fields.Boolean( + string="Requires Encryption", + default=False, + help="Fields at this level must be encrypted at rest", + ) + is_requires_masking = fields.Boolean( + string="Requires Masking", + default=False, + help="Fields displayed with masking by default (e.g., ****1234)", + ) + is_requires_audit = fields.Boolean( + string="Requires Audit", + default=False, + help="All access to fields at this level is logged", + ) + is_requires_consent = fields.Boolean( + string="Requires Consent", + default=False, + help="Explicit consent required before collection/processing", + ) + is_requires_purpose_limitation = fields.Boolean( + string="Requires Purpose Limitation", + default=False, + help="Data can only be used for stated purposes", + ) + + # === Retention === + retention_days = fields.Integer( + string="Retention Period (Days)", + default=0, + help="Auto-archive/delete after N days. 0 = indefinite retention.", + ) + retention_action = fields.Selection( + [ + ("none", "No Action"), + ("anonymize", "Anonymize"), + ("archive", "Archive"), + ("delete", "Delete"), + ], + string="Retention Action", + default="none", + help="Action to take when retention period expires", + ) + + # === Access Control === + min_group_id = fields.Many2one( + "res.groups", + string="Minimum Access Group", + help="Minimum security group required to view unmasked data at this level", + ) + + _unique_code = models.Constraint( + "UNIQUE(code)", + "Classification code must be unique", + ) + + def unlink(self): + """Prevent deletion of levels that are in use.""" + for record in self: + field_count = self.env["spp.field.classification"].search_count([("classification_id", "=", record.id)]) + if field_count > 0: + raise UserError( + _( + "Cannot delete classification level '%(name)s' - " + "it is used by %(count)d field classification(s)." + ) + % {"name": record.name, "count": field_count} + ) + return super().unlink() + + @api.model + def get_level_by_code(self, code): + """Get classification level by code. + + Args: + code: The classification code (e.g., 'RESTRICTED') + + Returns: + recordset: The matching classification level or empty recordset + """ + return self.search([("code", "=", code)], limit=1) + + def is_more_sensitive_than(self, other_level): + """Compare sensitivity between two levels. + + Args: + other_level: Another classification level to compare against + + Returns: + bool: True if this level is more sensitive (higher sequence) + """ + self.ensure_one() + if not other_level: + return True + return self.sequence > other_level.sequence diff --git a/spp_data_classification/models/classification_pattern.py b/spp_data_classification/models/classification_pattern.py new file mode 100644 index 00000000..8c6f2985 --- /dev/null +++ b/spp_data_classification/models/classification_pattern.py @@ -0,0 +1,481 @@ +# Part of OpenSPP. See LICENSE file for full copyright and licensing details. +import functools +import logging +import re +from types import SimpleNamespace + +from odoo import _, api, fields, models +from odoo.exceptions import ValidationError + +_logger = logging.getLogger(__name__) + + +@functools.lru_cache(maxsize=256) +def _compile_regex(pattern_str): + """Compile regex pattern with caching. + + Uses functools.lru_cache for thread-safe caching. + """ + try: + return re.compile(pattern_str, re.IGNORECASE) + except re.error as e: + _logger.warning("Invalid regex pattern '%s': %s", pattern_str, e) + return None + + +class ClassificationPattern(models.Model): + """Auto-detection patterns for PII fields. + + Supports two matching modes: + 1. Regex (default): Simple pattern matching against field names + 2. CEL: Complex expressions using field metadata (type, name, model, etc.) + + CEL expressions have access to: + - field.name: Field name (str) + - field.type: Field type like 'char', 'integer', etc. (str) + - field.store: Whether field is stored in DB (bool) + - field.required: Whether field is required (bool) + - model.model: Model technical name like 'res.partner' (str) + - model.name: Model display name (str) + + Example CEL expressions: + - field.type == 'char' && field.name.endsWith('_id') + - model.model.startsWith('spp.') && field.name matches 'national.*' + - field.type in ['char', 'text'] && field.store + + Usage: + Patterns are evaluated in priority order (higher = first). + When a field matches, it's automatically classified. + """ + + _name = "spp.classification.pattern" + _description = "Classification Auto-Detection Pattern" + _order = "priority desc, id" + + name = fields.Char( + required=True, + help="Descriptive name for this pattern", + ) + + # === Matching Mode === + match_mode = fields.Selection( + [ + ("regex", "Regex Pattern"), + ("cel", "CEL Expression"), + ], + string="Match Mode", + default="regex", + required=True, + help="How to match fields: Regex for simple name patterns, CEL for complex rules", + ) + + # Regex pattern (legacy/simple mode) + pattern = fields.Char( + help="Regex pattern to match field names. Case-insensitive. " + "E.g., '(national|passport|tax).*id' matches 'national_id', 'passport_id'", + ) + + # CEL expression (advanced mode) + cel_expression = fields.Text( + string="CEL Expression", + help="CEL expression for complex field matching. " + "Use 'field' for field metadata and 'model' for model metadata.\n" + "Example: field.type == 'char' && field.name matches 'national.*'", + ) + + classification_id = fields.Many2one( + "spp.data.classification.level", + string="Classification Level", + required=True, + ondelete="restrict", + help="Classification level to apply when pattern matches", + ) + pii_category = fields.Selection( + [ + ("direct_id", "Direct Identifier"), + ("quasi_id", "Quasi-Identifier"), + ("sensitive", "Sensitive Personal Data"), + ("financial", "Financial Information"), + ("contact", "Contact Information"), + ("biometric", "Biometric Data"), + ("health", "Health Information"), + ("political", "Political/Religious/Union"), + ("genetic", "Genetic Data"), + ("location", "Location Data"), + ], + string="PII Category", + help="PII category to assign when pattern matches", + ) + priority = fields.Integer( + default=50, + help="Higher priority patterns are evaluated first (0-100)", + ) + active = fields.Boolean(default=True) + + # === Default Handling === + default_mask_pattern = fields.Char( + string="Default Mask Pattern", + help="Mask pattern to apply (e.g., '****-****-####')", + ) + default_search_strategy = fields.Selection( + [ + ("none", "No Search Allowed"), + ("blind_index", "Blind Index (Exact Match)"), + ("partial_index", "Partial Index (Last N chars)"), + ("phonetic", "Phonetic Search (Names)"), + ("range", "Range Search (Dates)"), + ("full", "Full Search (Decrypted)"), + ], + string="Default Search Strategy", + default="blind_index", + ) + + # === Scope === + apply_to_model_pattern = fields.Char( + string="Model Pattern", + help="Optional: Only apply to models matching this pattern. " + "E.g., 'spp\\..*' for all OpenSPP models. Empty = all models.", + ) + + notes = fields.Text( + string="Notes", + help="Documentation about what this pattern matches", + ) + + @api.constrains("match_mode", "pattern", "cel_expression") + def _check_pattern_or_expression(self): + """Ensure either pattern or CEL expression is provided based on mode.""" + for record in self: + if record.match_mode == "regex": + if not record.pattern: + raise ValidationError(_("Regex pattern is required when using Regex match mode")) + elif record.match_mode == "cel": + if not record.cel_expression: + raise ValidationError(_("CEL expression is required when using CEL match mode")) + + @api.model + def _get_compiled_pattern(self, pattern_str): + """Get compiled regex pattern with caching. + + Uses module-level lru_cache for thread-safe caching. + """ + return _compile_regex(pattern_str) + + def matches_field(self, field_name, model_name=None, field_obj=None, model_obj=None): + """Check if this pattern matches a field. + + Args: + field_name: The field name to check + model_name: Optional model name for scope filtering + field_obj: Optional ir.model.fields record for CEL evaluation + model_obj: Optional ir.model record for CEL evaluation + + Returns: + bool: True if pattern matches + """ + self.ensure_one() + + # Check model scope if specified (applies to both modes) + if self.apply_to_model_pattern and model_name: + model_pattern = self._get_compiled_pattern(self.apply_to_model_pattern) + if model_pattern and not model_pattern.search(model_name): + return False + + # Dispatch to appropriate matcher based on mode + if self.match_mode == "cel": + return self._matches_field_cel(field_name, model_name, field_obj, model_obj) + else: + return self._matches_field_regex(field_name) + + def _matches_field_regex(self, field_name): + """Check if field name matches regex pattern. + + Args: + field_name: The field name to check + + Returns: + bool: True if pattern matches + """ + if not self.pattern: + return False + + pattern = self._get_compiled_pattern(self.pattern) + if pattern: + return bool(pattern.search(field_name)) + return False + + def _matches_field_cel(self, field_name, model_name=None, field_obj=None, model_obj=None): + """Check if field matches CEL expression. + + Builds a context with field and model metadata, then evaluates + the CEL expression. + + Args: + field_name: The field name + model_name: The model name + field_obj: Optional ir.model.fields record + model_obj: Optional ir.model record + + Returns: + bool: True if CEL expression evaluates to True + """ + if not self.cel_expression: + return False + + # Check if CEL service is available + cel_service = self.env.get("spp.cel.service") + if not cel_service: + _logger.warning( + "CEL service not available for pattern '%s'. Install spp_cel_domain module to use CEL patterns.", + self.name, + ) + return False + + # Build context for CEL evaluation + context = self._build_cel_context(field_name, model_name, field_obj, model_obj) + + try: + result = cel_service.evaluate_expression(self.cel_expression, context) + return bool(result) + except (ValueError, TypeError, KeyError, AttributeError) as e: + _logger.warning( + "CEL evaluation failed for pattern '%s': %s", + self.name, + str(e), + ) + return False + except SyntaxError as e: + _logger.warning( + "CEL syntax error in pattern '%s': %s", + self.name, + str(e), + ) + return False + + def _build_cel_context(self, field_name, model_name=None, field_obj=None, model_obj=None): + """Build CEL evaluation context with field and model metadata. + + Args: + field_name: The field name + model_name: The model name + field_obj: Optional ir.model.fields record + model_obj: Optional ir.model record + + Returns: + dict: Context for CEL evaluation + """ + # Build field context + field_ctx = { + "name": field_name, + "type": "unknown", + "store": True, + "required": False, + "readonly": False, + "translate": False, + } + + if field_obj: + field_ctx.update( + { + "type": field_obj.ttype, + "store": field_obj.store, + "required": field_obj.required, + "readonly": field_obj.readonly, + "translate": field_obj.translate, + "help": field_obj.help or "", + "relation": field_obj.relation or "", + } + ) + + # Build model context + model_ctx = { + "model": model_name or "", + "name": "", + "transient": False, + } + + if model_obj: + model_ctx.update( + { + "model": model_obj.model, + "name": model_obj.name, + "transient": model_obj.transient, + } + ) + + # Use SimpleNamespace for dot access in CEL expressions + return { + "field": SimpleNamespace(**field_ctx), + "model": SimpleNamespace(**model_ctx), + } + + @api.model + def find_matching_pattern(self, field_name, model_name=None, field_obj=None, model_obj=None): + """Find the first (highest priority) matching pattern. + + Args: + field_name: The field name to check + model_name: Optional model name for scope filtering + field_obj: Optional ir.model.fields record for CEL evaluation + model_obj: Optional ir.model record for CEL evaluation + + Returns: + recordset: The matching pattern or empty recordset + """ + patterns = self.search([("active", "=", True)]) + for pattern in patterns: + if pattern.matches_field(field_name, model_name, field_obj, model_obj): + return pattern + return self.browse() + + @api.model + def auto_classify_field(self, model_name, field_name, field_obj=None, model_obj=None): + """Attempt to auto-classify a field based on patterns. + + Args: + model_name: The model name + field_name: The field name + field_obj: Optional ir.model.fields record for CEL evaluation + model_obj: Optional ir.model record for CEL evaluation + + Returns: + recordset: The created classification or empty if no match + """ + pattern = self.find_matching_pattern(field_name, model_name, field_obj, model_obj) + if not pattern: + return self.env["spp.field.classification"].browse() + + _logger.info( + "Auto-classifying %s.%s as %s (pattern: %s, mode: %s)", + model_name, + field_name, + pattern.classification_id.code, + pattern.name, + pattern.match_mode, + ) + + return self.env["spp.field.classification"].ensure_classification( + model_name=model_name, + field_name=field_name, + level_code=pattern.classification_id.code, + source="auto", + pattern_id=pattern.id, + pii_category=pattern.pii_category, + mask_pattern=pattern.default_mask_pattern, + search_strategy=pattern.default_search_strategy, + ) + + @api.model + def scan_model_fields(self, model_name, skip_classified=True): + """Scan all fields of a model and auto-classify matches. + + Args: + model_name: The model to scan + skip_classified: Skip fields that already have classifications + + Returns: + list: List of (field_name, classification) tuples + """ + model = self.env["ir.model"].search([("model", "=", model_name)], limit=1) + if not model: + _logger.warning("Model not found: %s", model_name) + return [] + + results = [] + for field in model.field_id: + # Skip non-data fields + if field.ttype in ("one2many", "many2many"): + continue + + # Skip if already classified + if skip_classified: + existing = self.env["spp.field.classification"].get_classification(model_name, field.name) + if existing: + continue + + # Try to auto-classify (pass field and model objects for CEL evaluation) + classification = self.auto_classify_field(model_name, field.name, field, model) + if classification: + results.append((field.name, classification)) + + return results + + @api.model + def scan_all_models(self, model_pattern=None, skip_classified=True): + """Scan all models (or matching pattern) for PII fields. + + Args: + model_pattern: Optional regex pattern to filter models + skip_classified: Skip fields that already have classifications + + Returns: + dict: {model_name: [(field_name, classification), ...]} + """ + domain = [("transient", "=", False)] + models = self.env["ir.model"].search(domain) + + if model_pattern: + pattern = self._get_compiled_pattern(model_pattern) + if pattern: + models = models.filtered(lambda m: pattern.search(m.model)) + + results = {} + for model in models: + model_results = self.scan_model_fields(model.model, skip_classified) + if model_results: + results[model.model] = model_results + + return results + + def action_test_pattern(self): + """Test this pattern against a sample model to preview matches. + + Returns: + Action to display notification with test results + """ + self.ensure_one() + + # Use res.partner as sample model + test_model = self.env["ir.model"].search([("model", "=", "res.partner")], limit=1) + if not test_model: + return { + "type": "ir.actions.client", + "tag": "display_notification", + "params": { + "title": _("Test Failed"), + "message": _("Cannot find res.partner model for testing"), + "type": "warning", + "sticky": False, + }, + } + + # Test against all fields + matching_fields = [] + for field in test_model.field_id: + if field.ttype in ("one2many", "many2many"): + continue + if self.matches_field(field.name, test_model.model, field, test_model): + matching_fields.append(field.name) + + if matching_fields: + message = _("Pattern matches %d fields:\n%s") % ( + len(matching_fields), + ", ".join(sorted(matching_fields)[:20]), + ) + if len(matching_fields) > 20: + message += _(", ... and %d more") % (len(matching_fields) - 20) + msg_type = "success" + else: + message = _("Pattern does not match any fields in res.partner") + msg_type = "warning" + + return { + "type": "ir.actions.client", + "tag": "display_notification", + "params": { + "title": _("Pattern Test Results"), + "message": message, + "type": msg_type, + "sticky": True, + }, + } diff --git a/spp_data_classification/models/field_classification.py b/spp_data_classification/models/field_classification.py new file mode 100644 index 00000000..34151a60 --- /dev/null +++ b/spp_data_classification/models/field_classification.py @@ -0,0 +1,296 @@ +# Part of OpenSPP. See LICENSE file for full copyright and licensing details. +import logging + +from odoo import _, api, fields, models +from odoo.exceptions import ValidationError + +_logger = logging.getLogger(__name__) + + +class FieldClassification(models.Model): + """Maps model fields to classification levels. + + This is the central registry of all PII/sensitive fields in the system. + Each record links a specific field on a specific model to a classification + level and defines handling policies. + + Usage: + Classifications can be: + 1. Auto-detected from field name patterns + 2. Declared via XML data files + 3. Set via pii=True field attribute + 4. Manually configured via UI + """ + + _name = "spp.field.classification" + _description = "Field Classification" + _rec_name = "display_name" + _order = "model_id, field_id" + + model_id = fields.Many2one( + "ir.model", + string="Model", + required=True, + ondelete="cascade", + index=True, + help="The model containing the classified field", + ) + model_name = fields.Char( + related="model_id.model", + store=True, + index=True, + ) + field_id = fields.Many2one( + "ir.model.fields", + string="Field", + required=True, + ondelete="cascade", + domain="[('model_id', '=', model_id)]", + index=True, + help="The field being classified", + ) + field_name = fields.Char( + related="field_id.name", + store=True, + index=True, + ) + classification_id = fields.Many2one( + "spp.data.classification.level", + string="Classification Level", + required=True, + ondelete="restrict", + index=True, + help="The sensitivity level of this field", + ) + display_name = fields.Char( + compute="_compute_display_name", + store=True, + ) + + is_pii = fields.Boolean( + string="Is PII", + compute="_compute_is_pii", + store=True, + help="Set automatically when a PII category is assigned.", + ) + + # === PII Categorization === + pii_category = fields.Selection( + [ + ("direct_id", "Direct Identifier"), + ("quasi_id", "Quasi-Identifier"), + ("sensitive", "Sensitive Personal Data"), + ("financial", "Financial Information"), + ("contact", "Contact Information"), + ("biometric", "Biometric Data"), + ("health", "Health Information"), + ("political", "Political/Religious/Union"), + ("genetic", "Genetic Data"), + ("location", "Location Data"), + ], + string="PII Category", + help="Category of personally identifiable information", + ) + + # === Compliance Flags === + is_gdpr_special_category = fields.Boolean( + string="GDPR Special Category", + default=False, + help="GDPR Article 9 special category data (race, health, politics, etc.)", + ) + cross_border_restricted = fields.Boolean( + string="Cross-Border Restricted", + default=False, + help="Cannot be transferred outside jurisdiction without safeguards", + ) + child_data = fields.Boolean( + string="May Contain Child Data", + default=False, + help="Field may contain data about minors (requires extra protection)", + ) + + # === Handling Configuration === + mask_pattern = fields.Char( + string="Mask Pattern", + help="Display mask pattern. Use * for hidden, # for visible. E.g., '****-****-####' shows last 4 characters.", + ) + search_strategy = fields.Selection( + [ + ("none", "No Search Allowed"), + ("blind_index", "Blind Index (Exact Match)"), + ("partial_index", "Partial Index (Last N chars)"), + ("phonetic", "Phonetic Search (Names)"), + ("range", "Range Search (Dates)"), + ("full", "Full Search (Decrypted)"), + ], + string="Search Strategy", + default="blind_index", + help="How this field can be searched when encrypted", + ) + + # === Legal Basis === + legal_basis = fields.Selection( + [ + ("consent", "Consent"), + ("contract", "Contractual Necessity"), + ("legal", "Legal Obligation"), + ("vital", "Vital Interests"), + ("public", "Public Interest"), + ("legitimate", "Legitimate Interest"), + ], + string="Legal Basis", + help="Legal basis for processing this data (GDPR Article 6)", + ) + data_source = fields.Char( + string="Data Source", + help="Where this data originates (e.g., 'Government ID Document')", + ) + + # === Source Tracking === + source = fields.Selection( + [ + ("auto", "Auto-detected"), + ("attribute", "Field Attribute"), + ("xml", "XML Data"), + ("manual", "Manual"), + ], + string="Classification Source", + default="manual", + readonly=True, + help="How this classification was created", + ) + pattern_id = fields.Many2one( + "spp.classification.pattern", + string="Matched Pattern", + readonly=True, + help="If auto-detected, the pattern that matched", + ) + + notes = fields.Text( + string="Notes", + help="Additional notes about this classification", + ) + + _unique_model_field = models.Constraint( + "UNIQUE(model_id, field_id)", + "Each field can only have one classification", + ) + + @api.depends("model_id", "field_id", "classification_id") + def _compute_display_name(self): + for record in self: + if record.model_id and record.field_id: + record.display_name = ( + f"{record.model_id.model}.{record.field_id.name} [{record.classification_id.code or 'N/A'}]" + ) + else: + record.display_name = _("New Classification") + + @api.depends("pii_category") + def _compute_is_pii(self): + for record in self: + record.is_pii = bool(record.pii_category) + + @api.constrains("model_id", "field_id") + def _check_field_belongs_to_model(self): + for record in self: + if record.field_id and record.model_id: + if record.field_id.model_id != record.model_id: + raise ValidationError( + _("Field '%(field)s' does not belong to model '%(model)s'") + % { + "field": record.field_id.name, + "model": record.model_id.model, + } + ) + + @api.model + def get_classification(self, model_name, field_name): + """Get classification for a specific model field. + + Args: + model_name: The model name (e.g., 'res.partner') + field_name: The field name (e.g., 'national_id') + + Returns: + recordset: The field classification or empty recordset + """ + return self.search( + [ + ("model_name", "=", model_name), + ("field_name", "=", field_name), + ], + limit=1, + ) + + @api.model + def get_model_classifications(self, model_name): + """Get all classifications for a model. + + Args: + model_name: The model name (e.g., 'res.partner') + + Returns: + recordset: All field classifications for the model + """ + return self.search([("model_name", "=", model_name)]) + + @api.model + def get_fields_requiring_encryption(self, model_name=None): + """Get all fields that require encryption. + + Args: + model_name: Optional model name to filter by + + Returns: + recordset: Field classifications requiring encryption + """ + domain = [("classification_id.is_requires_encryption", "=", True)] + if model_name: + domain.append(("model_name", "=", model_name)) + return self.search(domain) + + @api.model + def ensure_classification(self, model_name, field_name, level_code, source="manual", **kwargs): + """Ensure a field classification exists, creating if needed. + + Args: + model_name: The model name + field_name: The field name + level_code: Classification level code (e.g., 'RESTRICTED') + source: Source of classification + **kwargs: Additional field values + + Returns: + recordset: The existing or created classification + """ + existing = self.get_classification(model_name, field_name) + if existing: + return existing + + model = self.env["ir.model"].search([("model", "=", model_name)], limit=1) + if not model: + _logger.warning("Model not found for classification: %s", model_name) + return self.browse() + + field = self.env["ir.model.fields"].search( + [("model_id", "=", model.id), ("name", "=", field_name)], + limit=1, + ) + if not field: + _logger.warning("Field not found for classification: %s.%s", model_name, field_name) + return self.browse() + + level = self.env["spp.data.classification.level"].get_level_by_code(level_code) + if not level: + _logger.warning("Classification level not found: %s", level_code) + return self.browse() + + values = { + "model_id": model.id, + "field_id": field.id, + "classification_id": level.id, + "source": source, + **kwargs, + } + return self.create(values) diff --git a/spp_data_classification/pyproject.toml b/spp_data_classification/pyproject.toml new file mode 100644 index 00000000..4231d0cc --- /dev/null +++ b/spp_data_classification/pyproject.toml @@ -0,0 +1,3 @@ +[build-system] +requires = ["whool"] +build-backend = "whool.buildapi" diff --git a/spp_data_classification/readme/DESCRIPTION.md b/spp_data_classification/readme/DESCRIPTION.md new file mode 100644 index 00000000..0f03f7d1 --- /dev/null +++ b/spp_data_classification/readme/DESCRIPTION.md @@ -0,0 +1,55 @@ +Data sensitivity classification registry for OpenSPP. Defines classification levels, maps model fields to those levels with PII categorization, and provides regex/CEL auto-detection patterns. This is the foundation other modules consume to decide what to encrypt, mask, or audit. + +### Key Capabilities + +- Define sensitivity levels (PUBLIC, INTERNAL, CONFIDENTIAL, RESTRICTED) with encryption, masking, audit, consent, and retention policy flags +- Map model fields to classification levels with PII categorization (direct identifiers, quasi-identifiers, biometric, health, financial, etc.) +- Flag classified fields as PII (`is_pii`, derived from the assigned PII category) so consumers can query them +- Auto-classify fields using regex (or CEL) patterns that match field names and metadata, via a manually-invoked scan + +### Key Models + +| Model | Description | +| ---------------------------------- | -------------------------------------------------------------------- | +| `spp.data.classification.level` | Sensitivity levels with encryption, masking, and retention policies | +| `spp.field.classification` | Maps specific model fields to classification levels and PII category | +| `spp.classification.pattern` | Auto-detection patterns using regex or CEL expressions | + +### Configuration + +After installing: + +1. Navigate to **Settings > Data Classification > Classification Levels** to review pre-loaded sensitivity levels (PUBLIC, INTERNAL, CONFIDENTIAL, RESTRICTED) +2. Review **Detection Patterns** to customize auto-classification rules for your organization +3. Map fields under **Field Classifications** + +### UI Location + +- **Menu**: Settings > Data Classification +- **Classification Levels**: Settings > Data Classification > Classification Levels +- **Field Classifications**: Settings > Data Classification > Field Classifications +- **Detection Patterns**: Settings > Data Classification > Detection Patterns + +### Security + +| Group | Access | +| ------------------------------------------------------ | -------------------------------------------------------- | +| `spp_data_classification.group_classification_admin` | Full CRUD on levels, patterns, and field classifications | +| `spp_data_classification.group_classification_manager` | Read/Write/Create field classifications (no delete); read-only on levels and patterns | +| `base.group_user` | Read-only access to classifications | + +The `pii_full_access` and `pii_confidential_access` groups are defined for downstream masking/access-control consumers. + +### Extension Points + +- Query `spp.field.classification` to discover classified/PII fields per model (e.g. `get_fields_requiring_encryption()`) +- Override `spp.classification.pattern.matches_field()` to implement custom detection logic +- Add custom PII categories by extending the `pii_category` selection field + +### Deferred subsystems + +PII-aware enforcement (automatic read-time masking + the `pii=`/`classification=` field attributes), GDPR DSAR handling, data-retention scheduling, and consent integration are part of this module's design but are delivered in a separate governance change. This module ships the classification registry only. + +### Dependencies + +`base`, `spp_security` diff --git a/spp_data_classification/security/ir.model.access.csv b/spp_data_classification/security/ir.model.access.csv new file mode 100644 index 00000000..c49f6d82 --- /dev/null +++ b/spp_data_classification/security/ir.model.access.csv @@ -0,0 +1,9 @@ +id,name,model_id:id,group_id:id,perm_read,perm_write,perm_create,perm_unlink +access_classification_level_admin,Classification Level Admin,model_spp_data_classification_level,group_classification_admin,1,1,1,1 +access_classification_level_manager,Classification Level Manager,model_spp_data_classification_level,group_classification_manager,1,0,0,0 +access_classification_level_user,Classification Level User,model_spp_data_classification_level,base.group_user,1,0,0,0 +access_field_classification_admin,Field Classification Admin,model_spp_field_classification,group_classification_admin,1,1,1,1 +access_field_classification_manager,Field Classification Manager,model_spp_field_classification,group_classification_manager,1,1,1,0 +access_field_classification_user,Field Classification User,model_spp_field_classification,base.group_user,1,0,0,0 +access_classification_pattern_admin,Classification Pattern Admin,model_spp_classification_pattern,group_classification_admin,1,1,1,1 +access_classification_pattern_manager,Classification Pattern Manager,model_spp_classification_pattern,group_classification_manager,1,0,0,0 diff --git a/spp_data_classification/security/security_groups.xml b/spp_data_classification/security/security_groups.xml new file mode 100644 index 00000000..9e4560e2 --- /dev/null +++ b/spp_data_classification/security/security_groups.xml @@ -0,0 +1,69 @@ + + + + + + + Administrator + + 10 + + + + Manager + + 20 + + + + Full Access (Admin) + + 30 + + + + Confidential Access (Officer) + + 40 + + + + + Data Classification Admin + + Full access to data classification configuration. Can create/edit classification levels, patterns, and field classifications. + + + + + + Data Classification Manager + + Can classify fields and view classification settings. Cannot modify classification levels or patterns. + + + + + + PII Full Access + + Can view unmasked PII data across all classification levels. Should be granted sparingly. + + + + + + PII Confidential Access + + Can view unmasked CONFIDENTIAL data. RESTRICTED data remains masked. + + + diff --git a/spp_data_classification/static/description/icon.png b/spp_data_classification/static/description/icon.png new file mode 100644 index 00000000..c7dbdaaf Binary files /dev/null and b/spp_data_classification/static/description/icon.png differ diff --git a/spp_data_classification/static/description/index.html b/spp_data_classification/static/description/index.html new file mode 100644 index 00000000..3990811f --- /dev/null +++ b/spp_data_classification/static/description/index.html @@ -0,0 +1,542 @@ + + + + + +OpenSPP Data Classification + + + +
+

OpenSPP Data Classification

+ + +

Alpha License: LGPL-3 OpenSPP/OpenSPP2

+

Data sensitivity classification registry for OpenSPP. Defines +classification levels, maps model fields to those levels with PII +categorization, and provides regex/CEL auto-detection patterns. This is +the foundation other modules consume to decide what to encrypt, mask, or +audit.

+
+

Key Capabilities

+
    +
  • Define sensitivity levels (PUBLIC, INTERNAL, CONFIDENTIAL, RESTRICTED) +with encryption, masking, audit, consent, and retention policy flags
    • +
  • Map model fields to classification levels with PII categorization +(direct identifiers, quasi-identifiers, biometric, health, financial, +etc.)
    • +
  • Flag classified fields as PII (is_pii, derived from the assigned +PII category) so consumers can query them
    • +
  • Auto-classify fields using regex (or CEL) patterns that match field +names and metadata, via a manually-invoked scan
    • +
+
+
+

Key Models

+ ++++ + + + + + + + + + + + + + + + + +
ModelDescription
spp.data.classification.levelSensitivity levels with +encryption, masking, and +retention policies
spp.field.classificationMaps specific model fields to +classification levels and PII +category
spp.classification.patternAuto-detection patterns using +regex or CEL expressions
+
+
+

Configuration

+

After installing:

+
    +
  1. Navigate to Settings > Data Classification > Classification +Levels to review pre-loaded sensitivity levels (PUBLIC, INTERNAL, +CONFIDENTIAL, RESTRICTED)
    2. +
  2. Review Detection Patterns to customize auto-classification rules +for your organization
    3. +
  3. Map fields under Field Classifications
    4. +
+
+
+

UI Location

+
    +
  • Menu: Settings > Data Classification
    • +
  • Classification Levels: Settings > Data Classification > +Classification Levels
    • +
  • Field Classifications: Settings > Data Classification > Field +Classifications
    • +
  • Detection Patterns: Settings > Data Classification > Detection +Patterns
    • +
+
+
+

Security

+ ++++ + + + + + + + + + + + + + + + + +
GroupAccess
spp_data_classification.group_classification_adminFull CRUD on levels, patterns, +and field classifications
spp_data_classification.group_classification_managerRead/Write/Create field +classifications (no delete); +read-only on levels and patterns
base.group_userRead-only access to +classifications
+

The pii_full_access and pii_confidential_access groups are +defined for downstream masking/access-control consumers.

+
+
+

Extension Points

+
    +
  • Query spp.field.classification to discover classified/PII fields +per model (e.g. get_fields_requiring_encryption())
    • +
  • Override spp.classification.pattern.matches_field() to implement +custom detection logic
    • +
  • Add custom PII categories by extending the pii_category selection +field
    • +
+
+
+

Deferred subsystems

+

PII-aware enforcement (automatic read-time masking + the +pii=/classification= field attributes), GDPR DSAR handling, +data-retention scheduling, and consent integration are part of this +module’s design but are delivered in a separate governance change. This +module ships the classification registry only.

+
+
+

Dependencies

+

base, spp_security

+
+

Important

+

This is an alpha version, the data model and design can change at any time without warning. +Only for development or testing purpose, do not use in production.

+
+

Table of contents

+ +
+

Bug Tracker

+

Bugs are tracked on GitHub Issues. +In case of trouble, please check there if your issue has already been reported. +If you spotted it first, help us to smash it by providing a detailed and welcomed +feedback.

+

Do not contact contributors directly about support or help with technical issues.

+
+
+

Credits

+
+

Authors

+
    +
  • OpenSPP.org
    • +
+
+
+

Maintainers

+

Current maintainers:

+

jeremi gonzalesedwin1123

+

This module is part of the OpenSPP/OpenSPP2 project on GitHub.

+

You are welcome to contribute.

+
+
+
+
+ + diff --git a/spp_data_classification/tests/__init__.py b/spp_data_classification/tests/__init__.py new file mode 100644 index 00000000..fc3a3a6c --- /dev/null +++ b/spp_data_classification/tests/__init__.py @@ -0,0 +1,4 @@ +# Part of OpenSPP. See LICENSE file for full copyright and licensing details. +from . import test_classification_level +from . import test_field_classification +from . import test_classification_pattern diff --git a/spp_data_classification/tests/test_classification_level.py b/spp_data_classification/tests/test_classification_level.py new file mode 100644 index 00000000..a7a9b661 --- /dev/null +++ b/spp_data_classification/tests/test_classification_level.py @@ -0,0 +1,101 @@ +# Part of OpenSPP. See LICENSE file for full copyright and licensing details. +from psycopg2 import IntegrityError + +from odoo.exceptions import UserError, ValidationError +from odoo.tests.common import TransactionCase +from odoo.tools import mute_logger + + +class TestClassificationLevel(TransactionCase): + """Tests for spp.data.classification.level model.""" + + @classmethod + def setUpClass(cls): + super().setUpClass() + cls.Level = cls.env["spp.data.classification.level"] + + # Get default levels + cls.level_public = cls.env.ref("spp_data_classification.level_public") + cls.level_internal = cls.env.ref("spp_data_classification.level_internal") + cls.level_confidential = cls.env.ref("spp_data_classification.level_confidential") + cls.level_restricted = cls.env.ref("spp_data_classification.level_restricted") + + def test_default_levels_exist(self): + """Test that default classification levels are created.""" + self.assertTrue(self.level_public) + self.assertTrue(self.level_internal) + self.assertTrue(self.level_confidential) + self.assertTrue(self.level_restricted) + + def test_level_ordering(self): + """Test that levels are ordered by sensitivity (sequence).""" + self.assertLess(self.level_public.sequence, self.level_internal.sequence) + self.assertLess(self.level_internal.sequence, self.level_confidential.sequence) + self.assertLess(self.level_confidential.sequence, self.level_restricted.sequence) + + def test_get_level_by_code(self): + """Test get_level_by_code method.""" + level = self.Level.get_level_by_code("RESTRICTED") + self.assertEqual(level, self.level_restricted) + + level = self.Level.get_level_by_code("NONEXISTENT") + self.assertFalse(level) + + def test_is_more_sensitive_than(self): + """Test sensitivity comparison.""" + self.assertTrue(self.level_restricted.is_more_sensitive_than(self.level_confidential)) + self.assertTrue(self.level_confidential.is_more_sensitive_than(self.level_internal)) + self.assertFalse(self.level_internal.is_more_sensitive_than(self.level_confidential)) + + def test_policy_flags(self): + """Test that policy flags are set correctly.""" + # Restricted requires encryption + self.assertTrue(self.level_restricted.is_requires_encryption) + self.assertTrue(self.level_restricted.is_requires_masking) + self.assertTrue(self.level_restricted.is_requires_audit) + + # Public has minimal requirements + self.assertFalse(self.level_public.is_requires_encryption) + self.assertFalse(self.level_public.is_requires_masking) + self.assertFalse(self.level_public.is_requires_audit) + + @mute_logger("odoo.sql_db") + def test_unique_code_constraint(self): + """Test that classification codes must be unique.""" + with self.assertRaises(Exception) as cm: + self.Level.create( + { + "name": "Duplicate", + "code": "PUBLIC", # Already exists + "sequence": 100, + } + ) + # Should be either ValidationError or IntegrityError + self.assertTrue( + isinstance(cm.exception, (ValidationError, IntegrityError)), + f"Expected ValidationError or IntegrityError, got {type(cm.exception)}", + ) + + def test_cannot_delete_used_level(self): + """Test that levels in use cannot be deleted.""" + # Create a field classification using this level + model = self.env["ir.model"].search([("model", "=", "res.partner")], limit=1) + field = self.env["ir.model.fields"].search( + [ + ("model_id", "=", model.id), + ("name", "=", "name"), + ], + limit=1, + ) + + self.env["spp.field.classification"].create( + { + "model_id": model.id, + "field_id": field.id, + "classification_id": self.level_confidential.id, + } + ) + + # Try to delete - should fail + with self.assertRaises(UserError): + self.level_confidential.unlink() diff --git a/spp_data_classification/tests/test_classification_pattern.py b/spp_data_classification/tests/test_classification_pattern.py new file mode 100644 index 00000000..ec23f73d --- /dev/null +++ b/spp_data_classification/tests/test_classification_pattern.py @@ -0,0 +1,109 @@ +# Part of OpenSPP. See LICENSE file for full copyright and licensing details. +from odoo.tests.common import TransactionCase + + +class TestClassificationPattern(TransactionCase): + """Tests for spp.classification.pattern model.""" + + @classmethod + def setUpClass(cls): + super().setUpClass() + cls.Pattern = cls.env["spp.classification.pattern"] + cls.Classification = cls.env["spp.field.classification"] + + cls.level_restricted = cls.env.ref("spp_data_classification.level_restricted") + cls.level_confidential = cls.env.ref("spp_data_classification.level_confidential") + + def test_pattern_matches_field(self): + """Test pattern matching against field names.""" + pattern = self.Pattern.create( + { + "name": "Test National ID", + "pattern": "(national|passport).*id", + "classification_id": self.level_restricted.id, + "pii_category": "direct_id", + "priority": 90, + } + ) + + self.assertTrue(pattern.matches_field("national_id")) + self.assertTrue(pattern.matches_field("passport_id")) + self.assertTrue(pattern.matches_field("national_identity_id")) + self.assertFalse(pattern.matches_field("name")) + self.assertFalse(pattern.matches_field("email")) + + def test_pattern_with_model_scope(self): + """Test pattern matching with model scope.""" + pattern = self.Pattern.create( + { + "name": "SPP Models Only", + "pattern": ".*_id", + "classification_id": self.level_confidential.id, + "apply_to_model_pattern": r"spp\..*", + "priority": 50, + } + ) + + # Should match SPP models + self.assertTrue(pattern.matches_field("partner_id", "spp.registry.id")) + + # Should not match non-SPP models + self.assertFalse(pattern.matches_field("partner_id", "res.partner")) + + def test_find_matching_pattern(self): + """Test finding best matching pattern.""" + # Create patterns with different priorities + self.Pattern.create( + { + "name": "Low Priority", + "pattern": ".*name", + "classification_id": self.level_confidential.id, + "priority": 10, + } + ) + high_priority = self.Pattern.create( + { + "name": "High Priority - Family Name", + "pattern": "family.*name", + "classification_id": self.level_restricted.id, + "priority": 90, + } + ) + + # family_name should match high priority pattern + result = self.Pattern.find_matching_pattern("family_name") + self.assertEqual(result, high_priority) + + def test_auto_classify_field(self): + """Test auto-classification of a field.""" + self.Pattern.create( + { + "name": "Test Phone", + "pattern": "phone|mobile", + "classification_id": self.level_confidential.id, + "pii_category": "contact", + "default_mask_pattern": "***-***-####", + "default_search_strategy": "partial_index", + "priority": 70, + } + ) + + # Auto-classify phone field on res.partner + classification = self.Pattern.auto_classify_field("res.partner", "phone") + + self.assertTrue(classification) + self.assertEqual(classification.source, "auto") + self.assertEqual(classification.pii_category, "contact") + self.assertEqual(classification.mask_pattern, "***-***-####") + + def test_default_patterns_exist(self): + """Test that default detection patterns are loaded.""" + patterns = self.Pattern.search([]) + self.assertGreater(len(patterns), 0) + + # Check for some expected patterns + national_id = self.Pattern.search([("name", "ilike", "national")]) + self.assertTrue(national_id) + + phone = self.Pattern.search([("name", "ilike", "phone")]) + self.assertTrue(phone) diff --git a/spp_data_classification/tests/test_field_classification.py b/spp_data_classification/tests/test_field_classification.py new file mode 100644 index 00000000..65369184 --- /dev/null +++ b/spp_data_classification/tests/test_field_classification.py @@ -0,0 +1,190 @@ +# Part of OpenSPP. See LICENSE file for full copyright and licensing details. +from psycopg2 import IntegrityError + +from odoo.exceptions import ValidationError +from odoo.tests.common import TransactionCase +from odoo.tools import mute_logger + + +class TestFieldClassification(TransactionCase): + """Tests for spp.field.classification model.""" + + @classmethod + def setUpClass(cls): + super().setUpClass() + cls.Classification = cls.env["spp.field.classification"] + cls.Level = cls.env["spp.data.classification.level"] + + # Get a model and field to test with + cls.partner_model = cls.env["ir.model"].search([("model", "=", "res.partner")], limit=1) + cls.name_field = cls.env["ir.model.fields"].search( + [ + ("model_id", "=", cls.partner_model.id), + ("name", "=", "name"), + ], + limit=1, + ) + cls.email_field = cls.env["ir.model.fields"].search( + [ + ("model_id", "=", cls.partner_model.id), + ("name", "=", "email"), + ], + limit=1, + ) + + cls.level_confidential = cls.env.ref("spp_data_classification.level_confidential") + cls.level_restricted = cls.env.ref("spp_data_classification.level_restricted") + + def test_create_classification(self): + """Test creating a field classification.""" + classification = self.Classification.create( + { + "model_id": self.partner_model.id, + "field_id": self.name_field.id, + "classification_id": self.level_confidential.id, + "pii_category": "direct_id", + } + ) + + self.assertTrue(classification) + self.assertEqual(classification.model_name, "res.partner") + self.assertEqual(classification.field_name, "name") + + @mute_logger("odoo.sql_db") + def test_unique_constraint(self): + """Test that each field can only have one classification.""" + # Use a different field to avoid conflicts with other tests + phone_field = self.env["ir.model.fields"].search( + [ + ("model_id", "=", self.partner_model.id), + ("name", "=", "phone"), + ], + limit=1, + ) + # Clean up any existing classification for this field + self.Classification.search( + [ + ("model_id", "=", self.partner_model.id), + ("field_id", "=", phone_field.id), + ] + ).unlink() + + self.Classification.create( + { + "model_id": self.partner_model.id, + "field_id": phone_field.id, + "classification_id": self.level_confidential.id, + } + ) + + with self.assertRaises(Exception) as cm: + self.Classification.create( + { + "model_id": self.partner_model.id, + "field_id": phone_field.id, + "classification_id": self.level_restricted.id, + } + ) + # Should be either ValidationError or IntegrityError + self.assertTrue( + isinstance(cm.exception, (ValidationError, IntegrityError)), + f"Expected ValidationError or IntegrityError, got {type(cm.exception)}", + ) + + def test_get_classification(self): + """Test get_classification method.""" + self.Classification.create( + { + "model_id": self.partner_model.id, + "field_id": self.email_field.id, + "classification_id": self.level_confidential.id, + } + ) + + result = self.Classification.get_classification("res.partner", "email") + self.assertTrue(result) + self.assertEqual(result.classification_id, self.level_confidential) + + # Non-existent field + result = self.Classification.get_classification("res.partner", "nonexistent") + self.assertFalse(result) + + def test_get_model_classifications(self): + """Test get_model_classifications method.""" + # Create multiple classifications + self.Classification.create( + { + "model_id": self.partner_model.id, + "field_id": self.name_field.id, + "classification_id": self.level_confidential.id, + } + ) + self.Classification.create( + { + "model_id": self.partner_model.id, + "field_id": self.email_field.id, + "classification_id": self.level_confidential.id, + } + ) + + results = self.Classification.get_model_classifications("res.partner") + self.assertGreaterEqual(len(results), 2) + + def test_ensure_classification(self): + """Test ensure_classification method.""" + # First call creates + result1 = self.Classification.ensure_classification( + "res.partner", + "name", + "CONFIDENTIAL", + source="manual", + ) + self.assertTrue(result1) + + # Second call returns existing + result2 = self.Classification.ensure_classification( + "res.partner", + "name", + "RESTRICTED", # Different level - should be ignored + ) + self.assertEqual(result1, result2) + + def test_get_fields_requiring_encryption(self): + """Test get_fields_requiring_encryption method.""" + self.Classification.create( + { + "model_id": self.partner_model.id, + "field_id": self.name_field.id, + "classification_id": self.level_restricted.id, # Requires encryption + } + ) + + results = self.Classification.get_fields_requiring_encryption("res.partner") + self.assertTrue(any(r.field_name == "name" for r in results)) + + def test_is_pii_computed_and_searchable(self): + """is_pii is True when a PII category is set, False otherwise, and searchable.""" + with_category = self.Classification.create( + { + "model_id": self.partner_model.id, + "field_id": self.name_field.id, + "classification_id": self.level_confidential.id, + "pii_category": "direct_id", + } + ) + without_category = self.Classification.create( + { + "model_id": self.partner_model.id, + "field_id": self.email_field.id, + "classification_id": self.level_confidential.id, + } + ) + + self.assertTrue(with_category.is_pii) + self.assertFalse(without_category.is_pii) + + # Stored + searchable: the [("is_pii", "=", True)] domain (used by the + # PII-encryption wizard and consent integration) must find it. + found = self.Classification.search([("is_pii", "=", True)]) + self.assertIn(with_category, found) + self.assertNotIn(without_category, found) diff --git a/spp_data_classification/views/classification_level_views.xml b/spp_data_classification/views/classification_level_views.xml new file mode 100644 index 00000000..5d36f7cb --- /dev/null +++ b/spp_data_classification/views/classification_level_views.xml @@ -0,0 +1,102 @@ + + + + + spp.data.classification.level.list + spp.data.classification.level + + + + + + + + + + + + + + + + + spp.data.classification.level.form + spp.data.classification.level + +
+ +
+ +
+ +
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + Classification Levels + spp.data.classification.level + list,form + +

+ Define data classification levels +

+

+ Classification levels define the sensitivity of data and the policies + that apply to it (encryption, masking, audit, retention). +

+ + + diff --git a/spp_data_classification/views/classification_pattern_views.xml b/spp_data_classification/views/classification_pattern_views.xml new file mode 100644 index 00000000..3e3814c1 --- /dev/null +++ b/spp_data_classification/views/classification_pattern_views.xml @@ -0,0 +1,212 @@ + + + + + spp.classification.pattern.list + spp.classification.pattern + + + + + + + + + + + + + + + + + + spp.classification.pattern.form + spp.classification.pattern + +
+
+
+ + +
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + spp.classification.pattern.search + spp.classification.pattern + + + + + + + + + + + + + + + + + + + + + + + Detection Patterns + spp.classification.pattern + list,form + + {'search_default_active': 1} + +

+ Define auto-detection patterns +

+

+ Patterns automatically classify fields based on matching rules. + Use Regex for simple name patterns or + CEL for complex rules with field metadata. + Higher priority patterns are evaluated first. +

+ + + diff --git a/spp_data_classification/views/field_classification_views.xml b/spp_data_classification/views/field_classification_views.xml new file mode 100644 index 00000000..2093c831 --- /dev/null +++ b/spp_data_classification/views/field_classification_views.xml @@ -0,0 +1,153 @@ + + + + + spp.field.classification.list + spp.field.classification + + + + + + + + + + + + + + + + spp.field.classification.form + spp.field.classification + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + spp.field.classification.search + spp.field.classification + + + + + + + + + + + + + + + + + + + + + + + + + + + + Field Classifications + spp.field.classification + list,form + + +

+ No field classifications yet +

+

+ Field classifications link specific model fields to classification levels. + They can be auto-detected from field names or manually configured. +

+ + + diff --git a/spp_data_classification/views/menu.xml b/spp_data_classification/views/menu.xml new file mode 100644 index 00000000..86281cf6 --- /dev/null +++ b/spp_data_classification/views/menu.xml @@ -0,0 +1,39 @@ + + + + + + + + + + + + + +