feat: add configurable sensitive word detection - #5
Conversation
Add SENSITIVE_WORD entity type that detects user-defined words from MASKER_SENSITIVE_WORDS env var (comma-separated). - Case-insensitive word boundary matching - Lowest priority in overlap resolution (EMAIL/CARD/PHONE/PERSON win) - Full support for detect, mask, and redact endpoints - Policy entry in default.yaml
kiku-jw
left a comment
There was a problem hiding this comment.
Blocking correctness/security issue in app/services/pii_detector.py:
SENSITIVE_WORD is documented as the lowest-priority entity, but _remove_overlaps() sorts primarily by start offset and then greedily accepts the first match. A configured phrase that starts before an overlapping EMAIL therefore suppresses the EMAIL match. Reproduction against the PR merged with current main:
settings.sensitive_words = "Contact test"
PIIDetector().detect("Contact test@example.com")
# => only SENSITIVE_WORD(0, 12)POST /api/v1/mask then returns ***@example.com, leaking part of the email address. Please make overlap selection honor entity priority across the whole overlapping range (not only identical start positions) and add a regression test where a sensitive phrase starts before an overlapping EMAIL.
Verification performed on the clean merge with current main: ruff check app tests, ruff format --check app, and the full test suite passed (100 passed, 19 skipped); the targeted overlap reproduction above still fails the security expectation.
|
Thank you for the contribution and for taking the time to add tests. After reviewing the project status, Masker is currently parked as a portfolio/maintenance-only project, so we are not taking on new feature expansion. The overlap issue in the review would require additional security-sensitive redesign, and it would not be fair to ask you to spend more time on that for a feature we do not plan to ship. Closing this as not planned rather than waiting for changes. The contribution remains preserved in the PR history. |
|
I reconsidered this after reviewing the product value: an external contribution is itself useful demand evidence. Your implementation has now landed on I added a safety fix so a configured phrase cannot partially override higher-priority PII (for example, expose part of an email), plus API/docs coverage. Thank you for contributing this feature. |
Summary
Adds a new
SENSITIVE_WORDentity type that allows detecting user-defined sensitive words via theMASKER_SENSITIVE_WORDSenvironment variable.Changes
SENSITIVE_WORDadded to EntityTypeMASKER_SENSITIVE_WORDSenv var (comma-separated word list)\bword boundariessensitive_word: maskin default policyUsage
MASKER_SENSITIVE_WORDS="secret,confidential,password" python -m app.mainTesting
tests/test_sensitive_words.pywith class-based testsMASKER_SENSITIVE_WORDS="СВО")