diff --git a/medcat-v2/README.md b/medcat-v2/README.md index ca7afa66b..c8d74b321 100644 --- a/medcat-v2/README.md +++ b/medcat-v2/README.md @@ -1,3 +1,218 @@ # MedCAT v2 -MedCAT v2 is now simply in at (../medcat)[./medcat]. \ No newline at end of file +<<<<<<< HEAD +MedCAT can be used to extract information from Electronic Health Records (EHRs) and link it to biomedical ontologies like SNOMED-CT, UMLS, or HPO (and potentially other ontologies). +Original paper for v1 on [arXiv](https://arxiv.org/abs/2010.01165). + +## Why MedCAT v2? + +MedCAT v2 is a comprehensive refactor designed to improve modularity, flexibility, and maintainability. The core library is now lightweight, with optional extras (spaCy tokenization, MetaCAT, DeID, RelCAT) available as separate installable features—allowing you to install only what you need. This modular approach reduces dependencies, enables smaller installs, and provides better separation of concerns. Additionally, v2 reduces internal coupling with spaCy, allowing for alternative tokenizers and greater extensibility. The new architecture makes it easier to create custom components and addons, while improving code maintainability and preparing the foundation for future enhancements. For most users, single-threaded inference APIs remain unchanged, ensuring a smooth transition. + +**There's a number of breaking changes in MedCAT v2 compared to v1.** +When moving from v1 to v2, please refer to the [migration guide](docs/migration_guide_v2.md). +Details on breaking are outlined [here](docs/breaking_changes.md). + +[![Build Status](https://github.com/CogStack/cogstack-nlp/actions/workflows/medcat-v2_main.yml/badge.svg?branch=main)](https://github.com/CogStack/cogstack-nlp/actions/workflows/medcat-v2_main.yml/badge.svg?branch=main) +[![Documentation Status](https://readthedocs.org/projects/cogstack-nlp/badge/?version=latest)](https://readthedocs.org/projects/cogstack-nlp/badge/?version=latest) +[![Latest release](https://img.shields.io/github/v/release/CogStack/cogstack-nlp?filter=medcat/*)](https://github.com/CogStack/cogstack-nlp/releases/latest) +[![pypi Version](https://img.shields.io/pypi/v/medcat.svg?style=flat-square&logo=pypi&logoColor=white)](https://pypi.org/project/medcat/) + +**Official Docs [here](https://cogstack-nlp.readthedocs.io/)** + +**Discussion Forum [discourse](https://discourse.cogstack.org/)** + +## Available Models + +We have 2 public v2 models available: +1) SnomedCT UK Clinical edition 39.0 (Oct 2024) and UK Drug Extension 39.0 (July 2024) based model enriched with UMLS 2024AA; trained only on MIMIC-IV +2) SnomedCT UK Clinical edition 40.2 (June 2025) and UK Drug Extension 40.3 (July 2024) based model enriched with UMLS 2024AA; trained only on MIMIC-IV + +There are also a number of MedCAT v1 models available that can automatically be converted if required. + +To download any of these models, please [follow this link](https://medcat.sites.er.kcl.ac.uk/auth-callback-api) and sign in using your NIH / UMLS API key. You will then be redirected to the MedCAT model download form. Please complete this form and you will be provided a download link. + +While we encourage you use MedCAT v2 and the models in that native format, if you download an older version MedCAT v2 will be able to load it and covnert it to the format it knows. However, the loading process will be considerably longerin those cases. + +If you wish you can also convert the v1 models into the v2 format (see [tutorial](../medcat-v2-tutorials/notebooks/introductory/migration/1._Migrate_v1_model_to_v2.ipynb)). + +```python +from medcat.utils.legacy import legacy_converter +from medcat.storage.serialisers import AvailableSerialisers +old_model = '' +new_model_dir = '' +legacy_converter.do_conversion(old_model_path, new_model_dir, AvailableSerialisers.dill) +``` +OR +```bash +model_path = "models/medcat1_model_pack.zip" +new_model_folder = "models" # file in this folder +! python -m medcat.utils.legacy.legacy_converter $model_path $new_model_folder --verbose +``` + +## News +- **New public 2024 and 2025** Snomed models were uploaded and made available 7. October 2025. +- **MedCAT 2.0.0** was released 18. August 2025. + +[News pre v2.0.0](docs/v1_news.md). + +## Installation + +MedCAT v2 has its first full release +``` +pip install medcat +``` +Do note that **this installs only the core MedCAT v2**. +**It does not necessary dependencies for `spacy`-based tokenizing or MetaCATs or DeID**. +However, all of those are supported as well. +You can install them as follows: +``` +pip install "medcat[spacy]" # for spacy-based tokenizer +pip install "medcat[meta-cat]" # for MetaCAT +pip install "medcat[deid]" # for DeID models +pip install "medcat[spacy,meta-cat,deid,rel-cat,dict-ner]" # for all of the above +``` + +### Installing plugins + +MedCAT v2 supports **external plugins** that can provide new components (e.g. alternative NER models, addons, tokenizers) via Python entry points. + +- **Curated plugins**: The `medcat.plugins.catalog` module ships with a curated plugin catalog that can be updated from a remote JSON file. +- **Installer**: The `medcat.plugins.installer.PluginInstallationManager` wraps a `pip`-based installer and knows how to resolve a compatible plugin version for your current MedCAT version. +- **CLI**: You can install curated plugins directly from the command line: + +```bash +python -m medcat plugins install medcat-gliner +``` + +This will: + +- look up `medcat-gliner` in the curated catalog, +- resolve a version compatible with your installed MedCAT, +- and install it using `pip`. + +You can also: + +- pass `--dry-run` to show what would be installed without making changes: + + ```bash + python -m medcat plugins install --dry-run medcat-gliner + ``` + +- override the version/ref explicitly (e.g. when testing a branch or tag): + + ```bash + python -m medcat plugins install medcat-gliner --force-version main + ``` + +If a plugin requires authentication (for example, private Git repositories), MedCAT will log a warning and the installer will surface pip’s error messages if credentials are missing or incorrect. + +### Version / update checking + +MedCAT now has the ability to check for newer versions of itself on PyPI (or a local mirror of it). +This is so users don't get left behind too far with older versions of our software. +This is configurable by evnironmental variables so that sys admins (e.g for JupyterHub) can specify the settings they wish. +Version checks are done once a week and the results are cached. + +Below is a table of the environmental variables that govern the version checking and their defaults. + +| Variable | Default | Description | +|-----------|----------|-------------| +| **`MEDCAT_DISABLE_VERSION_CHECK`** | *(unset)* | When set to `true`, `yes` or `disable`, disables the version update check entirely. Useful for CI environments, offline setups, or deployments where external network access is restricted. | +| **`MEDCAT_PYPI_URL`** | `https://pypi.org/pypi` | Base URL used to query package metadata. Can be changed to a PyPI mirror or internal repository that exposes the `/pypi/{pkg}/json` API. | +| **`MEDCAT_MINOR_UPDATE_THRESHOLD`** | `3` | Number of newer **minor** versions (e.g. `1.4.x`, `1.5.x`) that must exist before MedCAT emits a “newer version available” log message. | +| **`MEDCAT_PATCH_UPDATE_THRESHOLD`** | `3` | Number of newer **patch** versions (e.g. `1.3.1`, `1.3.2`, `1.3.3`) on the same minor line required before emitting an informational update message. | +| **`MEDCAT_VERSION_UPDATE_LOG_LEVEL`** | `INFO` | Logging level used when reporting available newer versions (minor/patch thresholds). Accepts any valid `logging` level string (`DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`). | +| **`MEDCAT_VERSION_UPDATE_YANKED_LOG_LEVEL`** | `WARNING` | Logging level used when reporting that the current version has been **yanked** on PyPI. Accepts the same values as above. | + +## Demo + +The MedCAT v2 demo web app is available [here](https://medcat.sites.er.kcl.ac.uk/). + +## Key Concepts + +- **Components**: The building blocks of MedCAT (NER, Entity Linking, preprocessing, etc.) +- **Addons**: Components that extend the core NER+EL pipeline with additional processing stages +- **Plugins**: External packages that provide new component implementations or other functionality via entry points + +See [Architecture Documentation](docs/architecture.md) for detailed information. + +## Tutorials +A guide on how to use MedCAT v2 is available at on the medcat documentation page on [docs.cogstack.org](https://docs.cogstack.org) + +## Contributing + +Please follow the [Contribution Guidelines](../CONTRIBUTING.md). + +When writing your own component (NER or linker), it is recommended making sure they follow the contracts for these components. +
+Example test for custom components in model pack + +```python +from unittest import TestCase +from medcat.components.contracting_testing import assert_component_contracts +# implement create_model_with_my_component +class MyComponentTest(TestCase): + def test_my_model_contract(self): + # create or load a model with your custom component(s) + # NOTE: This would (generally) need to be able to NER / link 1 entity in the example text + # The test time models in medcat would be sufficient + cat = create_model_with_my_component() + assert_component_contracts(cat) +``` +
+ + +## Acknowledgements +Entity extraction was trained on [MedMentions](https://github.com/chanzuckerberg/MedMentions) In total it has ~ 35K entites from UMLS + +The vocabulary was compiled from [Wiktionary](https://en.wiktionary.org/wiki/Wiktionary:Main_Page) In total ~ 800K unique words + +## Powered By +A big thank you goes to [spaCy](https://spacy.io/) and [Hugging Face](https://huggingface.co/) - who made life a million times easier. + + +## Citation +MedCAT v2 citation: +``` +@inproceedings{ratas-etal-2026-medcat, + title = "{M}ed{CAT} v2: a modular, extensible architecture for clinical named entity recognition and linking under real-world privacy and compute constraints", + author = "Ratas, Mart and + Searle, Thomas and + Sutton, Adam and + Dobson, Richard", + editor = "Demner-Fushman, Dina and + Ananiadou, Sophia and + Roberts, Kirk and + Tsujii, Junichi", + booktitle = "{B}io{NLP} 2026", + month = jul, + year = "2026", + address = "San Diego, California", + publisher = "Association for Computational Linguistics", + url = "https://aclanthology.org/2026.bionlp-1.17/", + doi = "10.18653/v1/2026.bionlp-1.17", + pages = "191--198", + ISBN = "979-8-89176-434-7", + abstract = "MedCAT is an open-source framework for clinical named entity recognition and linking (NER+L) widely used in research and healthcare settings. We present MedCAT v2, a re-engineered version designed to improve modularity, extensibility, and maintainability while preserving the core functionality and performance of previous releases. The new architecture introduces a registry-based component system and a flexible pipeline that enables easy substitution of components, integration of alternative methods, and future expansion, including support for pre-trained components across the full NER+L and contextualisation workflow. This enables systematic exploration of clinical NER+L design trade-offs by evaluating different components in the pipeline. Evaluation across multiple public datasets shows equivalent or improved performance compared to earlier versions, with reduced integration overhead and improved runtime flexibility. The framework also supports optional extensions such as meta-annotation, relation extraction, providing a unified and reproducible environment for clinical NLP in real-world settings." +} +``` +
+MedCAT v1 citation + +``` +@ARTICLE{Kraljevic2021-ln, + title="Multi-domain clinical natural language processing with {MedCAT}: The Medical Concept Annotation Toolkit", + author="Kraljevic, Zeljko and Searle, Thomas and Shek, Anthony and Roguski, Lukasz and Noor, Kawsar and Bean, Daniel and Mascio, Aurelie and Zhu, Leilei and Folarin, Amos A and Roberts, Angus and Bendayan, Rebecca and Richardson, Mark P and Stewart, Robert and Shah, Anoop D and Wong, Wai Keong and Ibrahim, Zina and Teo, James T and Dobson, Richard J B", + journal="Artif. Intell. Med.", + volume=117, + pages="102083", + month=jul, + year=2021, + issn="0933-3657", + doi="10.1016/j.artmed.2021.102083" +} +
+``` +======= +MedCAT v2 is now simply in at (../medcat)[./medcat]. +>>>>>>> main diff --git a/medcat/medcat/components/base.py b/medcat/medcat/components/base.py new file mode 100644 index 000000000..2d64b945c --- /dev/null +++ b/medcat/medcat/components/base.py @@ -0,0 +1,109 @@ +from typing import Protocol, runtime_checkable, Optional +from typing_extensions import Self +from enum import Enum + +from pydantic import BaseModel + +from medcat.tokenizing.tokens import MutableDocument +from medcat.tokenizing.tokenizers import BaseTokenizer +from medcat.cdb import CDB +from medcat.vocab import Vocab +from medcat.config.config import ComponentConfig + + +@runtime_checkable +class BaseComponent(Protocol): + + @property + def full_name(self) -> Optional[str]: + """Name with the component type (e.g ner, linking, meta).""" + pass + + @property + def name(self) -> str: + """The name of the component.""" + pass + + def is_core(self) -> bool: + """Whether the component is a core component or not. + + Returns: + bool: Whether this is a core component. + """ + pass + + def __call__(self, doc: MutableDocument) -> MutableDocument: + pass + + @classmethod + def create_new_component( + cls, cnf: ComponentConfig, tokenizer: BaseTokenizer, + cdb: CDB, vocab: Vocab, model_load_path: Optional[str]) -> Self: + """Create a new component or load one off disk if load path presented. + + This may raise an exception if the wrong type of config is provided. + + Args: + cnf (ComponentConfig): The config relevant to this components. + tokenizer (BaseTokenizer): The base tokenizer. + cdb (CDB): The CDB. + vocab (Vocab): The Vocab. + model_load_path (Optional[str]): Model load path (if present). + + Returns: + Self: The new components. + """ + pass + +class CollectionContract(BaseModel, frozen=True): + """Contract for a collection field — what each item in the collection provides.""" + field: str # e.g. 'ner_ents' + must_provide: frozenset[str] # fields every item must have + may_provide: frozenset[str] = frozenset() + + +class ComponentContract(BaseModel, frozen=True): + needs: frozenset[str] + must_provide: frozenset[str] + may_provide: frozenset[str] = frozenset() + collection_contracts: frozenset[CollectionContract] = frozenset() + + +class CoreComponentType(Enum): + tagging = ComponentContract( + needs=frozenset(), + must_provide=frozenset(), + # doesn't write for every token + may_provide=frozenset({'token.is_punctuation', 'token.to_skip'}), + collection_contracts=frozenset(), + ) + token_normalizing = ComponentContract( + needs=frozenset(), + # should write for every token + must_provide=frozenset({'token.norm'}), + may_provide=frozenset(), + collection_contracts=frozenset(), + ) + ner = ComponentContract( + needs=frozenset({'token.to_skip'}), + must_provide=frozenset({'doc.ner_ents'}), # the list must exist + may_provide=frozenset(), + collection_contracts=frozenset({ + CollectionContract( + field='doc.ner_ents', + must_provide=frozenset({'detected_name'}), + ) + }), + ) + linking = ComponentContract( + needs=frozenset({'doc.ner_ents'}), + # must write, but may be empty list + must_provide=frozenset({'doc.linked_ents'}), + may_provide=frozenset({}), + collection_contracts=frozenset({ + CollectionContract( + field='doc.linked_ents', + must_provide=frozenset({'cui', 'context_similarity'}), + ), + }), + ) diff --git a/medcat/medcat/components/contracting.py b/medcat/medcat/components/contracting.py new file mode 100644 index 000000000..8afe55e40 --- /dev/null +++ b/medcat/medcat/components/contracting.py @@ -0,0 +1,241 @@ +from typing import Callable +from logging import Logger + +from medcat.tokenizing.tokens import MutableDocument +from medcat.components.base import BaseComponent, ComponentContract +from medcat.components.contracting_utils import ( + AccessType, wrap_relevant_parts, ContractViolation) + + +logger = Logger(__name__) + + +def verify_part( + text: str, + doc_getter: Callable[[str], MutableDocument], + component: BaseComponent, + paths: list[str], + access_type: AccessType, + raise_on_violation: bool = True, + min_feedbacks: int = 0, +) -> list[str]: + """Verify the parts for this text. + + Args: + text (str): The text to use. + doc_getter (Callable[[str], MutableDocument]): The document getter. + component (BaseComponent): The component to check. + paths (list[str]): The paths to check. + access_type (AccessType): The type of access to check. + raise_on_violation (bool): Whether to raise on a violation. + Defaults to True. + min_feedbacks (int): The minimum number of feedbacks expected. + Defaults to 0. + + Raises: + ContractViolation: If there are violations and instructed to raise. + + Returns: + list[str]: The list of violations, if any. + """ + violations: list[str] = [] + for path in paths: + doc = doc_getter(text) + with wrap_relevant_parts(doc, path) as feedback: + doc = component(doc) + # verify each one access + accessed = sum(bool(fb) for fb in feedback) + total = len(feedback) + if accessed != total: + violations.append( + f"Component {component.full_name} does not {access_type.name} " + f"{path} ({accessed} / {total} accessed)") + logger.debug( + "Found a virolation in component '%s' for %s at '%s': " + "(%d / %d) accessed with feedback %s", + component.full_name, access_type.name, + path, accessed, total, feedback, + ) + elif total < min_feedbacks: + violations.append( + f"Component {component.full_name} did not {access_type.name} " + f"{path} enough ({total} with minimum {min_feedbacks})") + logger.debug( + "Component '%s' did not %s " + "'%s' enough (%d with minimum %d)", + component.full_name, access_type.name, + path, total, min_feedbacks, + ) + if raise_on_violation: + raise ContractViolation("\n".join(violations)) + return violations + + +def verify_needs( + text: str, + doc_getter: Callable[[str], MutableDocument], + component: BaseComponent, + contract: ComponentContract, + raise_on_violation: bool = True, + min_feedbacks: int = 0, +) -> list[str]: + """Verify the needs portion of a contract. + + Args: + text (str): The text to use. + doc_getter (Callable[[str], MutableDocument]): The document getter. + component (BaseComponent): The component to check. + contract (ComponentContract): The contract to check. + raise_on_violation (bool): Whether to raise on violations. + Defaults to True. + min_feedbacks (int,): The minimum number of feedbacks expected. + Defaults to 0. + + Returns: + list[str]: The list of violations, if any. + """ + return verify_part( + text, doc_getter, component, list(contract.needs), + AccessType.READ, raise_on_violation=raise_on_violation, + min_feedbacks=min_feedbacks, + ) + + +def verify_must_provide( + text: str, + doc_getter: Callable[[str], MutableDocument], + component: BaseComponent, + contract: ComponentContract, + raise_on_violation: bool = True, + min_feedbacks: int = 0, +) -> list[str]: + """Verify the must-provide portion of a contract. + + Args: + text (str): The text to use. + doc_getter (Callable[[str], MutableDocument]): The document getter. + component (BaseComponent): The component to check. + contract (ComponentContract): The contract to check. + raise_on_violation (bool): Whether to raise on violations. + Defaults to True. + min_feedbacks (int,): The minimum number of feedbacks expected. + Defaults to 0. + + Returns: + list[str]: The list of violations, if any. + """ + return verify_part( + text, doc_getter, component, list(contract.must_provide), + AccessType.WRITE, raise_on_violation=raise_on_violation, + min_feedbacks=min_feedbacks, + ) + + +def verify_collections_contracts( + text: str, + doc_getter: Callable[[str], MutableDocument], + component: BaseComponent, + contract: ComponentContract, + raise_on_violation: bool = True, + min_feedbacks: int = 0, +) -> list[str]: + """Verify the collection contracts portion of a contract. + + This method only really checks the length of the collection + and that each item in there has a truthy value for each required + field. The expectation is that the write action (for the collection) + is checked by other parts. And because these entities may be created + in order to put them in the lists (i.e for NER) without the data filled + in, it's fair to assume that if the data exists, it was filled in. + + Args: + text (str): The text to use. + doc_getter (Callable[[str], MutableDocument]): The document getter. + component (BaseComponent): The component to check. + contract (ComponentContract): The contract to check. + raise_on_violation (bool): Whether to raise on violations. + Defaults to True. + min_feedbacks (int): The minimum number of feedbacks expected. + Defaults to 0. + + Returns: + list[str]: The list of violations, if any. + """ + violations: list[str] = [] + if not contract.collection_contracts: + return violations + doc = doc_getter(text) + doc = component(doc) + for cc in contract.collection_contracts: + if not cc.field.startswith("doc."): + violations.append( + f"Collection contract field '{cc.field}' is not a doc-level " + f"field — only doc.* fields are currently supported") + continue + attr = cc.field.split(".", 1)[1] + try: + collection = getattr(doc, attr) + except AttributeError: + violations.append( + f"Component {component.full_name} did not provide " + f"collection '{cc.field}' at all") + continue + items = list(collection) + if len(items) < min_feedbacks: + violations.append( + f"Collection '{cc.field}' has too few items " + f"({len(items)} with minimum {min_feedbacks})") + for i, item in enumerate(items): + for field in cc.must_provide: + try: + val = getattr(item, field) + except AttributeError: + violations.append( + f"Item {i} in '{cc.field}' is missing " + f"required field '{field}'") + continue + if not val: + violations.append( + f"Item {i} in '{cc.field}' has falsy value " + f"for required field '{field}' (got {val!r})") + if violations and raise_on_violation: + raise ContractViolation("\n".join(violations)) + return violations + + +def verify_contract( + text: str, + doc_getter: Callable[[str], MutableDocument], + component: BaseComponent, + contract: ComponentContract, + raise_on_violation: bool = True, + min_feedbacks_need: int = 0, + min_feedbacks_provide: int = 0, + min_feedbacks_contracts: int = 0, +) -> list[str]: + """ + Verify a ComponentContract against a document before/after a component ran. + Returns a list of violation messages. + + Raises ContractViolation if violations found and raise_on_violation. + """ + # verify needs are met + violations = verify_needs( + text, doc_getter, component, contract, raise_on_violation=False, + min_feedbacks=min_feedbacks_need, + ) + # verify mandatory returns are done + violations += verify_must_provide( + text, doc_getter, component, contract, raise_on_violation=False, + min_feedbacks=min_feedbacks_provide, + ) + # verify collections contracts + violations += verify_collections_contracts( + text, doc_getter, component, contract, raise_on_violation=False, + min_feedbacks=min_feedbacks_contracts, + ) + + if violations and raise_on_violation: + raise ContractViolation("\n".join(violations)) + + return violations diff --git a/medcat/medcat/components/contracting_testing.py b/medcat/medcat/components/contracting_testing.py new file mode 100644 index 000000000..9cfffd75d --- /dev/null +++ b/medcat/medcat/components/contracting_testing.py @@ -0,0 +1,109 @@ +from typing import Optional + +from medcat.cat import CAT +from medcat.tokenizing.tokens import MutableDocument +from medcat.components.base import CoreComponentType +from medcat.components.types import CoreComponent +from medcat.components.contracting import verify_contract + + +_DEFAULT_CONTRACT_TEXT = """ +John had been diagnosed with acute Kidney - Failure the week before. +""" +_DEFAULT_COMP_TYPES_TO_CHECK = [ + CoreComponentType.ner, CoreComponentType.linking] + + +class ContractViolationError(ValueError): + + def __init__(self, component_type: CoreComponentType, violations: list): + self.component_type = component_type + self.violations = violations + super().__init__( + f"Contract violations for {component_type.name}: {violations}" + ) + + +def assert_single_component_holds( + model: CAT, + component: CoreComponent, + text: str = _DEFAULT_CONTRACT_TEXT, +): + """Assert a specific component's contract holds. + + Example: + + def test_my_ner_contract(self): + cat = create_model_with_my_ner() + my_ner = cat.pipe.get_component(CoreComponentType.ner) + assert_single_component_holds(cat, my_ner) + + Args: + model (CAT): The model with the specific component. + component (CoreComponent): The component under test. + text (str): The text to use for the check. + Defaults to _DEFAULT_CONTRACT_TEXT. + """ + component_type = component.get_type() + + def prep(t: str) -> MutableDocument: + return model.pipe.pipe_until(t, component_type) + + contract = component_type.value + min_feedbacks_need = ( + len(list(prep(text))) if component_type is CoreComponentType.ner else 1 + ) + if not min_feedbacks_need: + # NOTE: this would normally happen with NER if/when there's no tokens + raise ContractViolationError( + component_type, + ["Cannot check for feedback needs if minimum is 0 " + f"for {component.full_name}", ]) + violations = verify_contract( + text, prep, component, contract, + raise_on_violation=False, + min_feedbacks_need=min_feedbacks_need, + # NOTE: this means that the collection (ner_ents or linked_ents) is + # written to at least once + min_feedbacks_provide=1, + # NOTE: this means we expect at least 1 entity in output + min_feedbacks_contracts=1, + ) + if violations: + raise ContractViolationError(component_type, violations) + + +def assert_component_contracts( + model: CAT, + text: str = _DEFAULT_CONTRACT_TEXT, + to_check: Optional[list[CoreComponentType]] = None +): + """Verify that all components upholds its MedCAT contract. + + Intended for use in tests by external component implementers. + Raises ContractViolationError if the contract is not upheld. + + Example: + + def test_my_model_contract(self): + cat = create_model_with_my_component() + assert_component_contracts(cat) + + Args: + model (CAT): The model pack to use. This needs to refer to a model that + is able to NER and link at least 1 entity in the provided text. + This model needs to already have the relevant component(s) to be + checked. + to_check (Optional[list[CoreComponentType]]): The core component types + to check. Defaults to NER and linking. + text (str): The text to use for the check. + Defaults to _DEFAULT_CONTRACT_TEXT. + + Raises: + ContractViolationError: If there are any violations found. + """ + if to_check is None: + to_check = _DEFAULT_COMP_TYPES_TO_CHECK + for cct in to_check: + cur_comp = model.pipe.get_component(cct) + assert_single_component_holds(model, cur_comp, text) diff --git a/medcat/medcat/components/contracting_utils.py b/medcat/medcat/components/contracting_utils.py new file mode 100644 index 000000000..cde9ed741 --- /dev/null +++ b/medcat/medcat/components/contracting_utils.py @@ -0,0 +1,183 @@ +from typing import Any, Iterator, Type +from contextlib import contextmanager, ExitStack +from collections import defaultdict +from logging import Logger +from enum import Enum, auto + +from medcat.tokenizing.tokens import MutableDocument + + +logger = Logger(__name__) + +_SENTINEL = object() + + +class ContractViolation(Exception): + pass + + +class AccessType(Enum): + READ = auto() + WRITE = auto() + + +def iter_relevant_parts(doc: MutableDocument, path: str) -> Iterator[Any]: + if path.startswith("doc."): + yield doc + return + if path.startswith("token."): + yield from doc[:] + else: + raise ValueError(f"Unknown path: {path}") + + +class WrappedMember: + + def __init__( + self, + part: Any, + member_name: str, + feedback: list[Any], + access_type: AccessType, + ) -> None: + self.part = part + self.member_name = member_name + self.feedback = feedback + self.access_type = access_type + self._oirg_class = type(self.part) + self._install() + + def _install(self): + original_cls = type(self.part) + spy = self # capture for closure + + if self.access_type == AccessType.READ: + + class SpySubclass(original_cls): + def __getattribute__(self, name): + val = super().__getattribute__(name) + if name == spy.member_name: + # saving str copy of value + spy.feedback.append(str(val)) + return val + + elif self.access_type == AccessType.WRITE: + + class SpySubclass(original_cls): + def __setattr__(self, name, value): + if name in spy.member_name: + try: + old = super().__getattribute__(name) + except AttributeError: + old = AttributeError # sentinel: didn't exist yet + # saving str copies of state + spy.feedback.append((str(old), str(value))) + super().__setattr__(name, value) + + SpySubclass.__name__ = f"Spy({original_cls.__name__})" + SpySubclass.__qualname__ = SpySubclass.__name__ + self.part.__class__ = SpySubclass + + def __enter__(self): + return self + + def __exit__(self, *_): + self.part.__class__ = self._oirg_class + + +@contextmanager +def spy_token_class( + token_cls: Type, + watched_attr: str, + access_type: AccessType, +): + prev_getattr = token_cls.__dict__.get('__getattribute__', _SENTINEL) + prev_setattr = token_cls.__dict__.get('__setattr__', _SENTINEL) + + per_instance_spied: dict[Any, list[Any]] = defaultdict(list) + + def __getattribute__(self, name: str) -> Any: + val = object.__getattribute__(self, name) + if name == watched_attr: + per_instance_spied[self].append(str(val)) + return val + + def __setattr__(self, name: str, value: Any): + old = object.__getattribute__(self, name) + object.__setattr__(self, name, value) + if name == watched_attr: + per_instance_spied[self].append((str(old), str(value))) + + if access_type == AccessType.READ: + # NOTE: this should be fine, but mypy complains due to self + token_cls.__getattribute__ = __getattribute__ # type: ignore + elif access_type == AccessType.WRITE: + # NOTE: this should be fine, but mypy complains due to self + token_cls.__setattr__ = __setattr__ # type: ignore + else: + raise ValueError(f"Unknown access type: {access_type}") + try: + yield per_instance_spied + finally: + if prev_getattr is _SENTINEL: + del token_cls.__getattribute__ + else: + token_cls.__getattribute__ = prev_getattr + + if prev_setattr is _SENTINEL: + # NOTE: this should be fine, but mypy complains due to self + token_cls.__setattr__ = object.__setattr__ # type: ignore + else: + token_cls.__setattr__ = prev_setattr + + +@contextmanager +def wrap_relevant_parts( + doc: MutableDocument, + path: str, + access_type: AccessType = AccessType.READ, +): + if path.startswith("doc."): + with wrap_relevant_persistant_parts( + doc, path, access_type + ) as feedbacks: + yield feedbacks + elif path.startswith("token."): + with wrap_relevant_token_cls( + doc, path, access_type + ) as feedbacks: + yield feedbacks + + +@contextmanager +def wrap_relevant_token_cls( + doc: MutableDocument, + path: str, + access_type: AccessType = AccessType.READ, +): + _, attr_name = path.split(".", 1) + tkn_cls = type(next(iter(doc))) + with spy_token_class( + tkn_cls, attr_name, access_type + ) as per_instance_spied: + yield per_instance_spied.values() + + +@contextmanager +def wrap_relevant_persistant_parts( + doc: MutableDocument, + path: str, + access_type: AccessType = AccessType.READ, +): + member_name = path.split(".", 1)[1] + out_list: list[list[Any]] = [] + with ExitStack() as exit_stack: + for part in iter_relevant_parts(doc, path): + feedback: list[Any] = [] + exit_stack.enter_context( + WrappedMember( + part, member_name, + feedback, access_type=access_type) + ) + out_list.append(feedback) + yield out_list diff --git a/medcat/medcat/components/types.py b/medcat/medcat/components/types.py index dba485285..51a527308 100644 --- a/medcat/medcat/components/types.py +++ b/medcat/medcat/components/types.py @@ -1,7 +1,5 @@ from typing import Optional, Protocol, Callable, runtime_checkable, Union from typing import Literal -from typing_extensions import Self -from enum import Enum, auto from abc import ABC, abstractmethod from dataclasses import dataclass, field @@ -11,58 +9,7 @@ from medcat.cdb import CDB from medcat.vocab import Vocab from medcat.config.config import ComponentConfig - - -class CoreComponentType(Enum): - tagging = auto() - token_normalizing = auto() - ner = auto() - linking = auto() - - -@runtime_checkable -class BaseComponent(Protocol): - - @property - def full_name(self) -> Optional[str]: - """Name with the component type (e.g ner, linking, meta).""" - pass - - @property - def name(self) -> str: - """The name of the component.""" - pass - - def is_core(self) -> bool: - """Whether the component is a core component or not. - - Returns: - bool: Whether this is a core component. - """ - pass - - def __call__(self, doc: MutableDocument) -> MutableDocument: - pass - - @classmethod - def create_new_component( - cls, cnf: ComponentConfig, tokenizer: BaseTokenizer, - cdb: CDB, vocab: Vocab, model_load_path: Optional[str]) -> Self: - """Create a new component or load one off disk if load path presented. - - This may raise an exception if the wrong type of config is provided. - - Args: - cnf (ComponentConfig): The config relevant to this components. - tokenizer (BaseTokenizer): The base tokenizer. - cdb (CDB): The CDB. - vocab (Vocab): The Vocab. - model_load_path (Optional[str]): Model load path (if present). - - Returns: - Self: The new components. - """ - pass +from medcat.components.base import BaseComponent, CoreComponentType @runtime_checkable diff --git a/medcat/medcat/pipeline/pipeline.py b/medcat/medcat/pipeline/pipeline.py index 9da51e1a2..e22423580 100644 --- a/medcat/medcat/pipeline/pipeline.py +++ b/medcat/medcat/pipeline/pipeline.py @@ -341,8 +341,33 @@ def get_doc(self, text: str) -> MutableDocument: Returns: MutableDocument: The resulting document. """ + return self.pipe_until(text, None) + + def pipe_until( + self, + text: str, + comp_type: Optional[CoreComponentType] + ) -> MutableDocument: + """Run the pipe until the specific component (excluded). + + If `comp_type == None` then the entire pipe is run. + Otherwise the pipe is stopped before the specificied component is run. + + Args: + text (str): The text to run over. + comp_type (Optional[CoreComponentType]): The last component to run or None. + + Returns: + MutableDocument: The processed document. + """ doc = self._tokenizer(text) for comp in self._components: + if comp_type and comp.get_type() == comp_type: + logger.info( + "Finishing pipe before %s (%s) as requested", + comp.get_type().name, comp.full_name + ) + return doc logger.info("Running component %s for %d of text (%s)", comp.full_name, len(text), id(text)) doc = comp(doc) @@ -360,6 +385,7 @@ def get_doc(self, text: str) -> MutableDocument: ) return doc + def entity_from_tokens(self, tokens: list[MutableToken]) -> MutableEntity: """Get the entity from the list of tokens. diff --git a/medcat/tests/components/test_contracting.py b/medcat/tests/components/test_contracting.py new file mode 100644 index 000000000..225779a30 --- /dev/null +++ b/medcat/tests/components/test_contracting.py @@ -0,0 +1,21 @@ +from unittest import TestCase + +from medcat.cat import CAT +from medcat.components import contracting_testing +from medcat.components.base import CoreComponentType +from tests import UNPACKED_EXAMPLE_MODEL_PACK_PATH + + +class TestContractingForModel(TestCase): + + @classmethod + def setUpClass(cls) -> None: + cls._model = CAT.load_model_pack(UNPACKED_EXAMPLE_MODEL_PACK_PATH) + + def test_all_contracts_hold(self): + contracting_testing.assert_component_contracts(self._model) + + def test_individual_contracts_hold(self): + for ct in [CoreComponentType.ner, CoreComponentType.linking]: + comp = self._model.pipe.get_component(ct) + contracting_testing.assert_single_component_holds(self._model, comp) diff --git a/medcat/tests/components/test_contracting_testing.py b/medcat/tests/components/test_contracting_testing.py new file mode 100644 index 000000000..df96ae019 --- /dev/null +++ b/medcat/tests/components/test_contracting_testing.py @@ -0,0 +1,39 @@ +from unittest import TestCase +from contextlib import contextmanager + +from medcat.cat import CAT +from medcat.components import contracting_testing +from medcat.utils.cdb_state import captured_state_cdb +from tests import UNPACKED_EXAMPLE_MODEL_PACK_PATH + + +class ContractingTestingTests(TestCase): + + @classmethod + def setUpClass(cls) -> None: + cls._model = CAT.load_model_pack(UNPACKED_EXAMPLE_MODEL_PACK_PATH) + + @contextmanager + def empty_cdb(self): + with captured_state_cdb(self._model.cdb): + self._model.cdb.name2info.clear() + self._model.cdb.cui2info.clear() + yield + + def test_contracting_normally_passes(self): + contracting_testing.assert_component_contracts(self._model) + + def test_contracting_fails_with_empty_cdb(self): + with self.empty_cdb(): + with self.assertRaises(contracting_testing.ContractViolationError): + contracting_testing.assert_component_contracts(self._model) + + def test_contracting_fails_with_no_entity(self): + with self.assertRaises(contracting_testing.ContractViolationError): + contracting_testing.assert_component_contracts( + self._model, "Text with no entities") + + def test_contracting_fails_with_no_tokens(self): + with self.assertRaises(contracting_testing.ContractViolationError): + contracting_testing.assert_component_contracts( + self._model, "")