Currently, MEDomics doesn’t have a formal type-checking system. Instead, we rely on lots of if-else conditions to ensure things are typed correctly. However, a type checker would guarantee that both the input and output of functions/classes/and so on are correctly typed, which would make everyone in the MEDomics pipeline (i.e., from developers to users) happy. Ultimately, we prevent crashes before they occur and avoid creating cascading crash nightmares!
Right now, the codebase relies on Python's dynamic typing, which works fine at small scale but tends to surface hard-to-debug errors as the platform grows (I personally like this article from Dropbox https://dropbox.tech/application/our-journey-to-type-checking-4-million-lines-of-python). The Python ecosystem has, however, matured considerably on this matter, and two complementary tools have emerged that together may give us a much more robust foundation.
ty, built by Astral yes again! (https://github.com/astral-sh/ty), is a modern static type checker and language server for Python, considered SOTA in the field right now and is extremely fast. It catches type mismatches at development time (i.e., before the code ever runs).
beartype (https://github.com/beartype/beartype), on the other hand, operates at runtime, decorating functions and enforcing type annotations the moment they are actually called. It is O(1) in complexity (i.e., it adds essentially no overhead regardless of input size), which makes it realistic to leave enabled in production. Together, the two tools cover both ends: ty catches issues before runtime, beartype catches whatever slips through.
It might be a good idea to look into both as a way to bullet-proof the platform's internals without meaningfully impacting (overhead, etc.) performance.
References
Notes
Happy to contribute on this one, just let me know 👌
Currently, MEDomics doesn’t have a formal type-checking system. Instead, we rely on lots of if-else conditions to ensure things are typed correctly. However, a type checker would guarantee that both the input and output of functions/classes/and so on are correctly typed, which would make everyone in the MEDomics pipeline (i.e., from developers to users) happy. Ultimately, we prevent crashes before they occur and avoid creating cascading crash nightmares!
Right now, the codebase relies on Python's dynamic typing, which works fine at small scale but tends to surface hard-to-debug errors as the platform grows (I personally like this article from Dropbox https://dropbox.tech/application/our-journey-to-type-checking-4-million-lines-of-python). The Python ecosystem has, however, matured considerably on this matter, and two complementary tools have emerged that together may give us a much more robust foundation.
ty, built by Astral yes again! (https://github.com/astral-sh/ty), is a modern static type checker and language server for Python, considered SOTA in the field right now and is extremely fast. It catches type mismatches at development time (i.e., before the code ever runs).beartype(https://github.com/beartype/beartype), on the other hand, operates at runtime, decorating functions and enforcing type annotations the moment they are actually called. It is O(1) in complexity (i.e., it adds essentially no overhead regardless of input size), which makes it realistic to leave enabled in production. Together, the two tools cover both ends:tycatches issues before runtime,beartypecatches whatever slips through.It might be a good idea to look into both as a way to bullet-proof the platform's internals without meaningfully impacting (overhead, etc.) performance.
References
tyrepository: https://github.com/astral-sh/tybeartyperepository: https://github.com/beartype/beartypeNotes
Happy to contribute on this one, just let me know 👌