[dependabot]

Bumps nltk from 3.10.3 to 3.10.4.

Release notes

Sourced from nltk's releases.

v3.10.4

Version 3.10.4 2026-10-20

This release routes every file path, archive, download, URL fetch and external program through one sandbox that keeps reads and writes inside the trusted data directories, refuses path traversal, symlink escapes and decompression bombs, blocks requests to internal network addresses, and runs Java and other third-party tools only from trusted absolute locations with validated options and input. Pickled models now load through a restricted unpickler, XML and JSON parsing are hardened, and regular expressions, recursion and slow algorithms in the tokenizers, parsers, stemmers and corpus readers are bounded against denial-of-service input. Text that NLTK writes to the terminal, to CSV files, to the WordNet browser and to third-party tool input is sanitized against injection.

Security changes you may notice

Every refusal below raises with a message that starts with "Security Violation" and names the check, so a failure is easy to search for (PR #3877).

• Data directories (nltk.pathsec, nltk.data): corpora, models and every other resource are read and written only inside the directories listed in nltk.data.path. A path that leaves a data root, a symlink that points out of one, and a NUL byte or encoded traversal in a resource name are refused. A data root, or a directory on the way to a resource, that another account can write to is skipped with a warning naming the directory and the remedy. Keep private data in a private directory and add it to nltk.data.path; /tmp is not a data root. Model save and load methods (perceptron, maxent, named-entity chunker, Weka, crf, hunpos, Stanford) accept destinations inside a data root only (PR #3797, #3753, #3850, #3761, #3755, #3759, #3760, #3763, #3806, #3807, #3808, #3809, #3813, #3814, #3815, #3848, #3851, #3852, #3853, #3863, #3871, #3888, #3945, #3961, #3969, #3981)
• Downloader: downloaded package archives stay private to the installing account (0600). An install another account must read is extracted: a download made as root or outside the installing account's home extracts on its own, and nltk.download(..., extract=True) / python -m nltk.downloader --extract (or --no-extract) decide explicitly. This also unpacks the packages the index leaves zipped (wordnet, omw-1.4 ...); the extracted files are readable by every account and are found before the archive. find() names an archive it cannot read and says how to extract it instead of reporting it "not found". The downloader installs only into directories no other account can write to, treats an archive another account owns or can write to as stale and downloads it afresh, extracts only from an archive no other account could have changed since its checksum was verified, bounds the size of what it fetches, and refuses index entries and archive members that would write outside the package directory (PR #3929, #3804, #3850, #3778)
• Pickles: nltk.data.load(..., format="pickle") reconstructs plain data only; a pickle that names any class or function is refused. Model loaders that need classes use an explicit allowlist of audited globals, so a gadget such as os.system can never be reconstructed from a model file. chat80 shelve values load the same way (PR #3794, #3754, #3757, #3802, #3817, #3823, #3824)
• XML, JSON and YAML: nltk.xmlsec refuses any document that declares an entity or references an external one (billion-laughs, external-entity and DTD attacks) and bounds the tree depth, and every corpus XML view, the downloader index, Toolbox and internals parse through it. JSON resources load with a depth and size bound; YAML (the markdown corpus front matter and nltk.data.load(..., format="yaml")) loads through a bounded safe loader. Oversized or too deeply nested input raises ValueError naming the limit (PR #3754, #3805, #3828, #3933, #3972, #3974)
• Regular expressions (nltk.redos): every regular expression NLTK compiles, including one you pass to RegexpTokenizer, RegexpStemmer, RegexpParser, re_show, TokenSearcher.findall, tgrep, nltk.help tag-set lookups and the concordance and nemo GUIs, now goes through nltk.redos on the regex engine. A pattern longer than 100,000 characters, nested more than 100 deep, with more than 1,000 capturing groups or an excessive repetition product is refused with nltk.redos.error, a subclass of re.error; a match that runs longer than nltk.redos.DEFAULT_TIMEOUT (5 seconds) raises TimeoutError instead of hanging. Patterns keep re syntax; match objects are the regex engine's (PR #3825, #3829, #3830, #3831, #3832, #3833, #3834, #3835, #3836, #3837, #3838, #3839, #3840, #3841, #3842, #3843, #3844, #3845, #3846, #3796, #3801, #3906, #3907, #3908, #3910)
• Regular expressions made linear: the leading-run and re-anchoring patterns that a timeout cannot catch were rewritten or bounded in the word, Treebank, blankline, TextTiling, toktok and syllable tokenizers, the readline of corpus streams, the CoNLL SRL, TIMIT, senseval, pl196x, NKJP, Alpino, YCOE, IEER, ACE, VerbNet, reviews and Lin readers, relextract, the valuation parser, the CCG lexicon parser, dependency grammars and chomsky_normal_form, each verified on the real corpus (PR #3947, #3934, #3932, #3938, #3935, #3933, #3869, #3944, #3911, #3909, #3953, #3895)
• Size, depth and time bounds: trees deeper than nltk.tree.MAX_TREE_DEPTH (500), WordNet and other relation walks deeper than nltk.util.MAX_RECURSION_DEPTH (500), tgrep deeper than 50, Toolbox deeper than 500, dependency graphs deeper than 200, are refused with ValueError naming the limit rather than exhausting the stack, and the stemmers, distance metrics, skipgrams, parsers and grammar transforms run in linear or bounded time with identical output for legitimate input. chart_parse stops after nltk.parse.chart.DEFAULT_MAX_TIME (5 seconds of wall-clock, needed only for feature grammars; max_time=None disables it). Tree.repr no longer recurses, so a deep accepted tree prints (PR #3827, #3885, #3895, #3767, #3784, #3917, #3958, #3960, #3977)
• External tools and Java: a tool binary is resolved from an absolute path or a trusted search path only, never from the current directory; the binary, its directory and the model and data files handed to it must not be writable by another account (symlinked and multi-call executables such as a packaged Graphviz dot are supported). java() accepts a main class as the first argument only (no -jar, @​argfile or .java source file), validates its options against an allowlist, and loads jars only from a data root unless NLTK_ALLOW_UNSAFE_JARS=1 opts out with a warning. CoreNLP server options are allowlisted. Stanford, Malt, Senna, hunpos, Prover9, megam, TADM, Weka and crf wrappers validate their paths and the text they pipe to the tool against line and argument injection (PR #3772, #3771, #3975, #3743, #3782, #3788, #3768, #3798, #3887, #3858, #3950, #3951, #3812, #3816, #3868, #3894, #3941, #3850, #3925)
• Network: the downloader, nltk.data.load of a URL and the CHILDES reader fetch through pathsec.urlopen, which refuses addresses that are not globally routable (loopback, private, link-local and multicast, including tunnelled and obfuscated forms), pins the validated address for the connection, allows file:// with a local authority only, and refuses a proxied fetch unless NLTK_ALLOW_PROXIED_URLOPEN=1 (or nltk.pathsec.ALLOW_PROXIED_FETCH = True) asserts the proxy is trusted. The CoreNLP client talks to its local server with requests and is not affected (PR #3847, #3810, #3750, #3850)
• Terminal, CSV, DOT and HTML output: every print in the library goes through nltk.termsec.safe_print, which renders control characters, escape sequences and unbalanced bidi overrides from corpus text as visible \xNN escapes instead of letting them reach the terminal. The CSV files the twitter and sentiment helpers write neutralise formula-leading and control-sequence cells, AlignedSent DOT labels are escaped, the WordNet browser escapes corpus text, and nltk.data.load's verbose output no longer prints the full resource URL (PR #3914, #3915, #3937, #3946, #3931, #3930, #3856, #3902)
• Corpus stores and encodings: the PanLex Lite and chat80 sqlite stores open read-only on a pinned descriptor with their schema screened and a statement step budget, chat80 shelves are read by a key manifest, Toolbox SFM markers are sanitised, and a resource encoding that is not a text codec (bz2_codec and friends) is refused (PR #3961, #3964, #3959, #3888, #3800, #3962)
• Package contents: the wheel no longer ships nltk.test.unit (the attack harnesses, advisory probes and their fixtures), so pip install nltk is about half the size and import nltk.test.unit works from a source checkout or the sdist only. The doctests under nltk.test still ship (PR #3954)

Security test and CI infrastructure

• Attack and regression harnesses, advisory probes, the sandbox conftest fixtures, the pre-commit guards (no unsandboxed open, every regex through redos, every pickle through picklesec, every print through safe_print), load-invariant CPU-time timing rules, CI matrix and runner fixes, pinned third-party downloads, and the AUTHORS update (PR #3744, #3749, #3762, #3764, #3770, #3776, #3791, #3795, #3803, #3818, #3820, #3821, #3822, #3849, #3855, #3860, #3861, #3862, #3864, #3865, #3866, #3876, #3884, #3899, #3904, #3905, #3912, #3917, #3924, #3926, #3927, #3939, #3940, #3943, #3948, #3949, #3963, #3968, #3971, #3976, #3982, #3983, #3984, #3985)

Other changes

• Added RTL language syntactic tree visualization support (PR #3472)
• Fix #3394: Ensure punkt_tab is downloaded when punkt is requested (PR #3505)
• Show README.md as the PyPI long description (PR #3746)
• Add stopwords.words("english-extended") (PR #3747)
• Krippendorff's alpha: treat placeholder labels as missing data (#2865, #2732) (PR #3752)
• Fix shared defaults in chat80 Concept (PR #3773)
• Fix Java wrappers decoding str output from java() (universal_newlines) (PR #3777)
• Prevent IndexError in Hungarian stemmer on empty string (PR #3785)
• Isolate MTE tag filters per view (PR #3792)
• Fix newline in ellipsis tokens in TweetTokenizer (PR #3819)
• Isolate default WordNet reference relations (PR #3872)
• Fix resolution paths that cancel distinct literals (PR #3878)
• Fix UTF-8 text preprocessing in NKJP corpus reader (PR #3879)
• Remove shadowed FeatStruct._walk stub and preserve its docstring (PR #3893)
• Count all increasing pairs in RIBES kendall_tau (PR #3900)
• Fix the TweetTokenizer ZWJ emoji pattern capturing adjacent whitespace (PR #3913)
• Surface the CoreNLP server's reason for a non-2xx answer (PR #3942)
• Omit unaligned (NULL target) pairs from Alignment strings, so IBM model output prints (PR #3967)
• Fix DependencyGraph cycle detection, which returned False on every parsed graph (PR #3978)
• Show, range and draw an alignment with an unaligned pair on every surface (PR #3987)

Thanks to the following contributors to 3.10.4: Zhewen Tan, Yanhaoxi, Yazan Balawneh (Cystack.ps),

... (truncated)

Changelog

Sourced from nltk's changelog.

Version 3.10.4 2026-10-20

This release routes every file path, archive, download, URL fetch and external program through one sandbox that keeps reads and writes inside the trusted data directories, refuses path traversal, symlink escapes and decompression bombs, blocks requests to internal network addresses, and runs Java and other third-party tools only from trusted absolute locations with validated options and input. Pickled models now load through a restricted unpickler, XML and JSON parsing are hardened, and regular expressions, recursion and slow algorithms in the tokenizers, parsers, stemmers and corpus readers are bounded against denial-of-service input. Text that NLTK writes to the terminal, to CSV files, to the WordNet browser and to third-party tool input is sanitized against injection.

Security changes you may notice

Every refusal below raises with a message that starts with "Security Violation" and names the check, so a failure is easy to search for (PR #3877).

• Data directories (nltk.pathsec, nltk.data): corpora, models and every other resource are read and written only inside the directories listed in nltk.data.path. A path that leaves a data root, a symlink that points out of one, and a NUL byte or encoded traversal in a resource name are refused. A data root, or a directory on the way to a resource, that another account can write to is skipped with a warning naming the directory and the remedy. Keep private data in a private directory and add it to nltk.data.path; /tmp is not a data root. Model save and load methods (perceptron, maxent, named-entity chunker, Weka, crf, hunpos, Stanford) accept destinations inside a data root only (PR #3797, #3753, #3850, #3761, #3755, #3759, #3760, #3763, #3806, #3807, #3808, #3809, #3813, #3814, #3815, #3848, #3851, #3852, #3853, #3863, #3871, #3888, #3945, #3961, #3969, #3981)
• Downloader: downloaded package archives stay private to the installing account (0600). An install another account must read is extracted: a download made as root or outside the installing account's home extracts on its own, and nltk.download(..., extract=True) / python -m nltk.downloader --extract (or --no-extract) decide explicitly. This also unpacks the packages the index leaves zipped (wordnet, omw-1.4 ...); the extracted files are readable by every account and are found before the archive. find() names an archive it cannot read and says how to extract it instead of reporting it "not found". The downloader installs only into directories no other account can write to, treats an archive another account owns or can write to as stale and downloads it afresh, extracts only from an archive no other account could have changed since its checksum was verified, bounds the size of what it fetches, and refuses index entries and archive members that would write outside the package directory (PR #3929, #3804, #3850, #3778)
• Pickles: nltk.data.load(..., format="pickle") reconstructs plain data only; a pickle that names any class or function is refused. Model loaders that need classes use an explicit allowlist of audited globals, so a gadget such as os.system can never be reconstructed from a model file. chat80 shelve values load the same way (PR #3794, #3754, #3757, #3802, #3817, #3823, #3824)
• XML, JSON and YAML: nltk.xmlsec refuses any document that declares an entity or references an external one (billion-laughs, external-entity and DTD attacks) and bounds the tree depth, and every corpus XML view, the downloader index, Toolbox and internals parse through it. JSON resources load with a depth and size bound; YAML (the markdown corpus front matter and nltk.data.load(..., format="yaml")) loads through a bounded safe loader. Oversized or too deeply nested input raises ValueError naming the limit (PR #3754, #3805, #3828, #3933, #3972, #3974)
• Regular expressions (nltk.redos): every regular expression NLTK compiles, including one you pass to RegexpTokenizer, RegexpStemmer, RegexpParser, re_show, TokenSearcher.findall, tgrep, nltk.help tag-set lookups and the concordance and nemo GUIs, now goes through nltk.redos on the regex engine. A pattern longer than 100,000 characters, nested more than 100 deep, with more than 1,000 capturing groups or an excessive repetition product is refused with nltk.redos.error, a subclass of re.error; a match that runs longer than nltk.redos.DEFAULT_TIMEOUT (5 seconds) raises TimeoutError instead of hanging. Patterns keep re syntax; match objects are the regex engine's (PR #3825, #3829, #3830, #3831, #3832, #3833, #3834, #3835, #3836, #3837, #3838, #3839, #3840, #3841, #3842, #3843, #3844, #3845, #3846, #3796, #3801, #3906, #3907, #3908, #3910)
• Regular expressions made linear: the leading-run and re-anchoring patterns that a timeout cannot catch were rewritten or bounded in the word, Treebank, blankline, TextTiling, toktok and syllable tokenizers, the readline of corpus streams, the CoNLL SRL, TIMIT, senseval, pl196x, NKJP, Alpino, YCOE, IEER, ACE, VerbNet, reviews and Lin readers, relextract, the valuation parser, the CCG lexicon parser, dependency grammars and chomsky_normal_form, each verified on the real corpus (PR #3947, #3934, #3932, #3938, #3935, #3933, #3869, #3944, #3911, #3909, #3953, #3895)
• Size, depth and time bounds: trees deeper than nltk.tree.MAX_TREE_DEPTH (500), WordNet and other relation walks deeper than nltk.util.MAX_RECURSION_DEPTH (500), tgrep deeper than 50, Toolbox deeper than 500, dependency graphs deeper than 200, are refused with ValueError naming the limit rather than exhausting the stack, and the stemmers, distance metrics, skipgrams, parsers and grammar transforms run in linear or bounded time with identical output for legitimate input. chart_parse stops after nltk.parse.chart.DEFAULT_MAX_TIME (5 seconds of wall-clock, needed only for feature grammars; max_time=None disables it). Tree.repr no longer recurses, so a deep accepted tree prints (PR #3827, #3885, #3895, #3767, #3784, #3917, #3958, #3960, #3977)
• External tools and Java: a tool binary is resolved from an absolute path or a trusted search path only, never from the current directory; the binary, its directory and the model and data files handed to it must not be writable by another account (symlinked and multi-call executables such as a packaged Graphviz dot are supported). java() accepts a main class as the first argument only (no -jar, @​argfile or .java source file), validates its options against an allowlist, and loads jars only from a data root unless NLTK_ALLOW_UNSAFE_JARS=1 opts out with a warning. CoreNLP server options are allowlisted. Stanford, Malt, Senna, hunpos, Prover9, megam, TADM, Weka and crf wrappers validate their paths and the text they pipe to the tool against line and argument injection (PR #3772, #3771, #3975, #3743, #3782, #3788, #3768, #3798, #3887, #3858, #3950, #3951, #3812, #3816, #3868, #3894, #3941, #3850, #3925)
• Network: the downloader, nltk.data.load of a URL and the CHILDES reader fetch through pathsec.urlopen, which refuses addresses that are not globally routable (loopback, private, link-local and multicast, including tunnelled and obfuscated forms), pins the validated address for the connection, allows file:// with a local authority only, and refuses a proxied fetch unless NLTK_ALLOW_PROXIED_URLOPEN=1 (or nltk.pathsec.ALLOW_PROXIED_FETCH = True) asserts the proxy is trusted. The CoreNLP client talks to its local server with requests and is not affected (PR #3847, #3810, #3750, #3850)
• Terminal, CSV, DOT and HTML output: every print in the library goes through nltk.termsec.safe_print, which renders control characters, escape sequences and unbalanced bidi overrides from corpus text as visible \xNN escapes instead of letting them reach the terminal. The CSV files the twitter and sentiment helpers write neutralise formula-leading and control-sequence cells, AlignedSent DOT labels are escaped, the WordNet browser escapes corpus text, and nltk.data.load's verbose output no longer prints the full resource URL (PR #3914, #3915, #3937, #3946, #3931, #3930, #3856, #3902)
• Corpus stores and encodings: the PanLex Lite and chat80 sqlite stores open read-only on a pinned descriptor with their schema screened and a statement step budget, chat80 shelves are read by a key manifest, Toolbox SFM markers are sanitised, and a resource encoding that is not a text codec (bz2_codec and friends) is refused (PR #3961, #3964, #3959, #3888, #3800, #3962)
• Package contents: the wheel no longer ships nltk.test.unit (the attack harnesses, advisory probes and their fixtures), so pip install nltk is about half the size and import nltk.test.unit works from a source checkout or the sdist only. The doctests under nltk.test still ship (PR #3954)

Security test and CI infrastructure

• Attack and regression harnesses, advisory probes, the sandbox conftest fixtures, the pre-commit guards (no unsandboxed open, every regex through redos, every pickle through picklesec, every print through safe_print), load-invariant CPU-time timing rules, CI matrix and runner fixes, pinned third-party downloads, and the AUTHORS update (PR #3744, #3749, #3762, #3764, #3770, #3776, #3791, #3795, #3803, #3818, #3820, #3821, #3822, #3849, #3855, #3860, #3861, #3862, #3864, #3865, #3866, #3876, #3884, #3899, #3904, #3905, #3912, #3917, #3924, #3926, #3927, #3939, #3940, #3943, #3948, #3949, #3963, #3968, #3971, #3976, #3982, #3983, #3984, #3985)

Other changes

• Added RTL language syntactic tree visualization support (PR #3472)
• Fix #3394: Ensure punkt_tab is downloaded when punkt is requested (PR #3505)
• Show README.md as the PyPI long description (PR #3746)
• Add stopwords.words("english-extended") (PR #3747)
• Krippendorff's alpha: treat placeholder labels as missing data (#2865, #2732) (PR #3752)
• Fix shared defaults in chat80 Concept (PR #3773)
• Fix Java wrappers decoding str output from java() (universal_newlines) (PR #3777)
• Prevent IndexError in Hungarian stemmer on empty string (PR #3785)
• Isolate MTE tag filters per view (PR #3792)
• Fix newline in ellipsis tokens in TweetTokenizer (PR #3819)
• Isolate default WordNet reference relations (PR #3872)
• Fix resolution paths that cancel distinct literals (PR #3878)
• Fix UTF-8 text preprocessing in NKJP corpus reader (PR #3879)
• Remove shadowed FeatStruct._walk stub and preserve its docstring (PR #3893)
• Count all increasing pairs in RIBES kendall_tau (PR #3900)
• Fix the TweetTokenizer ZWJ emoji pattern capturing adjacent whitespace (PR #3913)
• Surface the CoreNLP server's reason for a non-2xx answer (PR #3942)
• Omit unaligned (NULL target) pairs from Alignment strings, so IBM model output prints (PR #3967)
• Fix DependencyGraph cycle detection, which returned False on every parsed graph (PR #3978)
• Show, range and draw an alignment with an unaligned pair on every surface (PR #3987)

Thanks to the following contributors to 3.10.4: Zhewen Tan, Yanhaoxi, Yazan Balawneh (Cystack.ps), Abdulmoyn Sommakieh, Golitsin Vyacheslav, Mingzhe Yu, 宁岑, Tai An, 区梓灏, Tom Aarsen, Raffi Enficiaud,

... (truncated)

Commits

• 5c41fc0 Merge pull request #3990 from alvations/deflake-span-teeth-control-20261021
• a517c64 Make the span_tokenize teeth control compute-bound so Windows reads its quadr...
• b8cfc96 Merge pull request #3989 from alvations/release-notes-thanks-20261021
• dd1254e Merge pull request #3988 from alvations/harness-3913-zwj-20261021
• 8fd879f Credit every contributor since 3.10.3 in the release notes and AUTHORS.md
• 7e33e78 Merge remote-tracking branch 'origin/develop' into release-notes-thanks-20261021
• 44f26f7 Merge pull request #3987 from alvations/audit-3967-alignment-20261021
• 1dffe0f Pin the TweetTokenizer ZWJ alternative: linear on repeated shapes, no whitesp...
• 3773795 Add #3913 and #3967, merged after the snapshot, to the 3.10.4 notes
• aaa6531 Show, range and draw an alignment with an unaligned pair on every surface
• Additional commits viewable in compare view

[coderabbitai]

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI
• Review profile: ASSERTIVE
• Plan: Advanced
• Run ID: bd667851-3ee3-4da1-a8c7-b21a6d83dfa9

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud