[DevKaurav-14]

Fixes #522

What was wrong

With consider_tld=True, domain() looked up the raw TLD in the IANA list. The list stores IDN TLDs in punycode, so Unicode TLDs such as .рф or .中国 were always rejected. hostname() and url() inherited this through domain().

Change

The TLD check now runs after IDNA encoding and uses the encoded TLD. Nothing else in the validation changes.

Tests

• test_domain.py: Unicode TLDs (пример.рф, пример.рф. with rfc_1034, 例子.中国) pass, and an unknown Unicode TLD (пример.рфы) fails.
• test_hostname.py, test_url.py: new consider_tld cases for ASCII, punycode and Unicode TLDs, valid and invalid.

The new tests fail without the fix. ruff check, ruff format --check and pyright are clean.

[nandgator]

Thanks, this is the fix I'd like to merge for #522, and it's preferred over #524. Before it can go in, two things:

1. Reject IDNA-mapped lookalikes in domains and simple hostnames #527 (now merged) makes domain() reject names that only become valid through IDNA 2003 mapping, such as fullwidth letters, soft hyphens and 。. Without it, looking up the encoded TLD would have let example.ｃｏｍ pass with consider_tld=True. With Reject IDNA-mapped lookalikes in domains and simple hostnames #527 in place, your change is safe: all 151 IDN TLDs on the current IANA list validate, and the lookalikes stay rejected.
2. The branch conflicts with master now, because the tests appended at the end of tests/test_hostname.py and tests/test_url.py collide with recently merged PRs. Could you rebase onto master and push? CI will run once you've pushed.

[DevKaurav-14]

@nandgator I've rebased onto master and pushed.Could you approve the workflow runs?

[nandgator]

Thanks for rebasing, and sorry: the conflicts this time came from my own merges (#533 and #534 added tests in the same places right after your push). Since that was my doing, I rebased your commit onto master and pushed it to your branch. The conflicts were test-only, and I kept both sides. Your change and your authorship are unchanged. I approved the CI runs and merged once they were green.