gh-83461: Don't allow datetime parsing to accept non-ASCII digits

[StanFromIreland]

• Issue: Don't allow datetime parsing to accept non-Ascii digits #83461

[picnixz]

The original issue was marked with a security bug but I haven't looked at the entire thread so we might consider it a simple bug fix

[StanFromIreland]

@picnixz

No it needs to be relabeled:

-versions -type-security +type-bug

[picnixz]

Is there a need to update the C implementation?

[StanFromIreland]

C calls the Python implementation.

[picnixz]

I actually wonder but what about languages for which their input has non-ASCII strings such as Japanese?

[picnixz]

I think the ASCII flag restriction is too broad here as it applies to all formats, not just the digit part.

[picnixz]

Please carefully address the following suggestions and please make sure that typos lines are correctly formatted.

[bedevere-app]

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

[picnixz]

Note: I'll review more this PR once I'm back because it will be easier when I'm on a laptop and not on mobile (so on Wednesday/Thursday)

.

[picnixz]

Also, I realized that the summary of "reject non-ASCII digits" isn't quite right, because there are actually locales where codes like %c and %X will generate strings with non-ASCII digits, e.g.:

Oh I wasn't aware of this so thank you for the corrections.

[picnixz]

Maybe in Lib/_striptime.py, you can add a small comment saying that the O* formats are locale-specific? (just above `for d in 'dmyHIMS'?

[picnixz]

Should we mention the "non-locale-specific numeric format codes" here or, since it's not officially supported, we can be a bit lazy?

[StanFromIreland]

@pganssle Do you want to do this too? Or should I?

[StanFromIreland]

@pganssle friendly ping :-)

[pganssle]

OK, this looks good to me now. Since @serhiy-storchaka asked us to wait on it, though, I'll leave it up to him as to when we merge it or if other changes are necessary.

[StanFromIreland]

Friendly ping @serhiy-storchaka

[StanFromIreland]

Any update on this @serhiy-storchaka ?

[github-actions]

This PR is stale because it has been open for 30 days with no activity.

[serhiy-storchaka]

@StanFromIreland, could you please update this PR?

[read-the-docs-community]

Documentation build overview

📚 cpython-previews | 🛠️ Build #34154350 | 📁 Comparing 2f2bc87 against main (e0293b0)

  🔍 Preview build  

90 files changed · ± 90 modified

± Modified

• index.html
• license.html
• c-api/arg.html
• c-api/capsule.html
• c-api/exceptions.html
• c-api/frame.html
• c-api/init_config.html
• c-api/intro.html
• c-api/stable.html
• c-api/sys.html
• and 80 more...

[serhiy-storchaka]

The new tests pass without the change: four of them raise ValueError for a different reason, even with ASCII digits.

>>> strptime('2025-02-29', '%Y-%m-%d')          # day 29 must be in range 1..28 for month 2 in year 2025
>>> strptime('2555', '%G')                      # ISO year directive '%G' must be used with the ISO week directive '%V' ...
>>> strptime('2/02 02a2', '%j/%y %I%p:%M:%S')   # time data does not match format
>>> strptime('02/2/2002', '%U/%V')              # unconverted data remains: /2002

Only '1٢:02:٢7' with %H:%M:%S and '٢5' with %y test what they are meant to test.

These parse with ASCII digits, so the non-ASCII variants of them fail for the right reason:

>>> strptime('2025-02-28', '%Y-%m-%d')
>>> strptime('2555 01 1', '%G %V %u')
>>> strptime('2/02 02AM', '%j/%y %I%p')
>>> strptime('02', '%U')