Modify linter config to check end-of-line characters

[jeremyleung521]

Is your feature request related to a problem?

Describe the solution you'd like

I noticed that some PRs have a huge amount of changed files, but the diffs basically show nothing changed. If I pull the file and open it with vim in Binary mode (vim -b, basically see the raw text), I see a carriage-return character (^M, equivalently \r), suggesting that those files might be edited/created on Windows/DOS and then transferred as a unix file, which means the end-of-line (EOL) contains \r\n instead of the \n used in unix. I suggest adding a hook/option to the linter to actually fix those so you don't get both windows EOL \r\n and unix EOL \n and very confusing, large git diffs/commits.

Example (Might be slow to load because of the large number of files)
https://github.com/MDAnalysis/mdanalysis/pull/5290/changes#diff-02997d9198d7518edcbe3d73550109ec5dd3fc4cb58ad035e22972b09622b6b0

More info about EOL characters: https://unix.stackexchange.com/questions/32001/what-is-m-and-how-do-i-get-rid-of-it

Python by default defaults to \n, unix format since a long while ago: https://peps.python.org/pep-0278/

Describe alternatives you've considered

I know pre-commit-hooks has an option called mixed-line-ending but not sure about the custom solution MDA uses.
pre-commit/pre-commit-hooks#233
https://github.com/pre-commit/pre-commit-hooks

Or we can continue to rely on contributors to run dos2unix (or something equivalent).

Additional context

Learned something new today :)

[udaysahu09]

Hi, I’ve been exploring this issue and wanted to share a possible approach.

The root cause seems to be inconsistent end-of-line characters (\r\n vs \n), which leads to misleading diffs where entire files appear modified despite no logical changes. This can make code reviews unnecessarily difficult and noisy.

A practical way to address this is to enforce consistent line endings at the tooling level rather than relying on contributors to manually normalize files. One approach is to introduce a pre-commit hook that automatically standardizes line endings before changes are committed.

For example, using the mixed-line-ending hook from pre-commit-hooks with --fix=lf would ensure all files are converted to Unix-style line endings (\n) consistently across platforms.

Conceptually, the flow would look like:

Developer edits file (Windows/Linux)
        ↓
Pre-commit hook runs
        ↓
Line endings normalized (\r\n → \n)
        ↓
Clean commit (no noisy diffs)

This keeps commits focused on actual code changes and avoids large, irrelevant diffs caused by formatting inconsistencies.

One thing to consider is scope: enabling this across the entire repository might touch many existing files, so it may be preferable to either:

• apply it incrementally, or
• limit it to newly modified files to avoid large one-time diffs.

Happy to experiment with a minimal configuration or adapt this approach based on maintainers’ preferences.

[orbeckst]

Ideally, git is configured appropriately (eg see PHY432 / version control / line endings) although I agree that it would be good to just flag contributions that come in with DOS line endings. The easiest solution might then simply be to tell the contributor to set

git config --global core.autocrlf true

Do you know how to configure the linter?

[Urvity03]

Hi! I'd like to work on this issue if it's still available. I'll investigate the current linting/pre-commit configuration and propose a solution.

[jeremyleung521]

Ideally, git is configured appropriately (eg see PHY432 / version control / line endings) although I agree that it would be good to just flag contributions that come in with DOS line endings. The easiest solution might then simply be to tell the contributor to set

git config --global core.autocrlf true

Do you know how to configure the linter?

Not sure, but looks like we can just add * text eol=lf in .gitattributes and run a renormalize (git add --renormalize .)

https://docs.github.com/en/get-started/git-basics/configuring-git-to-handle-line-endings#example

[jeremyleung521]

Ideally, git is configured appropriately (eg see PHY432 / version control / line endings) although I agree that it would be good to just flag contributions that come in with DOS line endings. The easiest solution might then simply be to tell the contributor to set
git config --global core.autocrlf true
Do you know how to configure the linter?

Not sure, but looks like we can just add * text eol=lf in .gitattributes and run a renormalize (git add --renormalize .)

https://docs.github.com/en/get-started/git-basics/configuring-git-to-handle-line-endings#example

Just submitted #5446 with the fixes in proposed above.

Regardless of whether #5445 gets merged in due to AI usage or not, the scope of that PR is to check whether bad commits went through. #5446 preemptively deals with problems before commit + combs through existing files. #5446 should be merged before #5445 (since this will solve CI/lint failure on that PR) should #5445 is deemed still needed.

[Urvity03]

Thanks for the clarification. I saw the discussion on issue #5315 and understand that #5446 is intended to normalize the existing line endings first, with this PR enabling the Pylint check afterwards. I'm happy to rebase or update this PR once #5446 is merged if that's the preferred workflow