Upgrade to Erlang/OTP 28 and Elixir 1.18.4 - #5109
Open
elias-ba wants to merge 2 commits into
Open
Conversation
OTP 27's :unicode.characters_to_nfc_binary/1 recomposes only at the head of a string. A Bengali, Tamil, Malayalam or Sinhala two-part vowel is decomposed and never put back, and a combining mark can be reordered across a character whose job is to block exactly that, which produces a string that is not equivalent to the one that went in. OTP 28 fixes both, as collateral of its Unicode 16 upgrade. It was never reported and carries no release note of its own. Lightning does not normalise anything today, so nothing depends on the old behaviour. That changes with #4577, which is why this goes first. Five other things move with it. mix_test_watch to 1.3+, because OTP 28 compiles a regex to a reference and 1.2 tries to escape one into a struct default. Dialyzer gets flags: [:no_opaque]: OTP 28 reworked opaque handling for EEP-69 nominal types, Elixir inlines MapSet.new/0, and every Ecto.Multi call downstream is flagged, 62 of them. Ecto declined to change its internals and Elixir 1.20 removes the opacity, so this comes out then. The task_worker skip in .dialyzer_ignore.exs is redundant under that flag. Three doctests in hex.ex compared regexes by equality, which OTP 28 no longer permits. And the Docker base moves to bookworm-20260421, because the pinned snapshot has no OTP 28 build.
Security Review ✅
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #5109 +/- ##
=====================================
Coverage 90.7% 90.7%
=====================================
Files 422 422
Lines 20151 20151
=====================================
Hits 18276 18276
Misses 1875 1875 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
elias-ba
added a commit
to OpenFn/apollo
that referenced
this pull request
Sep 1, 2026
The hand-written composer existed to reproduce Erlang 27's normaliser, because Lightning runs on it and the two have to agree on how a name is spelled. OpenFn/lightning#5109 moves Lightning to Erlang 28, which fixes the bug. Copying it stops being the right thing at that point. Measured against Erlang 28 over the 72,269-row corpus, restricted to inputs that could be a step name: the standard library disagrees on 16,622 and the hand-written composer on 16,898. So it is now marginally worse than one line. What remains in both is a Hangul shape that only a half-finished IME produces. Real Korean, Japanese, French and Vietnamese words normalise identically under Erlang 28, Python and ICU. Sixteen rows of a realistic corpus change, all Bengali and Tamil two-part vowels. They are exactly the rows where Erlang 27 and Erlang 28 disagree, and the new output matches Erlang 28 on every one. That is the bug going away. The grapheme clusterer, the trim set and their parity checks stay. They have nothing to do with composition and they guard a real rejection. One thing this does not do. The grapheme tables are still generated from Erlang 27, so they will need regenerating against 28 once Lightning actually ships it. Doing that now would make Apollo disagree with the Lightning that is running.
This was referenced Sep 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR moves us to Erlang 28 and Elixir 1.18.4.
The reason is a bug in Erlang 27. When it tidies up how a name is spelled, it only finishes the job on the first character and leaves the rest half done. In many languages that breaks the word. A vowel gets taken apart and never put back together, or an accent lands on the wrong letter, which quietly turns the name into something else.
This came out of #4577. Lightning doesn't tidy up names anywhere today, so nothing is broken right now, but that PR adds it on every name in the app. If it lands on Erlang 27 we start writing names into the database in the broken spelling, and then someone has to migrate them back out later. It seemed better to move the runtime first while there is nothing to migrate.
Erlang 28 fixes it. It was fixed as a side effect of their Unicode upgrade rather than as a bug in its own right, so there is no release note for it and nobody ever reported it.
Five other things had to move along with the runtime.
mix_test_watchgoes to 1.3. Erlang 28 handles regexes differently and 1.2 no longer compiles.:no_opaqueflag. Erlang 28 changed how it checks opaque types and now reports 62 warnings aboutEcto.Multithat are not real. The comment inmix.exssays where that came from and when to take it out.hex.excompared two regexes to each other, which Erlang 28 doesn't allow.Validation steps
asdf install, thenmix deps.get. Onlymix_test_watchshould move inmix.lock.mix compile --warnings-as-errors, thenmix test --exclude integration,cd assets && npm test, andmix dialyzer.iex -S mixand run:unicode.characters_to_nfc_binary(<<0x0995::utf8, 0x09CB::utf8>>). You should get the same two codepoints back. On Erlang 27 you get three, which is the bug.Additional notes for the reviewer
@adaptor_formatinadaptor_registry.exquietly gets stricter, and you will not see it in the diff because the file does not change. The pattern has no/uflag, so the old regex engine read bytes as Latin-1 and matched parts of UTF-8 characters as letters by accident. The new one does not. Nothing realistic changes, but an adaptor name with a non-ASCII character in it would now be rejected where it used to slip through.AI Usage
Please disclose whether you've used AI anywhere in this PR (it's cool, we just want to know!):
You can read more details in our Responsible AI Policy
Pre-submission checklist
/reviewwith Claude Code):owner,:admin,:editor,:viewer)