Skip to content

Return an error instead of throwing on invalid UTF-8 in deserialisation helpers #1249

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
palas wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Return an error instead of throwing on invalid UTF-8 in deserialisation helpers #1249

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
df4c0fd
Return an error instead of throwing on invalid UTF-8 in deserialisati…
palas Jul 8, 2026
3fec64a
Render the UTF-8 decoding error via a new Bech32InvalidUtf8 constructor
palas Jul 10, 2026
bc81804
Separate flow from error handling in deserialiseBech32 helpers
palas Jul 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changes/20260708_140000_cardano-api_pablo.lamela_nonthrowing_utf8_decode.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
description: |
Fixed `deserialiseInput`, `deserialiseInputAnyOf` and `deserialiseAnyVerificationKeyBech32` (and thereby `deserialiseAnyVerificationKey`) to return an error instead of throwing an impure exception when the input is not valid UTF-8. Added a `Bech32InvalidUtf8` constructor to `Bech32DecodeError`, which carries the rendered UTF-8 decoding error.
kind:
- bugfix
pr: 1249
project: cardano-api
9 changes: 7 additions & 2 deletions cardano-api/src/Cardano/Api/Key/Internal/SomeAddressVerificationKey.hs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,12 @@ import Cardano.Api.Serialise.TextEnvelope.Internal
import Cardano.Chain.Common qualified as Common
import Cardano.Crypto.Signing qualified as Crypto

import Control.Exception (displayException)
import Data.Aeson qualified as Aeson
import Data.Bifunctor (first)
import Data.ByteString (ByteString)
import Data.Text (Text)
import Data.Text qualified as Text
import Data.Text.Encoding qualified as Text
import Formatting (build, sformat, (%))

Expand Down Expand Up @@ -142,8 +144,11 @@ deserialiseAnyVerificationKey bs =

deserialiseAnyVerificationKeyBech32
:: ByteString -> Either Bech32DecodeError SomeAddressVerificationKey
deserialiseAnyVerificationKeyBech32 =
deserialiseAnyOfFromBech32 allBech32VerKey . Text.decodeUtf8
deserialiseAnyVerificationKeyBech32 bs =
case Text.decodeUtf8' bs of
-- The input is not valid UTF-8, so it cannot be valid Bech32.
Left err -> Left $ Bech32InvalidUtf8 $ Text.pack $ displayException err
Right text -> deserialiseAnyOfFromBech32 allBech32VerKey text
where
allBech32VerKey
:: [FromSomeType SerialiseAsBech32 SomeAddressVerificationKey]
Expand Down
5 changes: 5 additions & 0 deletions cardano-api/src/Cardano/Api/Serialise/Bech32.hs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,9 @@ data Bech32DecodeError
-- ^ Expected header
!Text
-- ^ Unexpected header
| -- | The input is not valid UTF-8, so it cannot be a Bech32-encoded
-- string. The field contains the rendered UTF-8 decoding error.
Bech32InvalidUtf8 !Text

@carbolymer carbolymer Jul 13, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the field is Text and not the exception itself? I'd suggest moving of rendering of the exception to Pretty instance instead of keeping it in two places.

deriving (Eq, Show, Data)

instance Error Bech32DecodeError where
Expand Down Expand Up @@ -181,3 +184,5 @@ instance Error Bech32DecodeError where
[ "Unexpected CIP-129 Bech32 header: the actual header is " <> pshow actual
, ", but it was expected to be " <> pshow expected
]
Bech32InvalidUtf8 decodeErr ->
"The Bech32-encoded string is not valid UTF-8: " <> pretty decodeErr
47 changes: 26 additions & 21 deletions cardano-api/src/Cardano/Api/Serialise/DeserialiseAnyOf.hs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@ import Data.ByteString.Char8 qualified as BSC
import Data.Char (toLower)
import Data.Data
import Data.List.NonEmpty (NonEmpty)
import Data.Text (Text)
import Data.Text.Encoding qualified as Text
import GHC.Exts (IsList (..))
import Prettyprinter
Expand Down Expand Up @@ -107,9 +106,6 @@ deserialiseInput
deserialiseInput acceptedFormats inputBs =
go (toList acceptedFormats)
where
inputText :: Text
inputText = Text.decodeUtf8 inputBs

go :: [InputFormat a] -> Either InputDecodeError a
go [] = Left InputInvalidError
go (kf : kfs) =
Expand Down Expand Up @@ -139,13 +135,19 @@ deserialiseInput acceptedFormats inputBs =
Left _ -> DeserialiseInputErrorFormatMismatch

deserialiseBech32 :: SerialiseAsBech32 a => DeserialiseInputResult a
deserialiseBech32 =
case deserialiseFromBech32 inputText of
Right res -> DeserialiseInputSuccess res
-- The input was not valid Bech32.
Left (Bech32DecodingError _) -> DeserialiseInputErrorFormatMismatch
-- The input was valid Bech32, but some other error occurred.
Left err -> DeserialiseInputError $ InputBech32DecodeError err
deserialiseBech32 = either id fromBech32 $ decodeText inputBs
where
-- The input was not valid UTF-8, so it cannot be valid Bech32.
decodeText =
first (const DeserialiseInputErrorFormatMismatch)
. Text.decodeUtf8'

@Jimbo4350 Jimbo4350 Jul 13, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are still throwing away the exception error here.

fromBech32 text =
case deserialiseFromBech32 text of
Right res -> DeserialiseInputSuccess res
-- The input was not valid Bech32.
Left (Bech32DecodingError _) -> DeserialiseInputErrorFormatMismatch
-- The input was valid Bech32, but some other error occurred.
Left err -> DeserialiseInputError $ InputBech32DecodeError err

deserialiseHex :: SerialiseAsRawBytes a => DeserialiseInputResult a
deserialiseHex
Expand Down Expand Up @@ -179,9 +181,6 @@ deserialiseInputAnyOf bech32Types textEnvTypes inputBs =
DeserialiseInputError err -> Left err
DeserialiseInputErrorFormatMismatch -> Left InputInvalidError
where
inputText :: Text
inputText = Text.decodeUtf8 inputBs

orTry
:: DeserialiseInputResult b
-> DeserialiseInputResult b
Expand All @@ -208,13 +207,19 @@ deserialiseInputAnyOf bech32Types textEnvTypes inputBs =
Left _ -> DeserialiseInputErrorFormatMismatch

deserialiseBech32 :: DeserialiseInputResult b
deserialiseBech32 =
case deserialiseAnyOfFromBech32 bech32Types inputText of
Right res -> DeserialiseInputSuccess res
-- The input was not valid Bech32.
Left (Bech32DecodingError _) -> DeserialiseInputErrorFormatMismatch
-- The input was valid Bech32, but some other error occurred.
Left err -> DeserialiseInputError $ InputBech32DecodeError err
deserialiseBech32 = either id fromBech32 $ decodeText inputBs
where
-- The input was not valid UTF-8, so it cannot be valid Bech32.
decodeText =

@Jimbo4350 Jimbo4350 Jul 13, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are still throwing away the exception error here.

first (const DeserialiseInputErrorFormatMismatch)
. Text.decodeUtf8'
fromBech32 text =
case deserialiseAnyOfFromBech32 bech32Types text of
Right res -> DeserialiseInputSuccess res
-- The input was not valid Bech32.
Left (Bech32DecodingError _) -> DeserialiseInputErrorFormatMismatch
-- The input was valid Bech32, but some other error occurred.
Left err -> DeserialiseInputError $ InputBech32DecodeError err

-- | Read formatted file
readFormattedFile
Expand Down
1 change: 1 addition & 0 deletions cardano-api/test/cardano-api-golden/Test/Golden/ErrorsSpec.hs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,7 @@ test_Bech32DecodeError =
, Bech32DeserialiseFromBytesError bytestring
, Bech32WrongPrefix text text
, Bech32UnexpectedHeader text text
, Bech32InvalidUtf8 text
]

test_InputDecodeError :: TestTree
Expand Down
1 change: 1 addition & 0 deletions ...-golden/files/errors/Cardano.Api.Serialise.Bech32.Bech32DecodeError/Bech32InvalidUtf8.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The Bech32-encoded string is not valid UTF-8: <text>
Loading