undefined becomes null after deserializing

const msgpack = require('@msgpack/msgpack');

let original = {x: undefined};
let serialized = msgpack.encode(original, {extensionCodec});
let deserialized = msgpack.decode(serialized, {extensionCodec});
console.log(original, deserialized, original.x === deserialized.x);

prints "{ x: undefined } { x: null } false". This is a problem because now code checking for strict equality fails.

I was also not able to handle this in an extension codec because that is only called for non-primitive values.

ignoreUndefined: true doesn't work in all cases:

let original = undefined;
let serialized = msgpack.encode(original, {ignoreUndefined: true});
let deserialized = msgpack.decode(serialized, {ignoreUndefined: true});
console.log(original, deserialized, original === deserialized); // undefined null false

This is stated in the readme, but is there no way to retain this with an extension codec?

Can you elaborate on your use case? In general, it's not a good idea to distinguish null and undefined in JavaScript, and supporting it with extension types could lead to extra overheads, so I'd like to know your use cases are worth adding more overheads.

Once case that I found was where we serialize/deserialize an entry of process.env (so {key: "foo", value: process.env["foo"]}) and then check if deserialized.value !== process.env.foo (and this breaks if the value is undefined). It's easy to workaround, but I fear there might be more situations like this in the codebase that haven't surfaced yet.

(I'm using msgpack as a browser replacement for require("v8").(de)serialize)

May I suggest allowing for overriding this in a codec? Currently undefined is handled internally by this package, and you can configure whether to a) ignore it (not include it in the encoded buffer) or b) coerce it into null. I understand that this makes sense when communication between different languages, where there is no distinction between undefined and null, which is why it's nil in the spec.

If this could instead be allowed to be overridden though, users can encode it if they want.

My use case is to transfer an object from one place to another and maintain it exactly as is. Turning undefined into null alters the datatype, and allowing undefined to be there is different from not having it. Object.keys(obj) e.g.

@gfx

@gfx
Suppose to encode the entire arguments of functions that have optional ones.
The values of the optional arguments are not null but undefined.
If the msgpack coearses them into null, the information that they were not supplied is lost.
We have to provide the additional bitmask that informs which parameters were really supplied.
The undefined means the existence of the key in the object.
The null means the absence of the value. They can't be treated as same things.

deserialized.value != process.env.foo would solve this in JS land as == with null and undefined it's the only case where it's true, everything else would be false, even false itself so this was trivial to solve to start with but I wonder how comes MessagePack decided to encode undefined at all ... I can see the == used here and there, but undefined usually means not defined and even JSON got it right, it's entirely pointless to carry non defined values, especially when every other PL might have just nil or explicit None around, but never undefined.

@gfx

Can you elaborate on your use case? In general, it's not a good idea to distinguish null and undefined in JavaScript, and supporting it with extension types could lead to extra overheads, so I'd like to know your use cases are worth adding more overheads.

@gfx can you elaborate on "not a good idea to distinguish"? It's 2 separate types in JavaScript. Any place that checks value === null won't work for undefined and vice-versa. Personally I think that in general people expect serialization/deserialization cycle to be loss-less. I was definitely caught off-guard with this one.

It's okay to be opinionated about it by default. But I think at least it'd be nice to allow people to extends it with extension codec instead of not providing them any path forward.

Navigation Menu

Repository navigation

`undefined` becomes `null` after deserializing #150

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

undefined becomes null after deserializing #150

Description

Activity

gfx commented on Dec 23, 2020

mischnic commented on Feb 8, 2021

grantila commented on Oct 23, 2021

genki commented on Jan 23, 2024

WebReflection commented on Feb 21, 2025

gyzerok commented on Jun 26, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

`undefined` becomes `null` after deserializing #150