No module named `zlib`

[radu-matei]

I think zlib is part of the standard library — https://docs.python.org/3/library/zlib.html.

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: ModuleNotFoundError: No module named 'zlib'', crates/spin-python-engine/src/lib.rs:385:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Error: the `wizer.initialize` function trapped

Caused by:
    wasm trap: wasm `unreachable` instruction executed
    wasm backtrace:
        0: 0x33f727 - <unknown>!__rust_start_panic
    note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable to may show more debugging information
    
Error: Couldn't create wasm from input

[jasonwashburn]

I played around with this briefly, trying out some related "fixes" from other python WASM compiled projects, but didn't make any real headway. Willing to keep poking at it as I have time since it seems to be the biggest thing preventing pip installation of a number of common packages I tried, but to prevent reinventing the wheel, just wondering if anyone has any existing thoughts or theories on what the problem might be or potential fixes that haven't been explored fully?

[dicej]

Looks like zlib is a native module which is conditionally compiled into CPython only if the upstream zlib C library is detected at configuration time: https://github.com/python/cpython/blob/a3a5b4bb232ba29875147baa3b56a5524eece353/PC/config.c#L58-L60. I'm guessing that's not being found when we build CPython for WASI, and thus the module is not included.

I haven't dug any deeper than that yet, but my guess is we need to supply a libz.a (built using wasi-sdk) and tell the CPython configuration machinery where to find it.

[jasonwashburn]

Ok, that was my take on it as well, I did try building/linking it to libz that I'd previously compiled using the wasi-sdk without effect previously, albeit it was fairly late one night and I wasn't being particularly systematic about my approach at the time, so its probably worth attempting again in a more controlled matter. I'll see if I can figure anything out.

[radu-matei]

Update: python/cpython#91246 (comment)