Skip to content

Commit 0c7da46

Browse files
authored
blosc: add build-blosc.yml for riscv64 wheels (#1500)
* **Package**: `blosc` * **Version**: `1.11.4` * **Source**: https://github.com/blosc/python-blosc * **Docs**: https://github.com/blosc/python-blosc Compiles the vendored c-blosc C library (with LZ4 and zlib codecs) into a Python extension via CMake/scikit-build. Upstream publishes no riscv64 wheel. Mirrors [upstream's `cibuildwheels.yml`](https://github.com/blosc/python-blosc/blob/v1.11.4/.github/workflows/cibuildwheels.yml). **Differs from upstream** - Nothing beyond the riscv64 image. **Testing** - same as upstream, `python -m blosc.test` **License**: Wheel bundles c-blosc's own BSD licence plus its statically-linked LZ4 (BSD) and zlib (zlib licence); the vendored zstd (BSD) copy ships no licence file at all, so the patch restores one. **Patches** - `0001-Ship-the-licences-of-what-the-wheel-actually-bundles.patch` - To upstream (not submitted, no cross-repo issue/PR). Adds a `license_files` list to setup.py; without it only the wrapper's own LICENSE.txt ships. Reproduces off riscv64 too (verified locally on macOS arm64, an equally SIMD-unsupported architecture for this c-blosc version). Built and tested locally (macOS arm64, no SIMD path, same fallback riscv64 will take): 38 passed, 1 skipped (psutil-gated leak test, same as upstream's own CI).
1 parent b96f7b7 commit 0c7da46

2 files changed

Lines changed: 208 additions & 0 deletions

File tree

.github/workflows/build-blosc.yml

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
# SPDX-FileCopyrightText: 2026 The RISE Project
2+
# SPDX-License-Identifier: MIT
3+
---
4+
# This workflow is based on the `build_wheels` job of
5+
# https://github.com/blosc/python-blosc/blob/v1.11.4/.github/workflows/cibuildwheels.yml
6+
name: Build blosc wheels (riscv64)
7+
8+
on:
9+
workflow_dispatch:
10+
inputs:
11+
version:
12+
description: 'blosc version to build (git tag without leading v, e.g. 1.11.4)'
13+
required: true
14+
default: '1.11.4'
15+
pull_request:
16+
paths:
17+
- '.github/workflows/build-blosc.yml'
18+
- 'patches/blosc/**'
19+
20+
concurrency:
21+
group: ${{ github.workflow }}-${{ inputs.version || '1.11.4' }}-${{ github.head_ref || github.run_id }}
22+
cancel-in-progress: true
23+
24+
permissions:
25+
contents: read # to fetch code (actions/checkout)
26+
27+
env:
28+
# `inputs.version` is empty on pull_request events; default to 1.11.4 there.
29+
BLOSC_VERSION: ${{ inputs.version || '1.11.4' }}
30+
MANYLINUX_RISCV64_IMAGE: quay.io/pypa/manylinux_2_39_riscv64
31+
32+
jobs:
33+
setup:
34+
uses: $/.github/workflows/_setup.yml
35+
36+
build_wheels:
37+
needs: [setup]
38+
name: Build blosc ${{ inputs.version || '1.11.4' }} ${{ matrix.python }}-manylinux_riscv64
39+
runs-on: ubuntu-24.04-riscv
40+
timeout-minutes: 45
41+
strategy:
42+
fail-fast: false
43+
matrix:
44+
python: [cp312, cp313, cp314, cp314t]
45+
46+
steps:
47+
- name: Checkout python-blosc v${{ env.BLOSC_VERSION }}
48+
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
49+
with:
50+
repository: blosc/python-blosc
51+
ref: v${{ env.BLOSC_VERSION }}
52+
submodules: recursive
53+
persist-credentials: false
54+
55+
- name: Checkout python-wheels
56+
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
57+
with:
58+
path: python-wheels
59+
persist-credentials: false
60+
61+
- name: Patch python-blosc source
62+
run: git apply python-wheels/patches/blosc/${{ env.BLOSC_VERSION }}/*.patch
63+
64+
- uses: pypa/cibuildwheel@1828c10ab37f080699c7b81cea34097c684a7074 # v4.2.0
65+
with:
66+
output-dir: wheelhouse/
67+
only: ${{ matrix.python }}-manylinux_riscv64
68+
env:
69+
CIBW_MANYLINUX_RISCV64_IMAGE: ${{ env.MANYLINUX_RISCV64_IMAGE }}
70+
# C-Blosc's own CMakeLists.txt gates SSE2/AVX2 on a recognized x86
71+
# CMAKE_SYSTEM_PROCESSOR and warns-not-fails otherwise, so riscv64
72+
# falls straight back to its portable shuffle-generic.c path -- no
73+
# patch needed for the SIMD codec gating itself.
74+
CIBW_ENVIRONMENT: >-
75+
PIP_EXTRA_INDEX_URL=https://pypi.riseproject.dev/simple/
76+
PIP_ONLY_BINARY=numpy
77+
CIBW_TEST_REQUIRES: numpy
78+
CIBW_TEST_COMMAND: python -m blosc.test
79+
80+
- name: Check wheel contents
81+
run: |
82+
python3 - wheelhouse/*.whl <<'EOF'
83+
import sys, zipfile
84+
85+
names = zipfile.ZipFile(sys.argv[1]).namelist()
86+
exts = {n for n in names if n.startswith("blosc/") and n.endswith(".so")}
87+
assert len(exts) == 1 and next(iter(exts)).startswith("blosc/blosc_extension"), exts
88+
89+
licences = {n.split(".dist-info/licenses/", 1)[1] for n in names
90+
if ".dist-info/licenses/" in n and not n.endswith("/")}
91+
assert licences == {
92+
"LICENSE.txt",
93+
"blosc/c-blosc/LICENSE.txt",
94+
"blosc/c-blosc/LICENSES/BITSHUFFLE.txt",
95+
"blosc/c-blosc/LICENSES/FASTLZ.txt",
96+
"blosc/c-blosc/LICENSES/LZ4.txt",
97+
"blosc/c-blosc/LICENSES/ZLIB.txt",
98+
"blosc/c-blosc/LICENSES/ZSTD.txt",
99+
}, licences
100+
EOF
101+
102+
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
103+
with:
104+
name: blosc-${{ env.BLOSC_VERSION }}-${{ matrix.python }}-manylinux_riscv64
105+
path: wheelhouse/*.whl
106+
if-no-files-found: error
107+
108+
publish:
109+
name: Publish blosc ${{ inputs.version || '1.11.4' }}
110+
needs: [setup, build_wheels]
111+
permissions:
112+
contents: write
113+
pull-requests: write
114+
uses: $/.github/workflows/_publish-wheel.yml
115+
with:
116+
artifact-pattern: blosc-${{ inputs.version || '1.11.4' }}-*-manylinux_riscv64
Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
2+
From: Ludovic Henry <git@ludovic.dev>
3+
Date: Tue, 8 Sep 2026 00:00:00 +0000
4+
Subject: [PATCH] Ship the licences of what the wheel actually bundles
5+
6+
setup.py sets no `license_files`, so setuptools falls back to its
7+
default `LICEN[CS]E*` glob at the project root, which only ever picks
8+
up this project's own LICENSE.txt (confirmed by building the wheel
9+
locally: `blosc-1.11.4.dist-info/licenses/` held just that one file).
10+
11+
That leaves out the licences of the c-blosc submodule this build
12+
statically compiles into the extension: c-blosc's own LICENSE.txt
13+
(blosc.c, blosclz.c, shuffle.c, ...), and the codecs its CMakeLists.txt
14+
includes by default (LZ4, zlib, and the FastLZ- and Bitshuffle-derived
15+
code in blosclz.c/bitshuffle-generic.c, all credited in
16+
c-blosc/LICENSES/). Snappy stays out (deactivated by default, matching
17+
this build) and so does the Windows-only STDINT.txt shim.
18+
19+
c-blosc/internal-complibs/zstd-1.5.6 is missing a licence file
20+
entirely -- confirmed byte-identical to upstream zstd v1.5.6's lib/
21+
tree, whose LICENSE (BSD, at the zstd repo root) never got carried
22+
over when it was vendored. Restoring that exact file lets
23+
license_files pick it up the same way as the other codecs' notices,
24+
rather than leaving zstd unlicensed in the wheel.
25+
26+
Upstream-Status: To upstream [not submitted: no cross-repo issue/PR contributions from this port]
27+
28+
Signed-off-by: Ludovic Henry <git@ludovic.dev>
29+
---
30+
blosc/c-blosc/LICENSES/ZSTD.txt | 30 ++++++++++++++++++++++++++++++
31+
setup.py | 9 +++++++++
32+
2 files changed, 39 insertions(+)
33+
create mode 100644 blosc/c-blosc/LICENSES/ZSTD.txt
34+
35+
diff --git a/blosc/c-blosc/LICENSES/ZSTD.txt b/blosc/c-blosc/LICENSES/ZSTD.txt
36+
new file mode 100644
37+
index 0000000..7580028
38+
--- /dev/null
39+
+++ b/blosc/c-blosc/LICENSES/ZSTD.txt
40+
@@ -0,0 +1,30 @@
41+
+BSD License
42+
+
43+
+For Zstandard software
44+
+
45+
+Copyright (c) Meta Platforms, Inc. and affiliates. All rights reserved.
46+
+
47+
+Redistribution and use in source and binary forms, with or without modification,
48+
+are permitted provided that the following conditions are met:
49+
+
50+
+ * Redistributions of source code must retain the above copyright notice, this
51+
+ list of conditions and the following disclaimer.
52+
+
53+
+ * Redistributions in binary form must reproduce the above copyright notice,
54+
+ this list of conditions and the following disclaimer in the documentation
55+
+ and/or other materials provided with the distribution.
56+
+
57+
+ * Neither the name Facebook, nor Meta, nor the names of its contributors may
58+
+ be used to endorse or promote products derived from this software without
59+
+ specific prior written permission.
60+
+
61+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
62+
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
63+
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
64+
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
65+
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
66+
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
67+
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
68+
+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
69+
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
70+
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
71+
diff --git a/setup.py b/setup.py
72+
index 8f1748d..2841ef3 100644
73+
--- a/setup.py
74+
+++ b/setup.py
75+
@@ -98,6 +98,15 @@ if __name__ == '__main__':
76+
setup_requires=['scikit-build'],
77+
tests_require=['numpy', 'psutil'],
78+
packages = ['blosc'],
79+
+ license_files = [
80+
+ 'LICENSE.txt',
81+
+ 'blosc/c-blosc/LICENSE.txt',
82+
+ 'blosc/c-blosc/LICENSES/BITSHUFFLE.txt',
83+
+ 'blosc/c-blosc/LICENSES/FASTLZ.txt',
84+
+ 'blosc/c-blosc/LICENSES/LZ4.txt',
85+
+ 'blosc/c-blosc/LICENSES/ZLIB.txt',
86+
+ 'blosc/c-blosc/LICENSES/ZSTD.txt',
87+
+ ],
88+
)
89+
elif __name__ == '__mp_main__':
90+
# This occurs from `cpuinfo 4.0.0` using multiprocessing to interrogate the
91+
--
92+
2.43.0

0 commit comments

Comments
 (0)