Wear Surfaces Datasets

Pipeline and viewer for high-resolution surface textures captured under controlled wear tests (distance × load × grit).

What's in the repo

data/
  parquet/              # Sequential Parquet leaves + sequences.json (aligned + Catmull + FFT/SLERP)
  shards/               # Sequential WebDataset shard-*.tar
  aligned/              # PNG align cache (gitignored)
  wear_extrapolated/    # PNG intermediate used to pack parquet/shards (gitignored)
docs/
  index.html            # WebGPU Material Viewer
  serve_local.py
  prepare_viewer_parquets.py
alignment/              # SIFT/ECC align + Catmull-Rom + FFT/SLERP bake
generate_datasets.py
generate_metadata.py
read_datasets.py
metadata.json           # Dataset parameter table + material profiles

Materials in the sequential viewer: Cartboard, FOAM, MDF, PLA, Sandpaper, Soft_Wood, Alu, Resin.

Parquet layout

data/parquet/<material>/<shape>/<direction>/S<grit>/<load>g/<distance>mm/data.parquet

Example: data/parquet/Cartboard/Raw/Normal/S60/200g/800mm/data.parquet

docs/static/parquet is a junction to data/parquet.

Quick start — WebGPU wear viewer

git clone https://github.com/ETSim/WearSurfacesDatasets
cd WearSurfacesDatasets
git lfs pull
pip install -r requirements.txt
python docs/serve_local.py
# open http://127.0.0.1:8080/docs/index.html

The local server mounts docs/ and data/ and exposes:

• GET /api/datasets — sequential dataset labels (all modes read data/parquet)
• GET /api/sequences?dataset=raw|aligned|catmull|extrapolated — same sequences.json
• POST /api/align/sequence — align normals ({ "sequence_id", "force?" })
• GET /api/align/sequence/{id} — cached alignment summary + PNG URLs

# one sequence then pack parquet + shards
python -m alignment.bake --sequence-id cartboard-raw-normal-s60-200g --pack

# all sequences then pack
python -m alignment.bake --all --pack

# resume missing wear_extrapolated sequences, then pack data/parquet + data/shards
python -m alignment.bake_extrapolate_all

Aligned / Catmull / Extrapolated all load data/parquet: aligned measured frames, Catmull midpoints, and FFT+SLERP future steps in one timeline.

1. Open the viewer and pick a sequence (e.g. Alu or Resin).
2. Click Load sequence.
3. Optionally Align sequence, or scrub Distance / Play.

Rebuild the sequence manifest:

python docs/prepare_viewer_parquets.py --source data/parquet --dest data/parquet

Install (Python pipeline)

pip install -r requirements.txt

Pipeline commands

python generate_datasets.py parquet --input-dir wear --output-dir data/parquet
python generate_datasets.py webdataset --input-dir wear --output-dir data/shards
python generate_datasets.py extract --parquet-dir data/parquet --output-dir data/images
python read_datasets.py decompress --shards-dir data/shards --output-dir data/images --workers 4

Metadata:

python generate_metadata.py leaf wear/
python generate_metadata.py index wear/

Dataset notes

• Maps in Parquet are JPEG bytes in image_data, labeled by map_type.
• Distance is the folder name (0mm, 400mm, Catmull midpoints, then extrapolated mm).
• data/parquet/sequences.json drives the WebGPU scrubber.
• Historical tokens Cartboard, Raw / Normal, and NoLinear are kept in paths.

License

MIT — see LICENSE.