Skip to content

Provide prebuilt native libraries by NuGet Packages.#257

Draft
ha-ves wants to merge 10 commits intoNetCordDev:alphafrom
ha-ves:feature/prebuilt-natives
Draft

Provide prebuilt native libraries by NuGet Packages.#257
ha-ves wants to merge 10 commits intoNetCordDev:alphafrom
ha-ves:feature/prebuilt-natives

Conversation

@ha-ves
Copy link
Copy Markdown

@ha-ves ha-ves commented Feb 9, 2026

Summary

Continues from #245. Implements CI-driven distribution of prebuilt native binary dependencies.

  • Download size: ~112 MB
  • Extracted size: ~429 MB

Supported Platforms & Build Variants

Platform RID NativeAOT Support Built Actual Test?
Windows x64 win-x64 ✓ (Static CRT /MT) ☑️
Windows ARM64 win-arm64 ✓ (Static CRT /MT) ☑️
Linux x64 linux-x64 ✓ (Dynamic CRT) ☑️
Linux ARM64 linux-arm64 ✓ (Dynamic CRT) ☑️
macOS x64 osx-x64 ✓ (Dynamic CRT) ☑️
macOS ARM64 osx-arm64 ✓ (Dynamic CRT) ☑️

Overview

This PR provides prebuilt native libraries via NuGet. The primary, provided distribution is a top-level package (NetCord.Natives) that includes prebuilt artifacts; per-RID packages are an option produced when building from source (local or CI) for users who prefer minimal per-platform artifacts. Each package bundles:

  • Runtime binaries (.dll, .so, .dylib) for standard .NET consumption
  • Static libraries for NativeAOT compilation
  • MSBuild integration to transparently resolve native paths based on target platform
  • Bundled licenses for all included dependencies

Packages are published automatically on successful CI builds to the configured NuGet feed.

Structure & Packaging Model

Dual Distribution Model:

  1. Top-Level Package (NetCord.Natives): Bundles all RID variants in a single package

    • Contains runtime binaries and static libraries for all 6 supported RIDs
    • ⚠️ Large package size due to static library payload for every platform (NativeAOT support)
    • Best for: multi-platform CI/CD, convenience, simplified dependency management
  2. Per-RID Targeted Packages: Produced when building from source (local or CI builds). These are not required for consumers who install the provided top-level package.

  • Intended for users building natives from source or CI jobs that produce per-RID artifacts
  • Package ID format: NetCord.Natives.{win|linux|osx}-{x64|arm64}
  • Example: NetCord.Natives.linux-x64, NetCord.Natives.win-arm64
  • Best for: producing minimal per-platform artifacts when building from source

Package Contents (both top-level and per-RID):

runtimes/{rid}/native/              ← Runtime binaries for standard .NET
staticlibs/{rid}/                   ← Static libraries for NativeAOT builds
build/NetCord.Natives.{rid}.*       ← MSBuild props/targets per platform
licenses/                           ← Copyright files from vcpkg ports

What Changed

Core Build & Packaging

  • NetCord.Natives.csproj: RID enumeration, conditional CI packaging, artifact aggregation, license extraction
  • .github/workflows/build-natives.yml: Parallel multi-platform CI matrix (6 RIDs), artifact caching, NuGet publishing pipeline
  • Custom vcpkg ports (natives-ports/):
    • libdave/portfile.cmake — builds from discord/libdave with MSVC patch
    • mlspp/portfile.cmake — provides static build for libdave's cryptographic dependencies

MSBuild Integration

Runtime Metadata

  • NativesHelper.cs: NativeLibraryVersionAttribute for runtime version discovery of bundled native libs

How to Use

Option 1: Top-Level Package (Simplest)

Add a single package reference for all platforms:

<ItemGroup>
  <PackageReference Include="NetCord.Natives" Version="1.0.0" />
</ItemGroup>

MSBuild automatically selects the correct RID variant at build time. Note: this package includes static libraries for all 6 RIDs, resulting in a larger download/installation footprint.

Option 2: Per-RID Targeted Package (Minimal Footprint)

If you build natives from source or use CI-produced per-RID artifacts, add only the package for your target platform:

<ItemGroup>
  <PackageReference Include="NetCord.Natives.win-x64" Version="1.0.0" />
</ItemGroup>

Smaller download and disk usage; useful when you or your CI produces per-RID packages from source.

NativeAOT Trimmed Deployments

Use the top-level or per-RID package (static libraries are automatically linked, runtime binaries excluded for AOT):

<ItemGroup>
  <PackageReference Include="NetCord.Natives" Version="1.0.0" />
  <!-- or for minimal footprint: -->
  <!-- <PackageReference Include="NetCord.Natives.linux-x64" Version="1.0.0" /> -->
</ItemGroup>
<PropertyGroup>
  <PublishAot>true</PublishAot>
  <RuntimeIdentifier>linux-x64</RuntimeIdentifier>
</PropertyGroup>

Multi-Target Build with Per-RID Packages

For projects targeting multiple platforms with minimal per-platform size:

<ItemGroup Condition="'$(RuntimeIdentifier)' == 'win-x64'">
  <PackageReference Include="NetCord.Natives.win-x64" Version="1.0.0" />
</ItemGroup>
<ItemGroup Condition="'$(RuntimeIdentifier)' == 'linux-x64'">
  <PackageReference Include="NetCord.Natives.linux-x64" Version="1.0.0" />
</ItemGroup>

Alternatively, use the top-level package and MSBuild handles RID selection automatically.

Maintainer Notes: vcpkg Baseline & Dependency Pinning

All native dependencies are pinned to a specific vcpkg baseline (vcpkg.json) to ensure reproducible builds.

When upgrading native library versions:

  1. Update version constraints in vcpkg.json
  2. Run local vcpkg build to validate against all RID triplets
  3. Update CI workflow caches if baseline changes
  4. Verify license extraction succeeds (hardcoded paths in csproj line 163–167)

Known build constraints:

  • ARM64 cross-compile: Linux ARM64 builds on Ubuntu runner require gcc-aarch64-linux-gnu cross-compiler toolchain
  • Memory limits: Native library builds cap parallelism with VCPKG_CONCURRENCY=4 to avoid OOM on resource-constrained runners

Testing

Comprehensive native library validation via Tests/NetCord.Natives.Tests/:

Unit Tests (NativesBuildTests.cs)

Framework: MSTest with method-level parallelization
Target: .NET 10.0

Three independent test methods validate the entire native stack:

NativeLoaded — Runtime Library Resolution

[TestMethod]
[DataRow("libdave")]
[DataRow("libsodium")]
[DataRow("opus")]
[DataRow("zstd")]
public void NativeLoaded(string libName)
  • Validates each of the four core native libraries can be successfully loaded via NativeLibrary.Load()
  • Runs on all supported platforms (test passes current RuntimeIdentifier)
  • Catches load failures, missing binaries, or corrupted artifacts

AllLibraryImportsExistInBinary — P/Invoke Symbol Verification

[TestMethod]
[DataRow("libdave", "NetCord.Gateway.Voice.Dave")]
[DataRow("libsodium", "NetCord.Gateway.Voice.Encryption.XChaCha20Poly1305")]
[DataRow("opus", "NetCord.Gateway.Voice.Opus")]
[DataRow("zstd", "NetCord.Gateway.Compression.Zstandard")]
public void AllLibraryImportsExistInBinary(string libName, string className)
  • Reflects over managed P/Invoke wrapper classes to extract all [LibraryImport] entry points
  • Loads each native binary and validates every exported symbol exists via NativeLibrary.TryGetExport()
  • Catches missing entry points or mismatched P/Invoke declarations

NativeAotStaticLinking — NativeAOT Full Integration

[TestMethod]
public void NativeAotStaticLinking(string libName)
  • Build phase: Invokes dotnet publish -c Release on NativeAotApp with NativeAOT enabled
    • Passes current RID to publish command (e.g., win-x64, linux-x64, osx-arm64)
    • Links static libraries for all four native libraries
  • Runtime phase: Executes the published AOT binary and validates:
    • All native calls execute successfully (non-zero version numbers returned)
    • Exit code is 0
    • Logs build and runtime output to TestContext for debugging

NativeAOT Test Application (Assets/NativeAotApp/)

Standalone console app configured for AOT compilation (PublishAot=true, IsAotCompatible=true):

Validation Logic (Program.cs):

  • Embeds direct P/Invoke calls to each native library:
    • libdave: daveMaxSupportedProtocolVersion()
    • libsodium: sodium_init()
    • opus: opus_get_version_string()
    • zstd: ZSTD_versionNumber()
  • Calls each native function to verify linking and successful execution at runtime
  • Returns exit code 0 if all native calls succeed; non-zero otherwise

Coverage: All 6 supported RIDs tested implicitly (test framework builds and runs for current platform)

Test Execution Workflow

  1. CI/local build: dotnet test NetCord.Natives.Tests.csproj
  2. Parallel execution: Methods run concurrently (MSTest MethodLevel parallelization)
  3. NativeAOT build: Triggered only on NativeAotStaticLinking test; uses current platform's runtime

Scope & Follow-Up Work

Included in This PR

✓ CI-driven multi-platform native package build and publish
✓ Dual distribution: top-level package + per-RID targeted packages
✓ NativeAOT static library support and MSBuild integration
✓ Custom vcpkg overlay ports for libdave and mlspp

Package Size Note

The top-level NetCord.Natives package bundles static libraries for all 6 RIDs, resulting in a significantly larger footprint compared to per-RID packages:

  • Download size: ~112 MB
  • Extracted size: ~429 MB

Choose based on your deployment scenario:

  • Top-level: Ideal for CI systems, multi-platform development, or convenience
  • Per-RID: Preferred for production deployments to minimize footprint per target platform

Out of Scope (Follow-Up Issues)

  • User documentation: instructions for referencing prebuilt packages in projects
  • User documentation: fallback build guide for unsupported platforms or custom RIDs
  • Per-native-library package split (future modularity enhancement)

@ha-ves ha-ves force-pushed the feature/prebuilt-natives branch from dd93de7 to 7ab57a9 Compare April 30, 2026 21:01
ha-ves added 5 commits May 5, 2026 21:36
* put into nuget package
* Add native ports for libdave and mlspp
* Update build workflows for native packaging
* Add local targets and CI helpers for natives
* Update README
- GitHub Actions tag triggers, NuGet publish job
- NativesHelper, NativeLibraryVersionAttribute
- vcpkg integration, static linking
- zstd support
- native library tests, import checks
- Native AOT test app
- NetCordNativesDir assembly metadata
* Add MSVC builtin add overflow patch for libdave
* Add vcpkg-non-windows.targets for cross-platform support
* Update libdave and mlspp portfiles with build fixes
* Update build-natives workflow and vcpkg.json versions
* Fix vcpkg dependency resolution for multi-platform builds
* Add dynamic library exclusion for AOT builds
* Add NativeAotApp test project
@ha-ves ha-ves force-pushed the feature/prebuilt-natives branch from 1c05c69 to 6565a35 Compare May 5, 2026 12:47
ha-ves added 5 commits May 6, 2026 03:58
* natives conditionals fix
* fix nativeaot test structure
* add natives tests to build workflow
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 5, 2026

The documentation preview is available at https://preview.netcord.dev/257.

@AraHaan
Copy link
Copy Markdown

AraHaan commented May 6, 2026

I feel like the best option is to do something like:
NetCord.Natives.<rid> that way only the target RID's static libraries are downloaded and used for NAOT and does not slow down the build with extracting unused native static libraries.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants