[Go] Add Go bindings for ONNX Runtime C API#29615
Open
dannyota wants to merge 3 commits into
Open
Conversation
Idiomatic Go wrapper for the ORT C API via CGO, matching the pattern of existing language bindings (rust/, java/, csharp/). Covers session management, typed/untyped tensor I/O, string tensors, IO binding, model metadata, run options, session options with CUDA and TensorRT execution providers, context cancellation, and concurrent inference. Tested with real Qwen3-Embedding-0.6B ONNX INT8 model.
Author
|
@microsoft-github-policy-service agree |
Fix TOCTOU race in NewIOBinding (use-after-free on concurrent Close), empty outputNames panic in Run/RunWithOptions, and metadata use-after- close segfault. Add 15 tests covering all identified coverage gaps. Remove dead code. Bump minimum ORT version to 1.27.0 for getter APIs.
Author
|
Updated with a second commit:
|
Try API version 27 first, fall back to 17 for ORT 1.17-1.26. GetExecutionMode and IsMemPatternEnabled return a clear error on older libraries instead of crashing. Minimum ORT is now 1.17.
Author
|
Added API version fallback: bindings now try API version 27 first, fall back to 17 for older ORT libraries. This means ORT 1.17+ is supported (previously required 1.27+). The two getter functions added in 1.27 (GetExecutionMode, IsMemPatternEnabled) return a clear error on older libraries instead of crashing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Official Go bindings for the ONNX Runtime C API via CGO, following
the pattern of existing language bindings (
rust/,java/,csharp/).Closes #9786
Motivation
Go is widely used for backend services that need ML inference
(embeddings, classification, NLP). There is strong community demand
(#9786, 19 comments) and no official Go support. This PR fills that gap.
What's included
inputs/outputs with dynamic dimension support
CreateTensor[T](Go 1.18+ generics),untyped
NewTensorFromBytes, string tensors, zero-length dimensionsmemory arena, profiling, free dimension overrides
key-value API for all other EPs
context.Contextsupport viaRunOptionsSetTerminateSession.Runsafe for concurrent use (ORT guarantee)Design
Wraps the C API vtable (
OrtApi) through ~80 C shim functions incshim.h/cshim.c— CGO cannot call C function pointers directly.Library loaded at runtime via
dlopen(Linux/macOS) orLoadDLL(Windows). Singleton
OrtEnvper process viasync.Mutex.Zero-copy tensor inputs via
runtime.Pinner. Output tensors wrapORT-allocated memory with views valid until
Close().Module path:
github.com/microsoft/onnxruntime/goPackage:
onnxruntime(import asort)Go 1.26 minimum. stdlib + CGO only — no third-party dependencies.
Tests
46 tests passing with
-race: