wasmtime-wasi-nn is Wasmtime’s host-side implementation of the wasi:nn interface. It lets a Wasmtime host provide machine-learning support to a WebAssembly component without implementing wasi:nn manually. section 5.4.1 section 5.4.3
A typical ONNX-enabled host integration follows these steps:
- Add wasmtime-wasi-nn with the onnx feature, which includes the ONNX backend alongside the host implementation. section 5.4.1
- Extend host state with WasiNnCtx, alongside the existing WASI context and application state. section 5.4.3
- Call preload to load available machine-learning backends and create a graph registry, then construct WasiNnCtx. section 5.4.3
- Register wasi:nn with the linker through add_to_linker, allowing the component to invoke neural-network functionality during execution. section 5.4.3
ONNX story generation in SmartCMS
The smartcms_test_host project combines key-value storage, WASI, and wasmtime-wasi-nn. Its component imports wasi:nn, the host preloads backends, links WasiNnCtx, and runs the component with cargo run --release, producing a generated story. section 5.4.1 section 5.4.3
Listing 5.6 Cargo.toml file for the smartcms_test_host project
[package]
name = "smartcms_test_host"
version = "0.1.0"
edition = "2021"
[dependencies]
wasmtime = "30"
wasmtime-wasi = "30"
wasmtime-wasi-nn = { version = "30",
➥features = ["onnx"] } #1
ort = { version = "=2.0.0-rc.6", default-features
➥= false, features = ["copy-dylibs",
➥"download-binaries"] } #2
ort-sys = { version = "=2.0.0-rc.6", #2
➥default-features = false, features = #2
➥["copy-dylibs", "download-binaries"] } #2
Inspect the host Cargo.toml configuration for Wasmtime, wasmtime-wasi-nn with ONNX support, and the ONNX runtime dependencies. section 5.4.1
let (backends, registry) = wasmtime_wasi_nn::preload(&[]).unwrap();
let wasi_nn_ctx = wasmtime_wasi_nn::wit::WasiNnCtx::new(backends, registry);
let state = State {
key_value: KeyValue { mem: std::collections::HashMap::new() },
wasi: (wasi_ctx, wasi_table),
wasi_nn: wasi_nn_ctx,
};
let mut store = wasmtime::Store::new(&engine, state);
Inspect the preload and WasiNnCtx initialization used to prepare machine-learning backends and graph storage. section 5.4.3
wasmtime_wasi_nn::wit::add_to_linker(&mut linker, |state: &mut State| {
wasmtime_wasi_nn::wit::WasiNnView::new(&mut state.wasi.1, &mut
➥state.wasi_nn)
}).unwrap();
component::smartcms::kvstore::add_to_linker(&mut linker, |state: &mut
➥State| &mut state.key_value).unwrap();
let app = App::instantiate(&mut store, &component, &linker).unwrap();
Inspect the linker registration that exposes wasi:nn and key-value storage before component instantiation. section 5.4.3
