ort::execution_providers

Trait ExecutionProvider

Source 
pub trait ExecutionProvider: Send + Sync {
    // Required methods
    fn as_str(&self) -> &'static str;
    fn register(&self, session_builder: &mut SessionBuilder) -> Result<()>;

    // Provided methods
    fn supported_by_platform(&self) -> bool { ... }
    fn is_available(&self) -> Result<bool> { ... }
}
Expand description

ONNX Runtime works with different hardware acceleration libraries through its extensible Execution Providers (EP) framework to optimally execute the ONNX models on the hardware platform. This interface enables flexibility for the AP application developer to deploy their ONNX models in different environments in the cloud and the edge and optimize the execution by taking advantage of the compute capabilities of the platform.

Required Methods§

Source

fn as_str(&self) -> &'static str

Returns the identifier of this execution provider used internally by ONNX Runtime.

This is the same as what’s used in ONNX Runtime’s Python API to register this execution provider, i.e. TVMExecutionProvider’s identifier is TvmExecutionProvider.

Source

fn register(&self, session_builder: &mut SessionBuilder) -> Result<()>

Attempts to register this execution provider on the given session.

Provided Methods§

Source

fn supported_by_platform(&self) -> bool

Returns whether this execution provider is supported on this platform.

For example, the CoreML execution provider implements this as:

ⓘ
impl ExecutionProvider for CoreMLExecutionProvider {
	fn supported_by_platform() -> bool {
		cfg!(any(target_os = "macos", target_os = "ios"))
	}
}
Source

fn is_available(&self) -> Result<bool>

Returns Ok(true) if ONNX Runtime was compiled with support for this execution provider, and Ok(false) otherwise.

An Err may be returned if a serious internal error occurs, in which case your application should probably just abort.

Note that this does not always mean the execution provider is usable for a specific session. A model may use operators not supported by an execution provider, or the EP may encounter an error while attempting to load dependencies during session creation. In most cases (i.e. showing the user an error message if CUDA could not be enabled), you’ll instead want to manually register this EP via ExecutionProvider::register and detect and handle any errors returned by that function.

Implementors§