Struct OpenVINOExecutionProvider

pub struct OpenVINOExecutionProvider { /* private fields */ }

Implementations

impl OpenVINOExecutionProvider

pub fn with_device_type(self, device_type: impl ToString) -> Self

Overrides the accelerator hardware type and precision with these values at runtime. If this option is not explicitly set, default hardware and precision specified during build time is used.

pub fn with_device_id(self, device_id: impl ToString) -> Self

Selects a particular hardware device for inference. If this option is not explicitly set, an arbitrary free device will be automatically selected by OpenVINO runtime.

pub fn with_num_threads(self, num_threads: usize) -> Self

Overrides the accelerator default value of number of threads with this value at runtime. If this option is not explicitly set, default value of 8 is used during build time.

pub fn with_cache_dir(self, dir: impl ToString) -> Self

Explicitly specify the path to save and load the blobs, enabling model caching.

pub fn with_opencl_context(self, context: *mut c_void) -> Self

This option is only alvailable when OpenVINO EP is built with OpenCL flags enabled. It takes in the remote context i.e the cl_context address as a void pointer.

pub fn with_opencl_throttling(self) -> Self

This option enables OpenCL queue throttling for GPU devices (reduces CPU utilization when using GPU).

pub fn with_dynamic_shapes(self) -> Self

This option if enabled works for dynamic shaped models whose shape will be set dynamically based on the infer input image/data shape at run time in CPU. This gives best result for running multiple inferences with varied shaped images/data.

pub fn with_npu_fast_compile(self) -> Self

pub fn build(self) -> ExecutionProviderDispatch

Trait Implementations

impl Clone for OpenVINOExecutionProvider

fn clone(&self) -> OpenVINOExecutionProvider

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for OpenVINOExecutionProvider

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for OpenVINOExecutionProvider

fn default() -> Self

Returns the “default value” for a type.

impl ExecutionProvider for OpenVINOExecutionProvider

fn as_str(&self) -> &'static str

Returns the identifier of this execution provider used internally by ONNX Runtime.

fn supported_by_platform(&self) -> bool

Returns whether this execution provider is supported on this platform.

fn register(&self, session_builder: &mut SessionBuilder) -> Result<()>

Attempts to register this execution provider on the given session.

fn is_available(&self) -> Result<bool>

Returns Ok(true) if ONNX Runtime was compiled with support for this execution provider, and Ok(false) otherwise.

impl From<OpenVINOExecutionProvider> for ExecutionProviderDispatch

fn from(value: OpenVINOExecutionProvider) -> Self

Converts to this type from the input type.

impl Send for OpenVINOExecutionProvider

impl Sync for OpenVINOExecutionProvider