Struct IvfBuildParams

pub struct IvfBuildParams {
    pub num_partitions: usize,
    pub max_iters: usize,
    pub centroids: Option<Arc<FixedSizeListArray>>,
    pub sample_rate: usize,
    pub precomputed_partitions_file: Option<String>,
    pub precomputed_shuffle_buffers: Option<(Path, Vec<String>)>,
    pub shuffle_partition_batches: usize,
    pub shuffle_partition_concurrency: usize,
    pub use_residual: bool,
    pub storage_options: Option<HashMap<String, String>>,
}

Parameters to build IVF partitions

Fields

num_partitions: usize

Number of partitions to build.

max_iters: usize

Max number of iterations to train kmeans.

centroids: Option<Arc<FixedSizeListArray>>

Use provided IVF centroids.

sample_rate: usize

precomputed_partitions_file: Option<String>

Precomputed partitions file (row_id -> partition_id) mutually exclusive with precomputed_shuffle_buffers

precomputed_shuffle_buffers: Option<(Path, Vec<String>)>

Precomputed shuffle buffers (row_id -> partition_id, pq_code) mutually exclusive with precomputed_partitions_file requires centroids to be set

The input is expected to be (/dir/to/buffers, [buffer1.lance, buffer2.lance, …])

shuffle_partition_batches: usize

shuffle_partition_concurrency: usize

use_residual: bool

Use residual vectors to build sub-vector.

storage_options: Option<HashMap<String, String>>

Storage options used to load precomputed partitions.

Implementations

impl IvfBuildParams

pub fn new(num_partitions: usize) -> Self

Create a new instance of IvfBuildParams.

pub fn try_with_centroids( num_partitions: usize, centroids: Arc<FixedSizeListArray>, ) -> Result<Self>

Create a new instance of IvfBuildParams with centroids.

Trait Implementations

impl Clone for IvfBuildParams

fn clone(&self) -> IvfBuildParams

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for IvfBuildParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for IvfBuildParams

fn default() -> Self

Returns the “default value” for a type.