Struct ColumnarBuffer

pub struct ColumnarBuffer<C> { /* private fields */ }

A columnar buffer intended to be bound with crate::Cursor::bind_buffer in order to obtain results from a cursor.

Binds to the result set column wise. This is usually helpful in dataengineering or data sciense tasks. This buffer type can be used in situations there the schema of the queried data is known at compile time, as well as for generic applications which do work with wide range of different data.

Example: Fetching results column wise with ColumnarBuffer.

Consider querying a table with two columns year and name.

use odbc_api::{
    Environment, Cursor, ConnectionOptions,
    buffers::{AnySlice, BufferDesc, Item, ColumnarAnyBuffer},
};

let env = Environment::new()?;

let batch_size = 1000; // Maximum number of rows in each row set
let buffer_description = [
    // We know year to be a Nullable SMALLINT
    BufferDesc::I16 { nullable: true },
    // and name to be a required VARCHAR
    BufferDesc::Text { max_str_len: 255 },
];

/// Creates a columnar buffer fitting the buffer description with the capacity of `batch_size`.
let mut buffer = ColumnarAnyBuffer::from_descs(batch_size, buffer_description);

let mut conn = env.connect(
    "YourDatabase", "SA", "My@Test@Password1",
    ConnectionOptions::default(),
)?;
if let Some(cursor) = conn.execute("SELECT year, name FROM Birthdays;", ())? {
    // Bind buffer to cursor. We bind the buffer as a mutable reference here, which makes it
    // easier to reuse for other queries, but we could have taken ownership.
    let mut row_set_cursor = cursor.bind_buffer(&mut buffer)?;
    // Loop over row sets
    while let Some(row_set) = row_set_cursor.fetch()? {
        // Process years in row set
        let year_col = row_set.column(0);
        for year in i16::as_nullable_slice(year_col)
            .expect("Year column buffer expected to be nullable Int")
        {
            // Iterate over `Option<i16>` with it ..
        }
        // Process names in row set
        let name_col = row_set.column(1);
        for name in name_col
            .as_text_view()
            .expect("Name column buffer expected to be text")
            .iter()
        {
            // Iterate over `Option<&CStr> ..
        }
    }
}

This second examples changes two things, we do not know the schema in advance and use the SQL DataType to determine the best fit for the buffers. Also we want to do everything in a function and return a Cursor with an already bound buffer. This approach is best if you have few and very long query, so the overhead of allocating buffers is negligible and you want to have an easier time with the borrow checker.

use odbc_api::{
    Connection, BlockCursor, Error, Cursor, Nullability, ResultSetMetadata,
    buffers::{ AnyBuffer, BufferDesc, ColumnarAnyBuffer, ColumnarBuffer }
};

fn get_birthdays<'a>(conn: &'a mut Connection)
    -> Result<BlockCursor<impl Cursor + 'a, ColumnarAnyBuffer>, Error>
{
    let mut cursor = conn.execute("SELECT year, name FROM Birthdays;", ())?.unwrap();
    let mut column_description = Default::default();
    let buffer_description : Vec<_> = (0..cursor.num_result_cols()?).map(|index| {
        cursor.describe_col(index as u16 + 1, &mut column_description)?;
        let nullable = matches!(
            column_description.nullability,
            Nullability::Unknown | Nullability::Nullable
        );
        let desc = BufferDesc::from_data_type(
            column_description.data_type,
            nullable
        ).unwrap_or(BufferDesc::Text{ max_str_len: 255 });
        Ok(desc)
    }).collect::<Result<_, Error>>()?;

    // Row set size of 5000 rows.
    let buffer = ColumnarAnyBuffer::from_descs(5000, buffer_description);
    // Bind buffer and take ownership over it.
    cursor.bind_buffer(buffer)
}

Implementations

impl ColumnarBuffer<AnyBuffer>

pub fn from_descs( capacity: usize, descs: impl IntoIterator<Item = BufferDesc>, ) -> Self

Allocates a ColumnarBuffer fitting the buffer descriptions.

pub fn try_from_descs( capacity: usize, descs: impl IntoIterator<Item = BufferDesc>, ) -> Result<Self, Error>

Allocates a ColumnarBuffer fitting the buffer descriptions. If not enough memory is available to allocate the buffers this function fails with Error::TooLargeColumnBufferSize. This function is slower than Self::from_descs which would just panic if not enough memory is available for allocation.

pub fn from_descs_and_indices( max_rows: usize, description: impl Iterator<Item = (u16, BufferDesc)>, ) -> ColumnarBuffer<AnyBuffer>

Allows you to pass the buffer descriptions together with a one based column index referring the column, the buffer is supposed to bind to. This allows you also to ignore columns in a result set, by not binding them at all. There is no restriction on the order of column indices passed, but the function will panic, if the indices are not unique.

impl<C: ColumnBuffer> ColumnarBuffer<C>

pub fn new(columns: Vec<(u16, C)>) -> Self

Create a new instance from columns with unique indicies. Capacity of the buffer will be the minimum capacity of the columns. The constructed buffer is always empty (i.e. the number of valid rows is considered to be zero).

You do not want to call this constructor directly unless you want to provide your own buffer implentation. Most users of this crate may want to use the constructors like crate::buffers::ColumnarAnyBuffer::from_descs or crate::buffers::TextRowSet::from_max_str_lens instead.

pub unsafe fn new_unchecked(capacity: usize, columns: Vec<(u16, C)>) -> Self

Safety

• Indices must be unique
• Columns all must have enough capacity.

pub fn num_rows(&self) -> usize

Number of valid rows in the buffer.

pub fn num_cols(&self) -> usize

Return the number of columns in the row set.

pub fn column(&self, buffer_index: usize) -> C::View<'_>

Use this method to gain read access to the actual column data.

Parameters

• buffer_index: Please note that the buffer index is not identical to the ODBC column index. For one it is zero based. It also indexes the buffer bound, and not the columns of the output result set. This is important, because not every column needs to be bound. Some columns may simply be ignored. That being said, if every column of the output is bound in the buffer, in the same order in which they are enumerated in the result set, the relationship between column index and buffer index is buffer_index = column_index - 1.

impl ColumnarBuffer<TextColumn<u8>>

pub fn for_cursor( batch_size: usize, cursor: &mut impl ResultSetMetadata, max_str_limit: Option<usize>, ) -> Result<TextRowSet, Error>

The resulting text buffer is not in any way tied to the cursor, other than that its buffer sizes a tailor fitted to result set the cursor is iterating over.

This method performs fallible buffer allocations, if no upper bound is set, so you may see a speedup, by setting an upper bound using max_str_limit.

Parameters

• batch_size: The maximum number of rows the buffer is able to hold.
• cursor: Used to query the display size for each column of the row set. For character data the length in characters is multiplied by 4 in order to have enough space for 4 byte utf-8 characters. This is a pessimization for some data sources (e.g. SQLite 3) which do interpret the size of a VARCHAR(5) column as 5 bytes rather than 5 characters.
• max_str_limit: Some queries make it hard to estimate a sensible upper bound and sometimes drivers are just not that good at it. This argument allows you to specify an upper bound for the length of character data. Any size reported by the driver is capped to this value. In case the upper bound can not inferred by the metadata reported by the driver the element size is set to this upper bound, too.

pub fn from_max_str_lens( row_capacity: usize, max_str_lengths: impl IntoIterator<Item = usize>, ) -> Result<Self, Error>

Creates a text buffer large enough to hold batch_size rows with one column for each item max_str_lengths of respective size.

pub fn at(&self, buffer_index: usize, row_index: usize) -> Option<&[u8]>

Access the element at the specified position in the row set.

pub fn at_as_str( &self, col_index: usize, row_index: usize, ) -> Result<Option<&str>, Utf8Error>

Access the element at the specified position in the row set.

pub fn indicator_at(&self, buf_index: usize, row_index: usize) -> Indicator

Indicator value at the specified position. Useful to detect truncation of data.

Example

use odbc_api::buffers::{Indicator, TextRowSet};

fn is_truncated(buffer: &TextRowSet, col_index: usize, row_index: usize) -> bool {
    match buffer.indicator_at(col_index, row_index) {
        // There is no value, therefore there is no value not fitting in the column buffer.
        Indicator::Null => false,
        // The value did not fit into the column buffer, we do not even know, by how much.
        Indicator::NoTotal => true,
        Indicator::Length(total_length) => {
            // If the maximum string length is shorter than the values total length, the
            // has been truncated to fit into the buffer.
            buffer.max_len(col_index) < total_length
        }
    }
}

pub fn max_len(&self, buf_index: usize) -> usize

Maximum length in bytes of elements in a column.

Trait Implementations

impl<C> RowSetBuffer for ColumnarBuffer<C>

where C: ColumnBuffer,

fn bind_type(&self) -> usize

Declares the bind type of the Row set buffer. 0 Means a columnar binding is used. Any non zero number is interpreted as the size of a single row in a row wise binding style.

fn row_array_size(&self) -> usize

The batch size for bulk cursors, if retrieving many rows at once.

fn mut_num_fetch_rows(&mut self) -> &mut usize

Mutable reference to the number of fetched rows.

unsafe fn bind_colmuns_to_cursor( &mut self, cursor: StatementRef<'_>, ) -> Result<(), Error>

Binds the buffer either column or row wise to the cursor.

fn find_truncation(&self) -> Option<TruncationInfo>

Find an indicator larger than the maximum element size of the buffer.