lance_file::format

Module pbfile

Source
Expand description

Protobuf definitions for Lance Format v2

Modules§

Structs§

  • Each column has a metadata block that is placed at the end of the file. These may be read individually to allow for column projection.
  • The deferred encoding is used to place the encoding itself in a different part of the file. This is most commonly used to allow encodings to be shared across different columns. For example, when writing a file with thousands of columns, where many pages have the exact same encoding, it can be useful to cut down on the size of the metadata by using a deferred encoding.
  • The encoding is placed directly in the metadata section
  • An encoding stores the information needed to decode a column or page