Enum DataFusionError

pub enum DataFusionError {
    ArrowError(ArrowError, Option<String>),
    ParquetError(ParquetError),
    AvroError(Error),
    ObjectStore(Error),
    IoError(Error),
    SQL(ParserError, Option<String>),
    NotImplemented(String),
    Internal(String),
    Plan(String),
    Configuration(String),
    SchemaError(SchemaError, Box<Option<String>>),
    Execution(String),
    ExecutionJoin(JoinError),
    ResourcesExhausted(String),
    External(GenericError),
    Context(String, Box<DataFusionError>),
    Substrait(String),
    Diagnostic(Box<Diagnostic>, Box<DataFusionError>),
    Collection(Vec<DataFusionError>),
    Shared(Arc<DataFusionError>),
}

DataFusion error

Variants

ArrowError(ArrowError, Option<String>)

Error returned by arrow.

2nd argument is for optional backtrace

ParquetError(ParquetError)

Available on crate feature parquet only.

Error when reading / writing Parquet data.

AvroError(Error)

Available on crate feature avro only.

Error when reading Avro data.

ObjectStore(Error)

Available on crate feature object_store only.

Error when reading / writing to / from an object_store (e.g. S3 or LocalFile)

IoError(Error)

Error when an I/O operation fails

SQL(ParserError, Option<String>)

Error when SQL is syntactically incorrect.

2nd argument is for optional backtrace

NotImplemented(String)

Error when a feature is not yet implemented.

These errors are sometimes returned for features that are still in development and are not entirely complete. Often, these errors are tracked in our issue tracker.

Internal(String)

Error due to bugs in DataFusion

This error should not happen in normal usage of DataFusion. It results from something that wasn’t expected/anticipated by the implementation and that is most likely a bug (the error message even encourages users to open a bug report). A user should not be able to trigger internal errors under normal circumstances by feeding in malformed queries, bad data, etc.

Note that I/O errors (or any error that happens due to external systems) do NOT fall under this category. See other variants such as Self::IoError and Self::External.

DataFusions has internal invariants that the compiler is not always able to check. This error is raised when one of those invariants does not hold for some reason.

Plan(String)

Error during planning of the query.

This error happens when the user provides a bad query or plan, for example the user attempts to call a function that doesn’t exist, or if the types of a function call are not supported.

Configuration(String)

Error for invalid or unsupported configuration options.

SchemaError(SchemaError, Box<Option<String>>)

Error when there is a problem with the query related to schema.

This error can be returned in cases such as when schema inference is not possible and when column names are not unique.

2nd argument is for optional backtrace Boxing the optional backtrace to prevent https://rust-lang.github.io/rust-clippy/master/index.html#/result_large_err

Execution(String)

Error during execution of the query.

This error is returned when an error happens during execution due to a malformed input. For example, the user passed malformed arguments to a SQL method, opened a CSV file that is broken, or tried to divide an integer by zero.

ExecutionJoin(JoinError)

JoinError during execution of the query.

This error can’t occur for unjoined tasks, such as execution shutdown.

ResourcesExhausted(String)

Error when resources (such as memory of scratch disk space) are exhausted.

This error is thrown when a consumer cannot acquire additional memory or other resources needed to execute the query from the Memory Manager.

External(GenericError)

Errors originating from outside DataFusion’s core codebase.

For example, a custom S3Error from the crate datafusion-objectstore-s3

Context(String, Box<DataFusionError>)

Error with additional context

Substrait(String)

Errors from either mapping LogicalPlans to/from Substrait plans or serializing/deserializing protobytes to Substrait plans

Diagnostic(Box<Diagnostic>, Box<DataFusionError>)

Error wrapped together with additional contextual information intended for end users, to help them understand what went wrong by providing human-readable messages, and locations in the source query that relate to the error in some way.

Collection(Vec<DataFusionError>)

A collection of one or more DataFusionError. Useful in cases where DataFusion can recover from an erroneous state, and produce more errors before terminating. e.g. when planning a SELECT clause, DataFusion can synchronize to the next SelectItem if the previous one had errors. The end result is that the user can see errors about all SelectItem, instead of just the first one.

Shared(Arc<DataFusionError>)

A DataFusionError which shares an underlying DataFusionError.

This is useful when the same underlying DataFusionError is passed to multiple receivers. For example, when the source of a repartition errors and the error is propagated to multiple consumers.

Implementations

impl DataFusionError

pub const BACK_TRACE_SEP: &'static str = "\n\nbacktrace: "

The separator between the error message and the backtrace

pub fn find_root(&self) -> &Self

Get deepest underlying DataFusionError

DataFusionErrors sometimes form a chain, such as DataFusionError::ArrowError() in order to conform to the correct error signature. Thus sometimes there is a chain several layers deep that can obscure the original error. This function finds the lowest level DataFusionError possible.

For example, find_root will returnDataFusionError::ResourceExhausted given the input

DataFusionError::ArrowError
  ArrowError::External
   Box(DataFusionError::Context)
     DataFusionError::ResourceExhausted

This may be the same as self.

pub fn context(self, description: impl Into<String>) -> Self

wraps self in Self::Context with a description

pub fn strip_backtrace(&self) -> String

Strips backtrace out of the error message If backtrace enabled then error has a format “message” Self::BACK_TRACE_SEP “backtrace” The method strips the backtrace and outputs “message”

pub fn get_back_trace() -> String

To enable optional rust backtrace in DataFusion:

• [Setup Env Variables]https://doc.rust-lang.org/std/backtrace/index.html#environment-variables
• Enable backtrace cargo feature

Example: cargo build –features ‘backtrace’ RUST_BACKTRACE=1 ./app

pub fn builder() -> DataFusionErrorBuilder

Return a DataFusionErrorBuilder to build a DataFusionError

pub fn message(&self) -> Cow<'_, str>

pub fn with_diagnostic(self, diagnostic: Diagnostic) -> Self

Wraps the error with contextual information intended for end users

pub fn with_diagnostic_fn<F: FnOnce(&DataFusionError) -> Diagnostic>( self, f: F, ) -> Self

Wraps the error with contextual information intended for end users. Takes a function that inspects the error and returns the diagnostic to wrap it with.

pub fn diagnostic(&self) -> Option<&Diagnostic>

Gets the Diagnostic associated with the error, if any. If there is more than one, only the outermost Diagnostic is returned.

pub fn iter(&self) -> impl Iterator<Item = &DataFusionError>

Return an iterator over this DataFusionError and any other DataFusionErrors in a DataFusionError::Collection.

Sometimes DataFusion is able to collect multiple errors in a SQL query before terminating, e.g. across different expressions in a SELECT statements or different sides of a UNION. This method returns an iterator over all the errors in the collection.

For this to work, the top-level error must be a DataFusionError::Collection, not something that contains it.

Trait Implementations

impl Debug for DataFusionError

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for DataFusionError

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Error for DataFusionError

fn source(&self) -> Option<&(dyn Error + 'static)>

Returns the lower-level source of this error, if any.

fn description(&self) -> &str

👎Deprecated since 1.42.0: use the Display impl or to_string()

fn cause(&self) -> Option<&dyn Error>

👎Deprecated since 1.33.0: replaced by Error::source, which can support downcasting

fn provide<'a>(&'a self, request: &mut Request<'a>)

🔬This is a nightly-only experimental API. (error_generic_member_access)

Provides type-based access to context intended for error reports.

impl From<&Arc<DataFusionError>> for DataFusionError

fn from(e: &Arc<DataFusionError>) -> Self

Converts to this type from the input type.

impl From<ArrowError> for DataFusionError

fn from(e: ArrowError) -> Self

Converts to this type from the input type.

impl From<Box<dyn Error + Sync + Send>> for DataFusionError

fn from(err: GenericError) -> Self

Converts to this type from the input type.

impl From<DataFusionError> for ArrowError

fn from(e: DataFusionError) -> Self

Converts to this type from the input type.

impl From<DataFusionError> for Error

fn from(e: DataFusionError) -> Self

Converts to this type from the input type.

impl From<DataFusionError> for PyErr

Available on crate feature pyarrow only.

fn from(err: DataFusionError) -> PyErr

Converts to this type from the input type.

impl From<Error> for DataFusionError

fn from(_e: Error) -> Self

Converts to this type from the input type.

impl From<Error> for DataFusionError

fn from(e: Error) -> Self

Converts to this type from the input type.

impl From<Error> for DataFusionError

Available on crate feature avro only.

fn from(e: AvroError) -> Self

Converts to this type from the input type.

impl From<Error> for DataFusionError

Available on crate feature object_store only.

fn from(e: Error) -> Self

Converts to this type from the input type.

impl From<Error> for DataFusionError

Available on crate feature object_store only.

fn from(e: Error) -> Self

Converts to this type from the input type.

impl From<ParquetError> for DataFusionError

Available on crate feature parquet only.

fn from(e: ParquetError) -> Self

Converts to this type from the input type.

impl From<ParserError> for DataFusionError

fn from(e: ParserError) -> Self

Converts to this type from the input type.