Enum PipelineData

pub enum PipelineData {
    Empty,
    Value(Value, Option<PipelineMetadata>),
    ListStream(ListStream, Option<PipelineMetadata>),
    ByteStream(ByteStream, Option<PipelineMetadata>),
}

The foundational abstraction for input and output to commands

This represents either a single Value or a stream of values coming into the command or leaving a command.

A note on implementation:

We’ve tried a few variations of this structure. Listing these below so we have a record.

• We tried always assuming a stream in Nushell. This was a great 80% solution, but it had some rough edges. Namely, how do you know the difference between a single string and a list of one string. How do you know when to flatten the data given to you from a data source into the stream or to keep it as an unflattened list?

• We tried putting the stream into Value. This had some interesting properties as now commands “just worked on values”, but lead to a few unfortunate issues.

The first is that you can’t easily clone Values in a way that felt largely immutable. For example, if you cloned a Value which contained a stream, and in one variable drained some part of it, then the second variable would see different values based on what you did to the first.

To make this kind of mutation thread-safe, we would have had to produce a lock for the stream, which in practice would have meant always locking the stream before reading from it. But more fundamentally, it felt wrong in practice that observation of a value at runtime could affect other values which happen to alias the same stream. By separating these, we don’t have this effect. Instead, variables could get concrete list values rather than streams, and be able to view them without non-local effects.

• A balance of the two approaches is what we’ve landed on: Values are thread-safe to pass, and we can stream them into any sources. Streams are still available to model the infinite streams approach of original Nushell.

Variants

Empty

Value(Value, Option<PipelineMetadata>)

ListStream(ListStream, Option<PipelineMetadata>)

ByteStream(ByteStream, Option<PipelineMetadata>)

Implementations

impl PipelineData

pub fn empty() -> PipelineData

pub fn metadata(&self) -> Option<PipelineMetadata>

pub fn set_metadata(self, metadata: Option<PipelineMetadata>) -> PipelineData

pub fn is_nothing(&self) -> bool

pub fn span(&self) -> Option<Span>

PipelineData doesn’t always have a Span, but we can try!

pub fn with_span(self, span: Span) -> PipelineData

Change the span of the PipelineData.

Returns Value(Nothing) with the given span if it was PipelineData::Empty.

pub fn get_type(&self) -> Type

Get a type that is representative of the PipelineData.

The type returned here makes no effort to collect a stream, so it may be a different type than would be returned by Value::get_type() on the result of .into_value().

Specifically, a ListStream results in list stream rather than the fully complete list type (which would require knowing the contents), and a ByteStream with unknown type results in any rather than string or binary.

pub fn into_value(self, span: Span) -> Result<Value, ShellError>

pub fn try_into_stream( self, engine_state: &EngineState, ) -> Result<PipelineData, PipelineData>

Converts any Value variant that can be represented as a stream into its stream variant.

This means that lists and ranges are converted into list streams, and strings and binary are converted into byte streams.

Returns an Err with the original stream if the variant couldn’t be converted to a stream variant. If the variant is already a stream variant, it is returned as-is.

pub fn write_to(self, dest: impl Write) -> Result<(), ShellError>

Drain and write this PipelineData to dest.

Values are converted to bytes and separated by newlines if this is a ListStream.

pub fn drain_to_out_dests( self, engine_state: &EngineState, stack: &mut Stack, ) -> Result<PipelineData, ShellError>

Drain this PipelineData according to the current stdout OutDests in stack.

For OutDest::Pipe and OutDest::PipeSeparate, this will return the PipelineData as is. For OutDest::Value, this will collect into a value and return it. For OutDest::Print, the PipelineData is drained and printed. Otherwise, the PipelineData is drained, but only printed if it is the output of an external command.

pub fn drain(self) -> Result<(), ShellError>

pub fn into_iter_strict( self, span: Span, ) -> Result<PipelineIterator, ShellError>

Try convert from self into iterator

It returns Err if the self cannot be converted to an iterator.

The span should be the span of the command or operation that would raise an error.

pub fn collect_string( self, separator: &str, config: &Config, ) -> Result<String, ShellError>

pub fn collect_string_strict( self, span: Span, ) -> Result<(String, Span, Option<PipelineMetadata>), ShellError>

Retrieves string from pipeline data.

As opposed to collect_string this raises error rather than converting non-string values. The span will be used if ListStream is encountered since it doesn’t carry a span.

pub fn follow_cell_path( self, cell_path: &[PathMember], head: Span, insensitive: bool, ) -> Result<Value, ShellError>

pub fn map<F>(self, f: F, signals: &Signals) -> Result<PipelineData, ShellError>

where PipelineData: Sized, F: FnMut(Value) -> Value + 'static + Send,

Simplified mapper to help with simple values also. For full iterator support use .into_iter() instead

pub fn flat_map<U, F>( self, f: F, signals: &Signals, ) -> Result<PipelineData, ShellError>

where PipelineData: Sized, U: IntoIterator<Item = Value> + 'static, <U as IntoIterator>::IntoIter: 'static + Send, F: FnMut(Value) -> U + 'static + Send,

Simplified flatmapper. For full iterator support use .into_iter() instead

pub fn filter<F>( self, f: F, signals: &Signals, ) -> Result<PipelineData, ShellError>

where PipelineData: Sized, F: FnMut(&Value) -> bool + 'static + Send,

pub fn try_expand_range(self) -> Result<PipelineData, ShellError>

Try to convert Value from Value::Range to Value::List. This is useful to expand Value::Range into array notation, specifically when converting to json or to nuon. 1..3 | to XX -> [1,2,3]

pub fn print( self, engine_state: &EngineState, stack: &mut Stack, no_newline: bool, to_stderr: bool, ) -> Result<(), ShellError>

Consume and print self data immediately.

no_newline controls if we need to attach newline character to output. to_stderr controls if data is output to stderr, when the value is false, the data is output to stdout.

pub fn print_raw( self, engine_state: &EngineState, no_newline: bool, to_stderr: bool, ) -> Result<(), ShellError>

Consume and print self data without any extra formatting.

This does not use the table command to format data, and also prints binary values and streams in their raw format without generating a hexdump first.

no_newline controls if we need to attach newline character to output. to_stderr controls if data is output to stderr, when the value is false, the data is output to stdout.

pub fn unsupported_input_error( self, expected_type: impl Into<String>, span: Span, ) -> ShellError

Trait Implementations

impl Debug for PipelineData

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl From<ByteStream> for PipelineData

fn from(stream: ByteStream) -> PipelineData

Converts to this type from the input type.

impl From<ListStream> for PipelineData

fn from(stream: ListStream) -> PipelineData

Converts to this type from the input type.

impl IntoIterator for PipelineData

type Item = Value

The type of the elements being iterated over.

type IntoIter = PipelineIterator

Which kind of iterator are we turning this into?

fn into_iter(self) -> <PipelineData as IntoIterator>::IntoIter

Creates an iterator from a value.