Struct ByteStream

pub struct ByteStream { /* private fields */ }

A potentially infinite, interruptible stream of bytes.

To create a ByteStream, you can use any of the following methods:

• read: takes any type that implements Read.
• file: takes a File.
• from_iter: takes an Iterator whose items implement AsRef<[u8]>.
• from_result_iter: same as from_iter, but each item is a Result<T, ShellError>.
• from_fn: uses a generator function to fill a buffer whenever it is empty. This has high performance because it doesn’t need to allocate for each chunk of data, and can just reuse the same buffer.

Byte streams have a type which is used to preserve type compatibility when they are the result of an internal command. It is important that this be set to the correct value. Unknown is used only for external sources where the type can not be inherently determined, and having it automatically act as a string or binary depending on whether it parses as UTF-8 or not is desirable.

The data of a ByteStream can be accessed using one of the following methods:

• reader: returns a Read-able type to get the raw bytes in the stream.
• lines: splits the bytes on lines and returns an Iterator where each item is a Result<String, ShellError>.
• chunks: returns an Iterator of Values where each value is either a string or binary. Try not to use this method if possible. Rather, please use reader (or lines if it matches the situation).

Additionally, there are few methods to collect a ByteStream into memory:

• into_bytes: collects all bytes into a Vec<u8>.
• into_string: collects all bytes into a String, erroring if utf-8 decoding failed.
• into_value: collects all bytes into a value typed appropriately for the type of this stream. If the type is Unknown, it will produce a string value if the data is valid UTF-8, or a binary value otherwise.

There are also a few other methods to consume all the data of a ByteStream:

• drain: consumes all bytes and outputs nothing.
• write_to: writes all bytes to the given Write destination.
• print: a convenience wrapper around write_to. It prints all bytes to stdout or stderr.

Internally, ByteStreams currently come in three flavors according to ByteStreamSource. See its documentation for more information.

Implementations

impl ByteStream

pub fn new( stream: ByteStreamSource, span: Span, signals: Signals, type_: ByteStreamType, ) -> ByteStream

Create a new ByteStream from a ByteStreamSource.

pub fn read( reader: impl Read + Send + 'static, span: Span, signals: Signals, type_: ByteStreamType, ) -> ByteStream

Create a ByteStream from an arbitrary reader. The type must be provided.

pub fn read_string(string: String, span: Span, signals: Signals) -> ByteStream

Create a ByteStream from a string. The type of the stream is always String.

pub fn read_binary(bytes: Vec<u8>, span: Span, signals: Signals) -> ByteStream

Create a ByteStream from a byte vector. The type of the stream is always Binary.

pub fn file(file: File, span: Span, signals: Signals) -> ByteStream

Create a ByteStream from a file.

The type is implicitly Unknown, as it’s not typically known whether files will return text or binary.

pub fn child(child: ChildProcess, span: Span) -> ByteStream

Create a ByteStream from a child process’s stdout and stderr.

The type is implicitly Unknown, as it’s not typically known whether child processes will return text or binary.

pub fn stdin(span: Span) -> Result<ByteStream, ShellError>

Create a ByteStream that reads from stdin.

The type is implicitly Unknown, as it’s not typically known whether stdin is text or binary.

pub fn from_fn( span: Span, signals: Signals, type_: ByteStreamType, generator: impl FnMut(&mut Vec<u8>) -> Result<bool, ShellError> + Send + 'static, ) -> ByteStream

Create a ByteStream from a generator function that writes data to the given buffer when called, and returns Ok(false) on end of stream.

pub fn with_type(self, type_: ByteStreamType) -> ByteStream

pub fn from_iter<I>( iter: I, span: Span, signals: Signals, type_: ByteStreamType, ) -> ByteStream

where I: IntoIterator, <I as IntoIterator>::IntoIter: Send + 'static, <I as IntoIterator>::Item: AsRef<[u8]> + Default + Send + 'static,

Create a new ByteStream from an Iterator of bytes slices.

The returned ByteStream will have a ByteStreamSource of Read.

pub fn from_result_iter<I, T>( iter: I, span: Span, signals: Signals, type_: ByteStreamType, ) -> ByteStream

where I: IntoIterator<Item = Result<T, ShellError>>, <I as IntoIterator>::IntoIter: Send + 'static, T: AsRef<[u8]> + Default + Send + 'static,

Create a new ByteStream from an Iterator of Result bytes slices.

The returned ByteStream will have a ByteStreamSource of Read.

pub fn with_known_size(self, size: Option<u64>) -> ByteStream

Set the known size, in number of bytes, of the ByteStream.

pub fn source(&self) -> &ByteStreamSource

Get a reference to the inner ByteStreamSource of the ByteStream.

pub fn source_mut(&mut self) -> &mut ByteStreamSource

Get a mutable reference to the inner ByteStreamSource of the ByteStream.

pub fn span(&self) -> Span

Returns the Span associated with the ByteStream.

pub fn with_span(self, span: Span) -> ByteStream

Changes the Span associated with the ByteStream.

pub fn type_(&self) -> ByteStreamType

Returns the ByteStreamType associated with the ByteStream.

pub fn known_size(&self) -> Option<u64>

Returns the known size, in number of bytes, of the ByteStream.

pub fn reader(self) -> Option<Reader>

Convert the ByteStream into its Reader which allows one to Read the raw bytes of the stream.

Reader is buffered and also implements BufRead.

If the source of the ByteStream is ByteStreamSource::Child and the child has no stdout, then the stream is considered empty and None will be returned.

pub fn lines(self) -> Option<Lines>

Convert the ByteStream into a Lines iterator where each element is a Result<String, ShellError>.

There is no limit on how large each line will be. Ending new lines (\n or \r\n) are stripped from each line. If a line fails to be decoded as utf-8, then it will become a ShellError.

If the source of the ByteStream is ByteStreamSource::Child and the child has no stdout, then the stream is considered empty and None will be returned.

pub fn chunks(self) -> Option<Chunks>

Convert the ByteStream into a Chunks iterator where each element is a Result<Value, ShellError>.

Each call to next reads the currently available data from the byte stream source, up to a maximum size. The values are typed according to the type of the stream, and if that type is Unknown, string values will be produced as long as the stream continues to parse as valid UTF-8, but binary values will be produced instead of the stream fails to parse as UTF-8 instead at any point. Any and all newlines are kept intact in each chunk.

Where possible, prefer reader or lines over this method. Those methods are more likely to be used in a semantically correct way (and reader is more efficient too).

If the source of the ByteStream is ByteStreamSource::Child and the child has no stdout, then the stream is considered empty and None will be returned.

pub fn into_source(self) -> ByteStreamSource

Convert the ByteStream into its inner ByteStreamSource.

pub fn into_stdio(self) -> Result<Stdio, ByteStream>

Attempt to convert the ByteStream into a Stdio.

This will succeed if the ByteStreamSource of the ByteStream is either:

• File
• Child and the child has a stdout that is Some(ChildPipe::Pipe(..)).

All other cases return an Err with the original ByteStream in it.

pub fn into_child(self) -> Result<ChildProcess, ByteStream>

Attempt to convert the ByteStream into a ChildProcess.

This will only succeed if the ByteStreamSource of the ByteStream is Child. All other cases return an Err with the original ByteStream in it.

pub fn into_bytes(self) -> Result<Vec<u8>, ShellError>

Collect all the bytes of the ByteStream into a Vec<u8>.

Any trailing new lines are kept in the returned Vec.

pub fn into_string(self) -> Result<String, ShellError>

Collect the stream into a String in-memory. This can only succeed if the data contained is valid UTF-8.

The trailing new line (\n or \r\n), if any, is removed from the String prior to being returned, if this is a stream coming from an external process or file.

If the type is specified as Binary, this operation always fails, even if the data would have been valid UTF-8.

pub fn into_value(self) -> Result<Value, ShellError>

Collect all the bytes of the ByteStream into a Value.

If this is a String stream, the stream is decoded to UTF-8. If the stream came from an external process or file, the trailing new line (\n or \r\n), if any, is removed from the String prior to being returned.

If this is a Binary stream, a Value::Binary is returned with any trailing new lines preserved.

If this is an Unknown stream, the behavior depends on whether the stream parses as valid UTF-8 or not. If it does, this is uses the String behavior; if not, it uses the Binary behavior.

pub fn drain(self) -> Result<(), ShellError>

Consume and drop all bytes of the ByteStream.

pub fn print(self, to_stderr: bool) -> Result<(), ShellError>

Print all bytes of the ByteStream to stdout or stderr.

pub fn write_to(self, dest: impl Write) -> Result<(), ShellError>

Write all bytes of the ByteStream to dest.

Trait Implementations

impl Debug for ByteStream

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl From<ByteStream> for PipelineData

fn from(stream: ByteStream) -> PipelineData

Converts to this type from the input type.