pub struct ByteStream { /* private fields */ }
Expand description
Stream of binary data
ByteStream
wraps a stream of binary data for ease of use.
§Getting data out of a ByteStream
ByteStream
provides two primary mechanisms for accessing the data:
-
With
.collect()
:.collect()
reads the complete ByteStream into memory and stores it inAggregatedBytes
, a non-contiguous ByteBuffer.use aws_smithy_types::byte_stream::{ByteStream, AggregatedBytes}; use aws_smithy_types::body::SdkBody; use bytes::Buf; async fn example() { let stream = ByteStream::new(SdkBody::from("hello! This is some data")); // Load data from the stream into memory: let data = stream.collect().await.expect("error reading data"); // collect returns a `bytes::Buf`: println!("first chunk: {:?}", data.chunk()); }
-
Via
.next()
or.try_next()
:For use-cases where holding the entire ByteStream in memory is unnecessary, use the
Stream
implementation:use aws_smithy_types::byte_stream::{ByteStream, AggregatedBytes, error::Error}; use aws_smithy_types::body::SdkBody; async fn example() -> Result<(), Error> { let mut stream = ByteStream::from(vec![1, 2, 3, 4, 5, 99]); let mut digest = crc32::Digest::new(); while let Some(bytes) = stream.try_next().await? { digest.write(&bytes); } println!("digest: {}", digest.finish()); Ok(()) }
-
Via
.into_async_read()
:Note: The
rt-tokio
feature must be active to use.into_async_read()
.It’s possible to convert a
ByteStream
into a struct that implementstokio::io::AsyncBufRead
.use aws_smithy_types::byte_stream::ByteStream; use aws_smithy_types::body::SdkBody; use tokio::io::AsyncBufReadExt; #[cfg(feature = "rt-tokio")] async fn example() -> std::io::Result<()> { let stream = ByteStream::new(SdkBody::from("hello!\nThis is some data")); // Convert the stream to a BufReader let buf_reader = stream.into_async_read(); let mut lines = buf_reader.lines(); assert_eq!(lines.next_line().await?, Some("hello!".to_owned())); assert_eq!(lines.next_line().await?, Some("This is some data".to_owned())); assert_eq!(lines.next_line().await?, None); Ok(()) }
§Getting data into a ByteStream
ByteStreams can be created in one of three ways:
-
From in-memory binary data: ByteStreams created from in-memory data are always retryable. Data will be converted into
Bytes
enabling a cheap clone during retries.use bytes::Bytes; use aws_smithy_types::byte_stream::ByteStream; let stream = ByteStream::from(vec![1,2,3]); let stream = ByteStream::from(Bytes::from_static(b"hello!"));
-
From a file: ByteStreams created from a path can be retried. A new file descriptor will be opened if a retry occurs.
#[cfg(feature = "tokio-rt")] use aws_smithy_types::byte_stream::ByteStream; let stream = ByteStream::from_path("big_file.csv");
-
From an
SdkBody
directly: For more advanced / custom use cases, a ByteStream can be created directly from an SdkBody. When created from an SdkBody, care must be taken to ensure retriability. An SdkBody is retryable when constructed from in-memory data or when usingSdkBody::retryable
.ⓘuse aws_smithy_types::byte_stream::ByteStream; use aws_smithy_types::body::SdkBody; use bytes::Bytes; let (mut tx, channel_body) = hyper::Body::channel(); // this will not be retryable because the SDK has no way to replay this stream let stream = ByteStream::new(SdkBody::from_body_0_4(channel_body)); tx.send_data(Bytes::from_static(b"hello world!")); tx.send_data(Bytes::from_static(b"hello again!")); // NOTE! You must ensure that `tx` is dropped to ensure that EOF is sent
Implementations§
Source§impl ByteStream
impl ByteStream
Sourcepub fn from_body_0_4<T, E>(body: T) -> Self
Available on crate feature http-body-0-4-x
only.
pub fn from_body_0_4<T, E>(body: T) -> Self
http-body-0-4-x
only.Construct a ByteStream
from a type that implements http_body_0_4::Body<Data = Bytes>
.
Note: This is only available when the http-body-0-4-x
feature is enabled.
Source§impl ByteStream
impl ByteStream
Sourcepub fn from_body_1_x<T, E>(body: T) -> Self
Available on crate feature http-body-1-x
only.
pub fn from_body_1_x<T, E>(body: T) -> Self
http-body-1-x
only.Construct a ByteStream
from a type that implements http_body_1_0::Body<Data = Bytes>
.
Note: This is only available when the http-body-1-x
feature is enabled.
Source§impl ByteStream
impl ByteStream
Sourcepub fn from_static(bytes: &'static [u8]) -> Self
pub fn from_static(bytes: &'static [u8]) -> Self
Create a new ByteStream
from a static byte slice.
Sourcepub fn into_inner(self) -> SdkBody
pub fn into_inner(self) -> SdkBody
Consume the ByteStream
, returning the wrapped SdkBody.
Sourcepub async fn next(&mut self) -> Option<Result<Bytes, Error>>
pub async fn next(&mut self) -> Option<Result<Bytes, Error>>
Return the next item in the ByteStream
.
There is also a sibling method try_next
, which returns a Result<Option<Bytes>, Error>
instead of an Option<Result<Bytes, Error>>
.
Sourcepub fn poll_next(
self: Pin<&mut Self>,
cx: &mut Context<'_>,
) -> Poll<Option<Result<Bytes, Error>>>
Available on crate feature byte-stream-poll-next
only.
pub fn poll_next( self: Pin<&mut Self>, cx: &mut Context<'_>, ) -> Poll<Option<Result<Bytes, Error>>>
byte-stream-poll-next
only.Attempt to pull out the next value of this stream, returning None
if the stream is
exhausted.
Sourcepub async fn try_next(&mut self) -> Result<Option<Bytes>, Error>
pub async fn try_next(&mut self) -> Result<Option<Bytes>, Error>
Consume and return the next item in the ByteStream
or return an error if an error is
encountered.
Similar to the next
method, but this returns a Result<Option<Bytes>, Error>
rather than
an Option<Result<Bytes, Error>>
, making for easy use with the ?
operator.
Sourcepub fn bytes(&self) -> Option<&[u8]>
pub fn bytes(&self) -> Option<&[u8]>
Returns a reference to the data if it is already available in memory
Sourcepub fn size_hint(&self) -> (u64, Option<u64>)
pub fn size_hint(&self) -> (u64, Option<u64>)
Return the bounds on the remaining length of the ByteStream
.
Sourcepub async fn collect(self) -> Result<AggregatedBytes, Error>
pub async fn collect(self) -> Result<AggregatedBytes, Error>
Read all the data from this ByteStream
into memory
If an error in the underlying stream is encountered, ByteStreamError
is returned.
Data is read into an AggregatedBytes
that stores data non-contiguously as it was received
over the network. If a contiguous slice is required, use into_bytes()
.
use bytes::Bytes;
use aws_smithy_types::body;
use aws_smithy_types::body::SdkBody;
use aws_smithy_types::byte_stream::{ByteStream, error::Error};
async fn get_data() {
let stream = ByteStream::new(SdkBody::from("hello!"));
let data: Result<Bytes, Error> = stream.collect().await.map(|data| data.into_bytes());
}
Sourcepub fn read_from() -> FsBuilder
Available on crate feature rt-tokio
only.
pub fn read_from() -> FsBuilder
rt-tokio
only.Returns a FsBuilder
, allowing you to build a ByteStream
with
full control over how the file is read (eg. specifying the length of
the file or the size of the buffer used to read the file).
use aws_smithy_types::byte_stream::{ByteStream, Length};
async fn bytestream_from_file() -> ByteStream {
let bytestream = ByteStream::read_from()
.path("docs/some-large-file.csv")
// Specify the size of the buffer used to read the file (in bytes, default is 4096)
.buffer_size(32_784)
// Specify the length of the file used (skips an additional call to retrieve the size)
.length(Length::Exact(123_456))
.build()
.await
.expect("valid path");
bytestream
}
Sourcepub async fn from_path(path: impl AsRef<Path>) -> Result<Self, Error>
Available on crate feature rt-tokio
only.
pub async fn from_path(path: impl AsRef<Path>) -> Result<Self, Error>
rt-tokio
only.Create a ByteStream that streams data from the filesystem
This function creates a retryable ByteStream for a given path
. The returned ByteStream
will provide a size hint when used as an HTTP body. If the request fails, the read will
begin again by reloading the file handle.
§Warning
The contents of the file MUST not change during retries. The length & checksum of the file will be cached. If the contents of the file change, the operation will almost certainly fail.
Furthermore, a partial write MAY seek in the file and resume from the previous location.
Note: If you want more control, such as specifying the size of the buffer used to read the file
or the length of the file, use a FsBuilder
as returned from ByteStream::read_from
.
§Examples
use aws_smithy_types::byte_stream::ByteStream;
use std::path::Path;
async fn make_bytestream() -> ByteStream {
ByteStream::from_path("docs/rows.csv").await.expect("file should be readable")
}
Sourcepub fn into_async_read(self) -> impl AsyncBufRead
Available on crate feature rt-tokio
only.
pub fn into_async_read(self) -> impl AsyncBufRead
rt-tokio
only.Convert this ByteStream
into a struct that implements AsyncBufRead
.
§Example
use tokio::io::AsyncBufReadExt;
use aws_smithy_types::byte_stream::ByteStream;
let mut lines = my_bytestream.into_async_read().lines();
while let Some(line) = lines.next_line().await? {
// Do something line by line
}
Trait Implementations§
Source§impl Debug for ByteStream
impl Debug for ByteStream
Source§impl Default for ByteStream
impl Default for ByteStream
Source§impl From<Body> for ByteStream
Available on crate features http-body-0-4-x
and hyper-0-14-x
only.
impl From<Body> for ByteStream
http-body-0-4-x
and hyper-0-14-x
only.Source§impl From<Bytes> for ByteStream
impl From<Bytes> for ByteStream
Construct a retryable ByteStream from bytes::Bytes
.
Source§impl From<SdkBody> for ByteStream
impl From<SdkBody> for ByteStream
Source§impl From<Vec<u8>> for ByteStream
impl From<Vec<u8>> for ByteStream
Construct a retryable ByteStream from a Vec<u8>
.
This will convert the Vec<u8>
into bytes::Bytes
to enable efficient retries.
impl<'__pin> Unpin for ByteStreamwhere
PinnedFieldsOf<__Origin<'__pin>>: Unpin,
Auto Trait Implementations§
impl !Freeze for ByteStream
impl !RefUnwindSafe for ByteStream
impl Send for ByteStream
impl Sync for ByteStream
impl !UnwindSafe for ByteStream
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self
into a Left
variant of Either<Self, Self>
if into_left
is true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self
into a Left
variant of Either<Self, Self>
if into_left(&self)
returns true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read more