Expand description
ByteStream Abstractions
When the SDK returns streaming binary data, the inner Http Body is
wrapped in ByteStream
. ByteStream provides misuse-resistant primitives
to make it easier to handle common patterns with streaming data.
§Examples
§Writing a ByteStream into a file:
use aws_smithy_types::byte_stream::ByteStream;
use std::error::Error;
use tokio::fs::File;
use tokio::io::AsyncWriteExt;
struct SynthesizeSpeechOutput {
audio_stream: ByteStream,
}
async fn audio_to_file(
output: SynthesizeSpeechOutput,
) -> Result<(), Box<dyn Error + Send + Sync>> {
let mut buf = output.audio_stream.collect().await?;
let mut file = File::open("audio.mp3").await?;
file.write_all_buf(&mut buf).await?;
file.flush().await?;
Ok(())
}
§Converting a ByteStream into Bytes
use bytes::Bytes;
use aws_smithy_types::byte_stream::ByteStream;
use std::error::Error;
struct SynthesizeSpeechOutput {
audio_stream: ByteStream,
}
async fn load_audio(
output: SynthesizeSpeechOutput,
) -> Result<Bytes, Box<dyn Error + Send + Sync>> {
Ok(output.audio_stream.collect().await?.into_bytes())
}
§Stream a ByteStream into a file
The previous example is recommended in cases where loading the entire file into memory first is desirable. For extremely large
files, you may wish to stream the data directly to the file system, chunk by chunk.
This is possible using the .next()
method.
use bytes::{Buf, Bytes};
use aws_smithy_types::byte_stream::ByteStream;
use std::error::Error;
use tokio::fs::File;
use tokio::io::AsyncWriteExt;
use tokio_stream::StreamExt;
struct SynthesizeSpeechOutput {
audio_stream: ByteStream,
}
async fn audio_to_file(
output: SynthesizeSpeechOutput,
) -> Result<(), Box<dyn Error + Send + Sync>> {
let mut file = File::open("audio.mp3").await?;
let mut stream = output.audio_stream;
while let Some(bytes) = stream.next().await {
let bytes: Bytes = bytes?;
file.write_all(&bytes).await?;
}
file.flush().await?;
Ok(())
}
§Create a ByteStream from a file
Note: This is only available with rt-tokio
enabled.
use aws_smithy_types::byte_stream::ByteStream;
use std::path::Path;
struct GetObjectInput {
body: ByteStream
}
async fn bytestream_from_file() -> GetObjectInput {
let bytestream = ByteStream::from_path("docs/some-large-file.csv")
.await
.expect("valid path");
GetObjectInput { body: bytestream }
}
If you want more control over how the file is read, such as specifying the size of the buffer used to read the file
or the length of the file, use an FsBuilder
.
use aws_smithy_types::byte_stream::{ByteStream, Length};
use std::path::Path;
struct GetObjectInput {
body: ByteStream
}
async fn bytestream_from_file() -> GetObjectInput {
let bytestream = ByteStream::read_from().path("docs/some-large-file.csv")
.buffer_size(32_784)
.length(Length::Exact(123_456))
.build()
.await
.expect("valid path");
GetObjectInput { body: bytestream }
}
Modules§
- Errors related to bytestreams.
- http_
body_ 0_ 4_ x http-body-0-4-x
This module is named after thehttp-body
version number since we anticipate needing to provide equivalent functionality for 1.x of that crate in the future. The name has a suffix_x
to avoid name collision with a third-partyhttp-body-0-4
. - http_
body_ 1_ x http-body-1-x
Adapters to use http-body 1.0 bodies with SdkBody & ByteStream
Structs§
- Non-contiguous Binary Data Storage
- Stream of binary data
- FsBuilder
rt-tokio
Builder for creatingByteStreams
from a file/path, with full control over advanced options.
Enums§
- Length
rt-tokio
The length (in bytes) to read. Determines whether or not a short read counts as an error.