pub struct Reader { /* private fields */ }Expand description
A pull based CSV reader.
This reader parses CSV data using a finite state machine. Callers can
extract parsed data incrementally using one of the read methods.
Note that this CSV reader is somewhat encoding agnostic. The source data
needs to be at least ASCII compatible. There is no support for specifying
the full gamut of Unicode delimiters/terminators/quotes/escapes. Instead,
any byte can be used, although callers probably want to stick to the ASCII
subset (<= 0x7F).
Usage
A reader has two different ways to read CSV data, each with their own trade offs.
read_field- Copies a single CSV field into an output buffer while unescaping quotes. This is simple to use and doesn’t require storing an entire record contiguously in memory, but it is slower.read_record- Copies an entire CSV record into an output buffer while unescaping quotes. The ending positions of each field are copied into an additional buffer. This is harder to use and requires larger output buffers, but it is faster thanread_fieldsince it amortizes more costs.
RFC 4180
RFC 4180 is the closest thing to a specification for CSV data. Unfortunately, CSV data that is seen in the wild can vary significantly. Often, the CSV data is outright invalid. Instead of fixing the producers of bad CSV data, we have seen fit to make consumers much more flexible in what they accept. This reader continues that tradition, and therefore, isn’t technically compliant with RFC 4180. In particular, this reader will never return an error and will always find a parse.
Here are some detailed differences from RFC 4180:
- CRLF, LF and CR are each treated as a single record terminator by default.
- Records are permitted to be of varying length.
- Empty lines (that do not include other whitespace) are ignored.
Implementations§
source§impl Reader
impl Reader
sourcepub fn reset(&mut self)
pub fn reset(&mut self)
Reset the parser such that it behaves as if it had never been used.
This may be useful when reading CSV data in a random access pattern.
sourcepub fn line(&self) -> u64
pub fn line(&self) -> u64
Return the current line number as measured by the number of occurrences
of \n.
Line numbers starts at 1 and are reset when reset is called.
sourcepub fn set_line(&mut self, line: u64)
pub fn set_line(&mut self, line: u64)
Set the line number.
This is useful after a call to reset where the caller knows the
line number from some additional context.
sourcepub fn read_field(
&mut self,
input: &[u8],
output: &mut [u8]
) -> (ReadFieldResult, usize, usize)
pub fn read_field( &mut self, input: &[u8], output: &mut [u8] ) -> (ReadFieldResult, usize, usize)
Parse a single CSV field in input and copy field data to output.
This routine requires a caller provided buffer of CSV data as the
input and a caller provided buffer, output, in which to store field
data extracted from input. The field data copied to output will
have its quotes unescaped.
Calling this routine parses at most a single field and returns
three values indicating the state of the parser. The first value, a
ReadFieldResult, tells the caller what to do next. For example, if
the entire input was read or if the output buffer was filled before
a full field had been read, then ReadFieldResult::InputEmpty or
ReadFieldResult::OutputFull is returned, respectively. See the
documentation for ReadFieldResult for more details.
The other two values returned correspond to the number of bytes
read from input and written to output, respectively.
Termination
This reader interprets an empty input buffer as an indication that
there is no CSV data left to read. Namely, when the caller has
exhausted all CSV data, the caller should continue to call read with
an empty input buffer until ReadFieldResult::End is returned.
Errors
This CSV reader can never return an error. Instead, it prefers a parse over no parse.
sourcepub fn read_record(
&mut self,
input: &[u8],
output: &mut [u8],
ends: &mut [usize]
) -> (ReadRecordResult, usize, usize, usize)
pub fn read_record( &mut self, input: &[u8], output: &mut [u8], ends: &mut [usize] ) -> (ReadRecordResult, usize, usize, usize)
Parse a single CSV record in input and copy each field contiguously
to output, with the end position of each field written to ends.
NOTE: This method is more cumbersome to use than read_field, but
it can be faster since it amortizes more work.
This routine requires a caller provided buffer of CSV data as the
input and two caller provided buffers to store the unescaped field
data (output) and the end position of each field in the record
(fields).
Calling this routine parses at most a single record and returns four
values indicating the state of the parser. The first value, a
ReadRecordResult, tells the caller what to do next. For example, if
the entire input was read or if the output buffer was filled before a
full field had been read, then ReadRecordResult::InputEmpty or
ReadRecordResult::OutputFull is returned, respectively. Similarly, if
the ends buffer is full, then ReadRecordResult::OutputEndsFull is
returned. See the documentation for ReadRecordResult for more
details.
The other three values correspond to the number of bytes read from
input, the number of bytes written to output and the number of
end positions written to ends, respectively.
The end positions written to ends are constructed as if there was
a single contiguous buffer in memory containing the entire row, even
if ReadRecordResult::OutputFull was returned in the middle of reading
a row.
Termination
This reader interprets an empty input buffer as an indication that
there is no CSV data left to read. Namely, when the caller has
exhausted all CSV data, the caller should continue to call read with
an empty input buffer until ReadRecordResult::End is returned.
Errors
This CSV reader can never return an error. Instead, it prefers a parse over no parse.