Struct CsvStreamer 

pub(crate) struct CsvStreamer<'a, S: ExportSink + Send> {
    pub(crate) sink: &'a mut S,
    pub(crate) bytes_sent: usize,
    pub(crate) asset: u32,
    pub(crate) start: DateTime<Utc>,
    pub(crate) end: DateTime<Utc>,
    pub(crate) filters: Option<ExportFilters>,
}

CSV streamer for direct WebSocket exports.

Builds the CSV on a CSV thread (spawn_csv) and hands it to an ExportSink in chunks of about Self::CHUNK_SIZE bytes. There is no ack protocol; backpressure comes from awaiting each send on the sink.

Fields

sink: &'a mut S

Transport that receives the frames.

bytes_sent: usize

CSV bytes handed to the sink so far.

asset: u32

Asset number used to locate the Parquet partitions.

start: DateTime<Utc>

Inclusive start of the export range.

end: DateTime<Utc>

Inclusive end of the export range.

filters: Option<ExportFilters>

Filters asked for; None for a plain export.

Implementations

impl<'a, S: ExportSink + Send> CsvStreamer<'a, S>

pub(crate) const CHUNK_SIZE: usize

Buffer size in bytes at which a chunk is sent to the sink (128 KiB).

pub(crate) fn new( sink: &'a mut S, asset: u32, start: DateTime<Utc>, end: DateTime<Utc>, ) -> Self

Creates a direct CSV streamer.

Arguments

• sink - Transport that receives the frames.
• asset - Asset number used to locate the Parquet partitions.
• start - Inclusive start of the export range.
• end - Inclusive end of the export range.

pub(crate) fn with_filters(self, filters: Option<ExportFilters>) -> Self

Sets the filters of the export. With Some, the header gets a filtered_value column and every row a value for it.

pub(crate) async fn stream_channels( &mut self, parquet_root: &Path, channels: &[u8], ) -> Result<Vec<u8>>

Streams all requested channels, one after another, in the given order.

The CSV is built by spawn_csv and every chunk is sent as soon as it is ready.

Arguments

• parquet_root - Root of the Parquet archive.
• channels - Channels to export.

Returns

The channels that produced no rows in the range, in request order.

Errors

Returns an error if the sink fails to send, or the first channel error from spawn_csv, prefixed with channel NN:, after the chunks completed before it.

pub(crate) async fn finish(self, missing_channels: Vec<u8>) -> Result<()>

Sends the summary and completion frames.

Call after Self::stream_channels, which sends every chunk. The missing channels are sorted and deduplicated.

Arguments

• missing_channels - Channels that produced no rows.

Errors

Returns an error if the sink fails to send.