pub(crate) fn read_rows(job: &CsvJob, tx: &SyncSender<ReadItem>)Expand description
Reads the rows of every channel of an export, in order, and hands them over.
A channel with filters goes through FilteredChannelRows. Otherwise, for each
UTC date from the start date to the end date inclusive, groups the .parquet
files in <root>/assetNNN/YYYY-MM-DD/chNN/ (skipping days whose directory does
not exist) with overlapping_file_groups, and hands over each group’s rows,
one merge slice at a time, in time order with every exact duplicate sample once
(see read_merged_rows). For an archive without overlapping files this is the
same as each file’s rows in turn. A file that fails to read is logged and
skipped, and the export continues; a day folder that cannot be listed ends it
with ReadItem::Failed.
tx is a rendezvous channel, so this runs one item ahead of the formatter and
returns as soon as the formatter has gone.