Skip to main content

overlapping_file_groups

Function overlapping_file_groups 

Source
pub(crate) fn overlapping_file_groups(
    day_dir: &Path,
    start_ns: i64,
    end_ns: i64,
) -> Result<Vec<Vec<FileSpan>>>
Expand description

Lists one channel-day folder and groups its files by overlapping time spans.

Files are found the same way the export always has: every name ending in .parquet. Files with no rows, or whose span lies outside [start_ns, end_ns], are left out. A file whose span cannot be read is logged and skipped, and the export continues.

The remaining files are grouped so that files whose spans overlap (or touch at the same nanosecond) share a group, directly or through other files. The aligned five-minute files the archiver writes never overlap, so each is a group of its own. Duplicate copies written by a replayed backlog overlap the originals and end up in one group, whatever their boundaries, which is what lets read_merged_rows drop the copies.

§Arguments

  • day_dir - <root>/assetNNN/YYYY-MM-DD/chNN/ folder.
  • start_ns - Inclusive range start, Unix nanoseconds.
  • end_ns - Inclusive range end, Unix nanoseconds.

§Returns

The groups in time order. Files inside a group are in file-name order.

§Errors

Returns an error if the folder cannot be listed.