Skip to content

Prunning expressions can reference rowcount #7589

Open
robert3005 wants to merge 4 commits intodevelopfrom
rk/rowcount
Open

Prunning expressions can reference rowcount #7589
robert3005 wants to merge 4 commits intodevelopfrom
rk/rowcount

Conversation

@robert3005
Copy link
Copy Markdown
Contributor

@robert3005 robert3005 commented Apr 22, 2026

This lets us effectively prune expressions like IsNotNull

fix #7187

Signed-off-by: Robert Kruszewski github@robertk.io

Comment thread vortex-layout/src/layouts/zoned/zone_map.rs Outdated
Comment thread vortex-layout/src/layouts/zoned/zone_map.rs Outdated
@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented Apr 22, 2026

Merging this PR will improve performance by 31.9%

⚡ 10 improved benchmarks
✅ 1153 untouched benchmarks

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation varbinview_zip_fragmented_mask 7.3 ms 6.5 ms +11.46%
Simulation varbinview_zip_block_mask 3.7 ms 2.9 ms +27.63%
Simulation old_alp_prim_test_between[f64, 16384] 211.9 µs 188.6 µs +12.31%
Simulation new_bp_prim_test_between[i32, 32768] 169 µs 139.9 µs +20.81%
Simulation old_bp_prim_test_between[i32, 32768] 232.7 µs 202.5 µs +14.89%
Simulation old_bp_prim_test_between[i64, 16384] 202.9 µs 171.7 µs +18.15%
Simulation new_alp_prim_test_between[f32, 32768] 182.7 µs 138.5 µs +31.9%
Simulation old_bp_prim_test_between[i32, 16384] 161.8 µs 146.8 µs +10.22%
Simulation new_alp_prim_test_between[f64, 16384] 147.2 µs 125.3 µs +17.56%
Simulation new_alp_prim_test_between[f64, 32768] 234.9 µs 191 µs +23.02%

Comparing rk/rowcount (5780a99) with develop (543dbe7)

Open in CodSpeed

Comment thread vortex-array/src/expr/pruning/pruning_expr.rs Outdated
@robert3005
Copy link
Copy Markdown
Contributor Author

I have tried making it an expression. The problem you run into is that this expression isn't constant so you can't rewrite it into literal but also that expression cannot be inferred from the existing arrays properties

@robert3005
Copy link
Copy Markdown
Contributor Author

Ok, this is good to go now, a lot more code than I'd wish but at least it's pure array rewrite

@robert3005 robert3005 changed the title Support returning row count in prunning aggregate expressions Prunning expressions can reference rowcount Apr 23, 2026
This lets us effectively prune expressions like IsNotNull

Signed-off-by: Robert Kruszewski <github@robertk.io>
Comment thread vortex-array/src/expr/exprs.rs
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Comment on lines 47 to 52
pub struct ZoneMap {
// The struct array backing the zone map
array: StructArray,
// The statistics that are included in the table.
stats: Arc<[Stat]>,
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its odd a zonemap doesn't know its row len?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the layout knows. We can plumb that through that's a good point

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/fix A bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add RowCount stat to zone maps for more precise pruning

2 participants