16201397870

Committed 10 Jul 2025 05:05PM UTC coverage: 81.145% (+0.06%) from 81.084%

Build # 16201397870

Build Type

push

github

Committed by

web-flow

Commit Message

Remove var expression (#3829)

Fixes #3671 

* Removes the Var expression, leaving instead a `root()` expression for
resolving the scope root.
* The expression scope can hold context variables, useful for passing in
auth tokens for example, but not variables that would impact the
return_dtype of the expression.
* ScopeDType has therefore been removed, because the dtype of the scope
_is_ just the dtype of the root array.
* Simplifies some transformation / partitioning logic where vars no
longer need to be considered.

Signed-off-by: Nicholas Gates <nick@nickgates.com>
Co-authored-by: Will Manning <will@spiraldb.com>

Run Details

164 of 175 new or added lines in 32 files covered. (93.71%)

18 existing lines in 6 files now uncovered.

45273 of 55793 relevant lines covered (81.14%)

146273.04 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

40.0

/vortex-expr/src/analysis.rs

// SPDX-License-Identifier: Apache-2.0
// SPDX-FileCopyrightText: Copyright the Vortex contributors

use vortex_array::stats::Stat;
use vortex_dtype::FieldPath;

use crate::ExprRef;

pub trait StatsCatalog {
    /// Given a field path and statist, return an expression that when evaluated over the catalog
    /// will return that stat for the referenced field.
    ///
    /// This is likely to be a column expression, or a literal.
    ///
    /// Returns `None` if the stat is not available for the field path.
    fn stats_ref(&mut self, _field_path: &FieldPath, _stat: Stat) -> Option<ExprRef> {
        None
    }
}

/// This can be used by expression to plug into vortex expression analysis, such as
/// pruning or expression simplification
pub trait AnalysisExpr {
    /// An expression over zone-statistics which implies all records in the zone evaluate to false.
    ///
    /// Given an expression, `e`, if `e.stat_falsification(..)` evaluates to true, it is guaranteed
    /// that `e` evaluates to false on all records in the zone. However, the inverse is not
    /// necessarily true: even if the falsification evaluates to false, `e` need not evaluate to
    /// true on all records.
    ///
    /// The `StatsCatalog` can be used to constrain or rename stats used in the final expr.
    ///
    /// # Examples
    ///
    /// - An expression over one variable: `x > 0` is false for all records in a zone if the maximum
    ///   value of the column `x` in that zone is less than or equal to zero: `max(x) <= 0`.
    /// - An expression over two variables: `x > y` becomes `max(x) <= min(y)`.
    /// - A conjunctive expression: `x > y AND z < x` becomes `max(x) <= min(y) OR min(z) >= max(x).
    ///
    /// Some expressions, in theory, have falsifications but this function does not support them
    /// such as `x < (y < z)` or `x LIKE "needle%"`.
    fn stat_falsification(&self, _catalog: &mut dyn StatsCatalog) -> Option<ExprRef> {
        None
    }

    /// An expression for the upper non-null bound of this expression, if available.
    ///
    /// This function returns None if there is no upper bound or it is difficult to compute.
    ///
    /// The returned expression evaluates to null if the maximum value is unknown. In that case, you
    /// _must not_ assume the array is empty _nor_ may you assume the array only contains non-null
    /// values.
    fn max(&self, _catalog: &mut dyn StatsCatalog) -> Option<ExprRef> {
        None
    }

    /// An expression for the lower non-null bound of this expression, if available.
    ///
    /// See [AnalysisExpr::max] for important details.
    fn min(&self, _catalog: &mut dyn StatsCatalog) -> Option<ExprRef> {
        None
    }

    fn field_path(&self) -> Option<FieldPath> {
        None
    }

    // TODO: add containment
}

1	// SPDX-License-Identifier: Apache-2.0
2	// SPDX-FileCopyrightText: Copyright the Vortex contributors
3
4	use vortex_array::stats::Stat;
5	use vortex_dtype::FieldPath;
6
7	use crate::ExprRef;
8
9	pub trait StatsCatalog {
10	/// Given a field path and statist, return an expression that when evaluated over the catalog
11	/// will return that stat for the referenced field.
12	///
13	/// This is likely to be a column expression, or a literal.
14	///
15	/// Returns `None` if the stat is not available for the field path.
NEW 16	fn stats_ref(&mut self, _field_path: &FieldPath, _stat: Stat) -> Option<ExprRef> {	×
17	None	×
18	}	×
19	}
20
21	/// This can be used by expression to plug into vortex expression analysis, such as
22	/// pruning or expression simplification
23	pub trait AnalysisExpr {
24	/// An expression over zone-statistics which implies all records in the zone evaluate to false.
25	///
26	/// Given an expression, `e`, if `e.stat_falsification(..)` evaluates to true, it is guaranteed
27	/// that `e` evaluates to false on all records in the zone. However, the inverse is not
28	/// necessarily true: even if the falsification evaluates to false, `e` need not evaluate to
29	/// true on all records.
30	///
31	/// The `StatsCatalog` can be used to constrain or rename stats used in the final expr.
32	///
33	/// # Examples
34	///
35	/// - An expression over one variable: `x > 0` is false for all records in a zone if the maximum
36	/// value of the column `x` in that zone is less than or equal to zero: `max(x) <= 0`.
37	/// - An expression over two variables: `x > y` becomes `max(x) <= min(y)`.
38	/// - A conjunctive expression: `x > y AND z < x` becomes `max(x) <= min(y) OR min(z) >= max(x).
39	///
40	/// Some expressions, in theory, have falsifications but this function does not support them
41	/// such as `x < (y < z)` or `x LIKE "needle%"`.
42	fn stat_falsification(&self, _catalog: &mut dyn StatsCatalog) -> Option<ExprRef> {	76✔
43	None	76✔
44	}	76✔
45
46	/// An expression for the upper non-null bound of this expression, if available.
47	///
48	/// This function returns None if there is no upper bound or it is difficult to compute.
49	///
50	/// The returned expression evaluates to null if the maximum value is unknown. In that case, you
51	/// _must not_ assume the array is empty _nor_ may you assume the array only contains non-null
52	/// values.
53	fn max(&self, _catalog: &mut dyn StatsCatalog) -> Option<ExprRef> {	×
54	None	×
55	}	×
56
57	/// An expression for the lower non-null bound of this expression, if available.
58	///
59	/// See [AnalysisExpr::max] for important details.
60	fn min(&self, _catalog: &mut dyn StatsCatalog) -> Option<ExprRef> {	1✔
61	None	1✔
62	}	1✔
63
NEW 64	fn field_path(&self) -> Option<FieldPath> {	×
65	None	×
66	}	×
67
68	// TODO: add containment
69	}

vortex-data / vortex / 16201397870

Source File Press 'n' to go to next uncovered line, 'b' for previous

Source File
Press 'n' to go to next uncovered line, 'b' for previous