24841644892

Committed 23 Apr 2026 02:44PM UTC coverage: 85.279% (-4.0%) from 89.306%

Build # 24841644892

Build Type

push

gh-ci

Committed by

web-flow

Commit Message

feat: v1.11.0 PostgreSQL compatibility and SQL feature expansion (#2090)

* Add structured audit logging with immutable audit trail

Introduces a new --audit-log flag that records all gRPC operations as
structured JSON events in immudb's tamper-proof KV store. Events are
stored under the audit: key prefix in systemdb, queryable via Scan and
verifiable via VerifiableGet. An async buffered writer ensures minimal
latency impact. Configurable event filtering (all/write/admin) via
--audit-log-events flag.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add PostgreSQL ORM compatibility layer and verification functions

Extend the pgsql wire protocol with immudb verification functions
(immudb_state, immudb_verify_row, immudb_verify_tx, immudb_history,
immudb_tx) accessible via standard SQL SELECT statements.

Add pg_catalog resolvers (pg_attribute, pg_index, pg_constraint,
pg_type, pg_settings, pg_description) and information_schema
resolvers (tables, columns, schemata, key_column_usage) to support
ORM introspection from Django, SQLAlchemy, GORM, and ActiveRecord.

Add PostgreSQL compatibility functions: current_database,
current_schema, current_user, format_type, pg_encoding_to_char,
pg_get_expr, pg_get_constraintdef, obj_description, col_description,
has_table_privilege, has_schema_privilege, and others.

Add SHOW statement emulation for common ORM config queries and
schema-qualified name stripping for information_schema and public
schema references.

* Implement EXISTS and IN subquery support in SQL engine

Replace the previously stubbed ExistsBoolExp and InSubQueryExp
implementations with working non-correlated subquery execution.

EXISTS subqueries resolve the inner SELECT and check if any rows
are returned. IN subqueries resolve the inner SELECT, iterate the
result set, and compare each value against the outer expression.
Both support NOT variants (NOT EXISTS, NOT IN).

Correlated subqueries (referencing outer query columns) ar... (continued)

Coverage Stats

7254 of 10471 new or added lines in 124 files covered. (69.28%)

115 existing lines in 18 files now uncovered.

44599 of 52298 relevant lines covered (85.28%)

127676.6 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

85.48

/embedded/sql/predicate_pushdown.go

/*
Copyright 2026 Codenotary Inc. All rights reserved.

SPDX-License-Identifier: BUSL-1.1
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    https://mariadb.com/bsl11/

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package sql

// splitAndConjuncts decomposes a (possibly nested) AND expression into a
// flat slice of conjuncts. Non-AND expressions are returned as a single-
// element slice. Used for predicate pushdown so each conjunct can be
// classified and re-located independently of the others.
func splitAndConjuncts(e ValueExp) []ValueExp {
        if e == nil {
                return nil
        }
        if be, ok := e.(*BinBoolExp); ok && be.op == And {
                return append(splitAndConjuncts(be.left), splitAndConjuncts(be.right)...)
        }
        return []ValueExp{e}
}

// andConjuncts re-assembles a slice of conjuncts into a single AND-tree.
// Returns nil for empty input.
func andConjuncts(parts []ValueExp) ValueExp {
        if len(parts) == 0 {
                return nil
        }
        res := parts[0]
        for _, p := range parts[1:] {
                res = &BinBoolExp{op: And, left: res, right: p}
        }
        return res
}

// collectColTables walks a ValueExp tree, recording the set of explicit
// table aliases referenced by *ColSelector nodes.
//
// Returns:
//
//        tables         — set of distinct non-empty table aliases referenced
//        hasUnqualified — true if any ColSelector has empty table (ambiguous in joins)
//        safe           — false on any unrecognized node type. Callers MUST treat
//                         a non-safe expression as un-classifiable and refrain
//                         from pushdown/relocation: a constant-folder, subquery,
//                         aggregate, etc. could otherwise be silently moved across
//                         a join boundary and produce wrong results.
//
// Recognized node types cover the bulk of WHERE/ON predicates seen in
// practice: comparisons, AND/OR/NOT, LIKE, IN-list, arithmetic, casts,
// column refs, parameters, and concrete TypedValues. Subqueries, EXISTS,
// function calls and aggregates intentionally fall through to safe=false.
func collectColTables(e ValueExp) (tables map[string]struct{}, hasUnqualified bool, safe bool) {
        tables = make(map[string]struct{})
        safe = collectColTablesRec(e, tables, &hasUnqualified)
        return
}

func collectColTablesRec(e ValueExp, tables map[string]struct{}, hasUnqualified *bool) bool {
        switch n := e.(type) {
        case nil:
                return true
        case *ColSelector:
                if n.table == "" {
                        *hasUnqualified = true
                } else {
                        tables[n.table] = struct{}{}
                }
                return true
        case *CmpBoolExp:
                return collectColTablesRec(n.left, tables, hasUnqualified) &&
                        collectColTablesRec(n.right, tables, hasUnqualified)
        case *BinBoolExp:
                return collectColTablesRec(n.left, tables, hasUnqualified) &&
                        collectColTablesRec(n.right, tables, hasUnqualified)
        case *NotBoolExp:
                return collectColTablesRec(n.exp, tables, hasUnqualified)
        case *LikeBoolExp:
                return collectColTablesRec(n.val, tables, hasUnqualified) &&
                        collectColTablesRec(n.pattern, tables, hasUnqualified)
        case *NumExp:
                return collectColTablesRec(n.left, tables, hasUnqualified) &&
                        collectColTablesRec(n.right, tables, hasUnqualified)
        case *Cast:
                return collectColTablesRec(n.val, tables, hasUnqualified)
        case *InListExp:
                if !collectColTablesRec(n.val, tables, hasUnqualified) {
                        return false
                }
                for _, v := range n.values {
                        if !collectColTablesRec(v, tables, hasUnqualified) {
                                return false
                        }
                }
                return true
        case *Param:
                return true
        case TypedValue:
                // All concrete value types (Integer, Number, Varchar, Bool,
                // NullValue, Blob, JSON, Timestamp, etc.) implement TypedValue
                // and reference no columns.
                return true
        default:
                // Unknown node type — bail. Subqueries, EXISTS clauses, function
                // calls and aggregates land here intentionally.
                return false
        }
}

// pushdownInnerOnlyConjuncts splits stmt.where, attaches conjuncts that
// reference only an INNER join's inner table to that join's cond, and
// returns the residual WHERE plus the (possibly modified) joins slice.
//
// The original stmt.where and stmt.joins are NOT mutated — a local copy
// of any modified JoinSpec is returned in joinsOut. Callers that care
// only about the WHERE/joins for this Resolve call should use the returned
// values; for cases with no eligible pushdown, this function returns the
// originals unchanged (joinsOut == joins).
//
// Eligibility per conjunct:
//   - References exactly one explicit table alias (no unqualified columns).
//   - That alias matches the *tableRef alias of an INNER, non-LATERAL,
//     non-NATURAL join's ds.
//   - All sub-expressions are recognised by collectColTables (so subqueries
//     and other unsupported forms stay in WHERE).
//
// LEFT/RIGHT/CROSS/FULL OUTER joins are skipped: pushing an inner-only
// predicate into the inner of a LEFT JOIN changes semantics (an inner
// row that fails the predicate would NULL-extend the outer row instead
// of being filtered out by the outer WHERE).
func pushdownInnerOnlyConjuncts(where ValueExp, joins []*JoinSpec) (whereOut ValueExp, joinsOut []*JoinSpec) {
        if where == nil || len(joins) == 0 {
                return where, joins
        }

        // Map inner table alias → join index for eligible (INNER, non-LATERAL,
        // non-NATURAL, *tableRef ds) joins only.
        joinByAlias := make(map[string]int, len(joins))
        for i, jspec := range joins {
                if jspec.joinType != InnerJoin || jspec.lateral || jspec.natural {
                        continue
                }
                tref, ok := jspec.ds.(*tableRef)
                if !ok {
                        continue
                }
                // First-write-wins on alias collision (shouldn't happen — duplicate
                // aliases are a parse-time error — but be defensive).
                if _, exists := joinByAlias[tref.Alias()]; !exists {
                        joinByAlias[tref.Alias()] = i
                }
        }

        if len(joinByAlias) == 0 {
                return where, joins
        }

        conjuncts := splitAndConjuncts(where)
        residual := make([]ValueExp, 0, len(conjuncts))
        pushPerJoin := make(map[int][]ValueExp)

        for _, c := range conjuncts {
                tables, hasUnqualified, safe := collectColTables(c)
                if !safe || hasUnqualified || len(tables) != 1 {
                        residual = append(residual, c)
                        continue
                }
                var only string
                for t := range tables {
                        only = t
                }
                if joinIdx, ok := joinByAlias[only]; ok {
                        pushPerJoin[joinIdx] = append(pushPerJoin[joinIdx], c)
                } else {
                        residual = append(residual, c)
                }
        }

        if len(pushPerJoin) == 0 {
                return where, joins
        }

        joinsOut = make([]*JoinSpec, len(joins))
        for i, jspec := range joins {
                extras, ok := pushPerJoin[i]
                if !ok {
                        joinsOut[i] = jspec
                        continue
                }
                cp := *jspec
                merged := jspec.cond
                for _, e := range extras {
                        if merged == nil {
                                merged = e
                        } else {
                                merged = &BinBoolExp{op: And, left: merged, right: e}
                        }
                }
                cp.cond = merged
                joinsOut[i] = &cp
        }

        return andConjuncts(residual), joinsOut
}

1	/*
2	Copyright 2026 Codenotary Inc. All rights reserved.
3
4	SPDX-License-Identifier: BUSL-1.1
5	you may not use this file except in compliance with the License.
6	You may obtain a copy of the License at
7
8	https://mariadb.com/bsl11/
9
10	Unless required by applicable law or agreed to in writing, software
11	distributed under the License is distributed on an "AS IS" BASIS,
12	WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13	See the License for the specific language governing permissions and
14	limitations under the License.
15	*/
16
17	package sql
18
19	// splitAndConjuncts decomposes a (possibly nested) AND expression into a
20	// flat slice of conjuncts. Non-AND expressions are returned as a single-
21	// element slice. Used for predicate pushdown so each conjunct can be
22	// classified and re-located independently of the others.
23	func splitAndConjuncts(e ValueExp) []ValueExp {	20✔
24	if e == nil {	20✔
NEW 25	return nil	×
NEW 26	}	×
27	if be, ok := e.(*BinBoolExp); ok && be.op == And {	24✔
28	return append(splitAndConjuncts(be.left), splitAndConjuncts(be.right)...)	4✔
29	}	4✔
30	return []ValueExp{e}	16✔
31	}
32
33	// andConjuncts re-assembles a slice of conjuncts into a single AND-tree.
34	// Returns nil for empty input.
35	func andConjuncts(parts []ValueExp) ValueExp {	592✔
36	if len(parts) == 0 {	1,168✔
37	return nil	576✔
38	}	576✔
39	res := parts[0]	16✔
40	for _, p := range parts[1:] {	16✔
NEW 41	res = &BinBoolExp{op: And, left: res, right: p}	×
NEW 42	}	×
43	return res	16✔
44	}
45
46	// collectColTables walks a ValueExp tree, recording the set of explicit
47	// table aliases referenced by *ColSelector nodes.
48	//
49	// Returns:
50	//
51	// tables — set of distinct non-empty table aliases referenced
52	// hasUnqualified — true if any ColSelector has empty table (ambiguous in joins)
53	// safe — false on any unrecognized node type. Callers MUST treat
54	// a non-safe expression as un-classifiable and refrain
55	// from pushdown/relocation: a constant-folder, subquery,
56	// aggregate, etc. could otherwise be silently moved across
57	// a join boundary and produce wrong results.
58	//
59	// Recognized node types cover the bulk of WHERE/ON predicates seen in
60	// practice: comparisons, AND/OR/NOT, LIKE, IN-list, arithmetic, casts,
61	// column refs, parameters, and concrete TypedValues. Subqueries, EXISTS,
62	// function calls and aggregates intentionally fall through to safe=false.
63	func collectColTables(e ValueExp) (tables map[string]struct{}, hasUnqualified bool, safe bool) {	37✔
64	tables = make(map[string]struct{})	37✔
65	safe = collectColTablesRec(e, tables, &hasUnqualified)	37✔
66	return	37✔
67	}	37✔
68
69	func collectColTablesRec(e ValueExp, tables map[string]struct{}, hasUnqualified *bool) bool {	115✔
70	switch n := e.(type) {	115✔
NEW 71	case nil:	×
NEW 72	return true	×
73	case *ColSelector:	31✔
74	if n.table == "" {	32✔
75	*hasUnqualified = true	1✔
76	} else {	31✔
77	tables[n.table] = struct{}{}	30✔
78	}	30✔
79	return true	31✔
80	case *CmpBoolExp:	36✔
81	return collectColTablesRec(n.left, tables, hasUnqualified) &&	36✔
82	collectColTablesRec(n.right, tables, hasUnqualified)	36✔
83	case *BinBoolExp:	1✔
84	return collectColTablesRec(n.left, tables, hasUnqualified) &&	1✔
85	collectColTablesRec(n.right, tables, hasUnqualified)	1✔
NEW 86	case *NotBoolExp:	×
NEW 87	return collectColTablesRec(n.exp, tables, hasUnqualified)	×
NEW 88	case *LikeBoolExp:	×
NEW 89	return collectColTablesRec(n.val, tables, hasUnqualified) &&	×
NEW 90	collectColTablesRec(n.pattern, tables, hasUnqualified)	×
91	case *NumExp:	1✔
92	return collectColTablesRec(n.left, tables, hasUnqualified) &&	1✔
93	collectColTablesRec(n.right, tables, hasUnqualified)	1✔
NEW 94	case *Cast:	×
NEW 95	return collectColTablesRec(n.val, tables, hasUnqualified)	×
96	case *InListExp:	1✔
97	if !collectColTablesRec(n.val, tables, hasUnqualified) {	1✔
NEW 98	return false	×
NEW 99	}	×
100	for _, v := range n.values {	2✔
101	if !collectColTablesRec(v, tables, hasUnqualified) {	1✔
NEW 102	return false	×
NEW 103	}	×
104	}
105	return true	1✔
106	case *Param:	1✔
107	return true	1✔
108	case TypedValue:	43✔
109	// All concrete value types (Integer, Number, Varchar, Bool,	43✔
110	// NullValue, Blob, JSON, Timestamp, etc.) implement TypedValue	43✔
111	// and reference no columns.	43✔
112	return true	43✔
113	default:	1✔
114	// Unknown node type — bail. Subqueries, EXISTS clauses, function	1✔
115	// calls and aggregates land here intentionally.	1✔
116	return false	1✔
117	}
118	}
119
120	// pushdownInnerOnlyConjuncts splits stmt.where, attaches conjuncts that
121	// reference only an INNER join's inner table to that join's cond, and
122	// returns the residual WHERE plus the (possibly modified) joins slice.
123	//
124	// The original stmt.where and stmt.joins are NOT mutated — a local copy
125	// of any modified JoinSpec is returned in joinsOut. Callers that care
126	// only about the WHERE/joins for this Resolve call should use the returned
127	// values; for cases with no eligible pushdown, this function returns the
128	// originals unchanged (joinsOut == joins).
129	//
130	// Eligibility per conjunct:
131	// - References exactly one explicit table alias (no unqualified columns).
132	// - That alias matches the *tableRef alias of an INNER, non-LATERAL,
133	// non-NATURAL join's ds.
134	// - All sub-expressions are recognised by collectColTables (so subqueries
135	// and other unsupported forms stay in WHERE).
136	//
137	// LEFT/RIGHT/CROSS/FULL OUTER joins are skipped: pushing an inner-only
138	// predicate into the inner of a LEFT JOIN changes semantics (an inner
139	// row that fails the predicate would NULL-extend the outer row instead
140	// of being filtered out by the outer WHERE).
141	func pushdownInnerOnlyConjuncts(where ValueExp, joins []JoinSpec) (whereOut ValueExp, joinsOut []JoinSpec) {	67✔
142	if where == nil \|\| len(joins) == 0 {	118✔
143	return where, joins	51✔
144	}	51✔
145
146	// Map inner table alias → join index for eligible (INNER, non-LATERAL,
147	// non-NATURAL, *tableRef ds) joins only.
148	joinByAlias := make(map[string]int, len(joins))	16✔
149	for i, jspec := range joins {	35✔
150	if jspec.joinType != InnerJoin \|\| jspec.lateral \|\| jspec.natural {	23✔
151	continue	4✔
152	}
153	tref, ok := jspec.ds.(*tableRef)	15✔
154	if !ok {	16✔
155	continue	1✔
156	}
157	// First-write-wins on alias collision (shouldn't happen — duplicate
158	// aliases are a parse-time error — but be defensive).
159	if _, exists := joinByAlias[tref.Alias()]; !exists {	28✔
160	joinByAlias[tref.Alias()] = i	14✔
161	}	14✔
162	}
163
164	if len(joinByAlias) == 0 {	20✔
165	return where, joins	4✔
166	}	4✔
167
168	conjuncts := splitAndConjuncts(where)	12✔
169	residual := make([]ValueExp, 0, len(conjuncts))	12✔
170	pushPerJoin := make(map[int][]ValueExp)	12✔
171		12✔
172	for _, c := range conjuncts {	28✔
173	tables, hasUnqualified, safe := collectColTables(c)	16✔
174	if !safe \|\| hasUnqualified \|\| len(tables) != 1 {	18✔
175	residual = append(residual, c)	2✔
176	continue	2✔
177	}
178	var only string	14✔
179	for t := range tables {	28✔
180	only = t	14✔
181	}	14✔
182	if joinIdx, ok := joinByAlias[only]; ok {	20✔
183	pushPerJoin[joinIdx] = append(pushPerJoin[joinIdx], c)	6✔
184	} else {	14✔
185	residual = append(residual, c)	8✔
186	}	8✔
187	}
188
189	if len(pushPerJoin) == 0 {	18✔
190	return where, joins	6✔
191	}	6✔
192
193	joinsOut = make([]*JoinSpec, len(joins))	6✔
194	for i, jspec := range joins {	14✔
195	extras, ok := pushPerJoin[i]	8✔
196	if !ok {	10✔
197	joinsOut[i] = jspec	2✔
198	continue	2✔
199	}
200	cp := *jspec	6✔
201	merged := jspec.cond	6✔
202	for _, e := range extras {	12✔
203	if merged == nil {	6✔
NEW 204	merged = e	×
205	} else {	6✔
206	merged = &BinBoolExp{op: And, left: merged, right: e}	6✔
207	}	6✔
208	}
209	cp.cond = merged	6✔
210	joinsOut[i] = &cp	6✔
211	}
212
213	return andConjuncts(residual), joinsOut	6✔
214	}

codenotary / immudb / 24841644892

Source File Press 'n' to go to next uncovered line, 'b' for previous

Source File
Press 'n' to go to next uncovered line, 'b' for previous