• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

codenotary / immudb / 24841571249

23 Apr 2026 02:44PM UTC coverage: 85.275% (-4.0%) from 89.306%
24841571249

push

gh-ci

web-flow
feat: v1.11.0 PostgreSQL compatibility and SQL feature expansion (#2090)

* Add structured audit logging with immutable audit trail

Introduces a new --audit-log flag that records all gRPC operations as
structured JSON events in immudb's tamper-proof KV store. Events are
stored under the audit: key prefix in systemdb, queryable via Scan and
verifiable via VerifiableGet. An async buffered writer ensures minimal
latency impact. Configurable event filtering (all/write/admin) via
--audit-log-events flag.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add PostgreSQL ORM compatibility layer and verification functions

Extend the pgsql wire protocol with immudb verification functions
(immudb_state, immudb_verify_row, immudb_verify_tx, immudb_history,
immudb_tx) accessible via standard SQL SELECT statements.

Add pg_catalog resolvers (pg_attribute, pg_index, pg_constraint,
pg_type, pg_settings, pg_description) and information_schema
resolvers (tables, columns, schemata, key_column_usage) to support
ORM introspection from Django, SQLAlchemy, GORM, and ActiveRecord.

Add PostgreSQL compatibility functions: current_database,
current_schema, current_user, format_type, pg_encoding_to_char,
pg_get_expr, pg_get_constraintdef, obj_description, col_description,
has_table_privilege, has_schema_privilege, and others.

Add SHOW statement emulation for common ORM config queries and
schema-qualified name stripping for information_schema and public
schema references.

* Implement EXISTS and IN subquery support in SQL engine

Replace the previously stubbed ExistsBoolExp and InSubQueryExp
implementations with working non-correlated subquery execution.

EXISTS subqueries resolve the inner SELECT and check if any rows
are returned. IN subqueries resolve the inner SELECT, iterate the
result set, and compare each value against the outer expression.
Both support NOT variants (NOT EXISTS, NOT IN).

Correlated subqueries (referencing outer query columns) ar... (continued)

7254 of 10471 new or added lines in 124 files covered. (69.28%)

119 existing lines in 18 files now uncovered.

44597 of 52298 relevant lines covered (85.27%)

127591.66 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

61.94
/embedded/sql/hash_join_reader.go
1
/*
2
Copyright 2026 Codenotary Inc. All rights reserved.
3

4
SPDX-License-Identifier: BUSL-1.1
5
you may not use this file except in compliance with the License.
6
You may obtain a copy of the License at
7

8
    https://mariadb.com/bsl11/
9

10
Unless required by applicable law or agreed to in writing, software
11
distributed under the License is distributed on an "AS IS" BASIS,
12
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
See the License for the specific language governing permissions and
14
limitations under the License.
15
*/
16

17
package sql
18

19
import (
20
        "context"
21
        "fmt"
22
        "strings"
23
)
24

25
// hashJoinTable is an in-memory hash table built by scanning the inner
26
// DataSource exactly once. Rows are grouped by a composite join-key so that
27
// subsequent outer rows can probe in O(1) instead of opening a new inner scan
28
// (which has high per-scan overhead in immudb's B-tree store).
29
//
30
// innerColSels holds the ordered list of inner column selectors that form the
31
// join key. For a simple equi-join (ON a = b) there is one entry; for a
32
// compound equi-join (ON a = b AND c = d) there are two, and the hash key is
33
// the '\x01'-separated concatenation of the individual hashJoinKey values.
34
type hashJoinTable struct {
35
        rows         map[string][]*Row // composite hash key → matching inner rows
36
        innerColSels []string          // inner join column selectors, in key order
37
        cols         []ColDescriptor   // column descriptors of the inner table
38
}
39

40
// compositeHashKey builds a stable string key from one or more TypedValues.
41
// A single-value join uses hashJoinKey directly (no separator overhead).
42
// A multi-value join concatenates individual keys with '\x01' as separator;
43
// '\x01' cannot appear in any hashJoinKey output, so there are no collisions.
44
func compositeHashKey(vals []TypedValue) string {
808✔
45
        if len(vals) == 1 {
1,509✔
46
                return hashJoinKey(vals[0])
701✔
47
        }
701✔
48
        parts := make([]string, len(vals))
107✔
49
        for i, v := range vals {
321✔
50
                parts[i] = hashJoinKey(v)
214✔
51
        }
214✔
52
        return strings.Join(parts, "\x01")
107✔
53
}
54

55
// buildJoinHashTable performs a single full scan of ds, grouping rows by the
56
// composite value of innerColSels. Rows where any join-key column is missing
57
// are skipped.
58
//
59
// innerWhere, when non-nil, is applied as a WHERE clause to the scan. It must
60
// reference only columns of ds (i.e. be inner-only and stable across outer
61
// rows) — used by D7 predicate pushdown to filter inner rows at scan time.
62
//
63
// The DataSource is wrapped in a SelectStmt so that genScanSpecs() produces
64
// valid ScanSpecs (with a concrete Index). Calling tableRef.Resolve(nil)
65
// directly causes ErrIllegalArguments because newRawRowReader requires
66
// non-nil ScanSpecs with a non-nil Index.
67
func buildJoinHashTable(
68
        ctx context.Context,
69
        tx *SQLTx,
70
        ds DataSource,
71
        innerColSels []string,
72
        innerWhere ValueExp,
73
        params map[string]interface{},
74
) (*hashJoinTable, error) {
49✔
75
        fullScan := &SelectStmt{ds: ds, where: innerWhere}
49✔
76
        reader, err := fullScan.Resolve(ctx, tx, params, nil)
49✔
77
        if err != nil {
50✔
78
                return nil, err
1✔
79
        }
1✔
80
        defer reader.Close()
48✔
81

48✔
82
        cols, err := reader.Columns(ctx)
48✔
83
        if err != nil {
48✔
NEW
84
                return nil, err
×
NEW
85
        }
×
86

87
        ht := &hashJoinTable{
48✔
88
                rows:         make(map[string][]*Row),
48✔
89
                innerColSels: innerColSels,
48✔
90
                cols:         cols,
48✔
91
        }
48✔
92

48✔
93
        for {
367✔
94
                row, err := reader.Read(ctx)
319✔
95
                if err == ErrNoMoreRows {
367✔
96
                        break
48✔
97
                }
98
                if err != nil {
271✔
NEW
99
                        return nil, err
×
NEW
100
                }
×
101

102
                vals := make([]TypedValue, 0, len(innerColSels))
271✔
103
                allFound := true
271✔
104
                for _, sel := range innerColSels {
562✔
105
                        val, ok := row.ValuesBySelector[sel]
291✔
106
                        if !ok {
291✔
NEW
107
                                allFound = false
×
NEW
108
                                break
×
109
                        }
110
                        vals = append(vals, val)
291✔
111
                }
112
                if !allFound {
271✔
NEW
113
                        continue
×
114
                }
115

116
                // immudb treats NULL = NULL as equal (NullValue.Compare returns 0 when
117
                // both sides are nil), so we include NULL-keyed rows in the hash table.
118
                key := compositeHashKey(vals)
271✔
119
                ht.rows[key] = append(ht.rows[key], row)
271✔
120
        }
121

122
        return ht, nil
48✔
123
}
124

125
// hashJoinKey returns a string map key for a TypedValue.
126
// The type tag prevents collisions between, e.g., Integer(1) and Bool(true).
127
func hashJoinKey(val TypedValue) string {
915✔
128
        raw := val.RawValue()
915✔
129
        // []byte (Blob) is not fmt-printable in a stable way, so handle it explicitly.
915✔
130
        if b, ok := raw.([]byte); ok {
915✔
NEW
131
                return "blob\x00" + string(b)
×
NEW
132
        }
×
133
        return fmt.Sprintf("%T\x00%v", raw, raw)
915✔
134
}
135

136
// extractEquiJoinPairs detects whether a (reduced) join condition consists
137
// entirely of equi-join predicates connected by AND, each comparing a concrete
138
// outer TypedValue to an inner ColSelector.
139
//
140
// After reduceSelectors, outer columns become concrete TypedValues while inner
141
// columns remain *ColSelectors. Returns the parallel slices (outerVals,
142
// innerSels, true) on success, or (nil, nil, false) if any part of the
143
// condition is not a simple equi-join.
144
//
145
// Examples of what this recognises:
146
//
147
//        ON a.id = b.id          →  single pair
148
//        ON a.id = b.id AND a.x = b.x  →  two pairs (compound equi-join)
NEW
149
func extractEquiJoinPairs(reduced ValueExp) (outerVals []TypedValue, innerSels []string, ok bool) {
×
NEW
150
        switch e := reduced.(type) {
×
NEW
151
        case *CmpBoolExp:
×
NEW
152
                if e.op != EQ {
×
NEW
153
                        return nil, nil, false
×
NEW
154
                }
×
NEW
155
                ov, is, pairOk := extractSingleEquiPair(e)
×
NEW
156
                if !pairOk {
×
NEW
157
                        return nil, nil, false
×
NEW
158
                }
×
NEW
159
                return []TypedValue{ov}, []string{is}, true
×
160

NEW
161
        case *BinBoolExp:
×
NEW
162
                if e.op != And {
×
NEW
163
                        return nil, nil, false
×
NEW
164
                }
×
NEW
165
                leftVals, leftSels, leftOk := extractEquiJoinPairs(e.left)
×
NEW
166
                if !leftOk {
×
NEW
167
                        return nil, nil, false
×
NEW
168
                }
×
NEW
169
                rightVals, rightSels, rightOk := extractEquiJoinPairs(e.right)
×
NEW
170
                if !rightOk {
×
NEW
171
                        return nil, nil, false
×
NEW
172
                }
×
NEW
173
                return append(leftVals, rightVals...), append(leftSels, rightSels...), true
×
174
        }
175

NEW
176
        return nil, nil, false
×
177
}
178

179
// extractEquiJoinPlan splits a (reduced) join condition into equi-pairs and
180
// an inner-only residual filter. innerAlias is the inner table's alias.
181
//
182
// A leaf is classified as:
183
//   - equi-pair: CmpBoolExp(op=EQ) where one side is a TypedValue (reduced
184
//     outer ref) and the other is *ColSelector (inner col).
185
//   - inner-only: contains only ColSelector references whose explicit table
186
//     == innerAlias, plus TypedValues / Params. Stable across outer rows.
187
//   - anything else: returns ok=false (e.g. mixed-table non-equi, outer-only
188
//     post-reduce, subquery, EXISTS).
189
//
190
// Returns (outerVals, innerSels) for the equi-pairs (parallel slices) and
191
// innerResidual as the AND of inner-only conjuncts (or nil). ok=false when
192
// the cond cannot be cleanly classified — caller falls back to slow path.
193
func extractEquiJoinPlan(reduced ValueExp, innerAlias string) (
194
        outerVals []TypedValue,
195
        innerSels []string,
196
        innerResidual ValueExp,
197
        ok bool,
198
) {
592✔
199
        var residuals []ValueExp
592✔
200
        if !extractEquiJoinPlanRec(reduced, innerAlias, &outerVals, &innerSels, &residuals) {
597✔
201
                return nil, nil, nil, false
5✔
202
        }
5✔
203
        if len(outerVals) == 0 {
588✔
204
                return nil, nil, nil, false
1✔
205
        }
1✔
206
        innerResidual = andConjuncts(residuals)
586✔
207
        return outerVals, innerSels, innerResidual, true
586✔
208
}
209

210
func extractEquiJoinPlanRec(
211
        e ValueExp,
212
        innerAlias string,
213
        outerVals *[]TypedValue,
214
        innerSels *[]string,
215
        residuals *[]ValueExp,
216
) bool {
814✔
217
        if be, ok := e.(*BinBoolExp); ok && be.op == And {
925✔
218
                return extractEquiJoinPlanRec(be.left, innerAlias, outerVals, innerSels, residuals) &&
111✔
219
                        extractEquiJoinPlanRec(be.right, innerAlias, outerVals, innerSels, residuals)
111✔
220
        }
111✔
221

222
        if cmp, ok := e.(*CmpBoolExp); ok && cmp.op == EQ {
1,387✔
223
                if ov, is, pairOk := extractSingleEquiPair(cmp); pairOk {
1,366✔
224
                        *outerVals = append(*outerVals, ov)
682✔
225
                        *innerSels = append(*innerSels, is)
682✔
226
                        return true
682✔
227
                }
682✔
228
        }
229

230
        tables, hasUnqualified, safe := collectColTables(e)
21✔
231
        if !safe || hasUnqualified {
21✔
NEW
232
                return false
×
NEW
233
        }
×
234
        if len(tables) == 1 {
37✔
235
                for t := range tables {
32✔
236
                        if t == innerAlias {
32✔
237
                                *residuals = append(*residuals, e)
16✔
238
                                return true
16✔
239
                        }
16✔
240
                }
241
        }
242

243
        return false
5✔
244
}
245

246
// extractSingleEquiPair extracts a single (outerVal, innerSel) pair from a
247
// CmpBoolExp with op == EQ where exactly one side is a concrete TypedValue
248
// (the reduced outer column) and the other is a *ColSelector (inner column).
249
func extractSingleEquiPair(cmp *CmpBoolExp) (outerVal TypedValue, innerSel string, ok bool) {
684✔
250
        if lv, isTV := cmp.left.(TypedValue); isTV {
1,069✔
251
                if rsel, isSel := cmp.right.(*ColSelector); isSel {
768✔
252
                        return lv, rsel.Selector(), true
383✔
253
                }
383✔
254
        }
255
        if rv, isTV := cmp.right.(TypedValue); isTV {
602✔
256
                if lsel, isSel := cmp.left.(*ColSelector); isSel {
600✔
257
                        return rv, lsel.Selector(), true
299✔
258
                }
299✔
259
        }
260
        return nil, "", false
2✔
261
}
262

263
// hashProbeReader is a minimal RowReader that serves pre-matched rows from a
264
// hash table probe. It replaces the per-outer-row inner scan in a hash join,
265
// eliminating the expensive Resolve() + B-tree-seek overhead for each outer row.
266
type hashProbeReader struct {
267
        rows   []*Row
268
        pos    int
269
        cols   []ColDescriptor
270
        tx     *SQLTx
271
        params map[string]interface{}
272
}
273

274
func newHashProbeReader(rows []*Row, cols []ColDescriptor, tx *SQLTx, params map[string]interface{}) *hashProbeReader {
363✔
275
        if rows == nil {
363✔
NEW
276
                rows = []*Row{}
×
NEW
277
        }
×
278
        return &hashProbeReader{rows: rows, cols: cols, tx: tx, params: params}
363✔
279
}
280

281
// hashProbeReader implements RowReader. Most methods are stubs since the reader
282
// only needs to serve pre-fetched rows inside jointRowReader.Read().
NEW
283
func (r *hashProbeReader) onClose(_ func())                   {}
×
NEW
284
func (r *hashProbeReader) Tx() *SQLTx                         { return r.tx }
×
NEW
285
func (r *hashProbeReader) TableAlias() string                 { return "" }
×
NEW
286
func (r *hashProbeReader) OrderBy() []ColDescriptor           { return nil }
×
NEW
287
func (r *hashProbeReader) ScanSpecs() *ScanSpecs              { return &ScanSpecs{} }
×
288
func (r *hashProbeReader) Close() error                       { return nil }
363✔
NEW
289
func (r *hashProbeReader) Parameters() map[string]interface{} { return r.params }
×
290

NEW
291
func (r *hashProbeReader) Columns(_ context.Context) ([]ColDescriptor, error) {
×
NEW
292
        return r.cols, nil
×
NEW
293
}
×
294

NEW
295
func (r *hashProbeReader) colsBySelector(_ context.Context) (map[string]ColDescriptor, error) {
×
NEW
296
        ret := make(map[string]ColDescriptor, len(r.cols))
×
NEW
297
        for _, c := range r.cols {
×
NEW
298
                ret[c.Selector()] = c
×
NEW
299
        }
×
NEW
300
        return ret, nil
×
301
}
302

NEW
303
func (r *hashProbeReader) colsByPos(_ context.Context) ([]ColDescriptor, error) {
×
NEW
304
        return r.cols, nil
×
NEW
305
}
×
306

NEW
307
func (r *hashProbeReader) InferParameters(_ context.Context, _ map[string]SQLValueType) error {
×
NEW
308
        return nil
×
NEW
309
}
×
310

311
func (r *hashProbeReader) Read(_ context.Context) (*Row, error) {
733✔
312
        if r.pos >= len(r.rows) {
1,089✔
313
                return nil, ErrNoMoreRows
356✔
314
        }
356✔
315
        row := r.rows[r.pos]
377✔
316
        r.pos++
377✔
317
        return row, nil
377✔
318
}
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2026 Coveralls, Inc