24841571249

Committed 23 Apr 2026 02:44PM UTC coverage: 85.275% (-4.0%) from 89.306%

Build # 24841571249

Build Type

push

gh-ci

Committed by

web-flow

Commit Message

feat: v1.11.0 PostgreSQL compatibility and SQL feature expansion (#2090)

* Add structured audit logging with immutable audit trail

Introduces a new --audit-log flag that records all gRPC operations as
structured JSON events in immudb's tamper-proof KV store. Events are
stored under the audit: key prefix in systemdb, queryable via Scan and
verifiable via VerifiableGet. An async buffered writer ensures minimal
latency impact. Configurable event filtering (all/write/admin) via
--audit-log-events flag.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add PostgreSQL ORM compatibility layer and verification functions

Extend the pgsql wire protocol with immudb verification functions
(immudb_state, immudb_verify_row, immudb_verify_tx, immudb_history,
immudb_tx) accessible via standard SQL SELECT statements.

Add pg_catalog resolvers (pg_attribute, pg_index, pg_constraint,
pg_type, pg_settings, pg_description) and information_schema
resolvers (tables, columns, schemata, key_column_usage) to support
ORM introspection from Django, SQLAlchemy, GORM, and ActiveRecord.

Add PostgreSQL compatibility functions: current_database,
current_schema, current_user, format_type, pg_encoding_to_char,
pg_get_expr, pg_get_constraintdef, obj_description, col_description,
has_table_privilege, has_schema_privilege, and others.

Add SHOW statement emulation for common ORM config queries and
schema-qualified name stripping for information_schema and public
schema references.

* Implement EXISTS and IN subquery support in SQL engine

Replace the previously stubbed ExistsBoolExp and InSubQueryExp
implementations with working non-correlated subquery execution.

EXISTS subqueries resolve the inner SELECT and check if any rows
are returned. IN subqueries resolve the inner SELECT, iterate the
result set, and compare each value against the outer expression.
Both support NOT variants (NOT EXISTS, NOT IN).

Correlated subqueries (referencing outer query columns) ar... (continued)

Coverage Stats

7254 of 10471 new or added lines in 124 files covered. (69.28%)

119 existing lines in 18 files now uncovered.

44597 of 52298 relevant lines covered (85.27%)

127591.66 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

84.62

/embedded/appendable/multiapp/appendable_cache.go

/*
Copyright 2026 Codenotary Inc. All rights reserved.

SPDX-License-Identifier: BUSL-1.1
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    https://mariadb.com/bsl11/

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package multiapp

import (
        "sync"

        "github.com/codenotary/immudb/embedded/appendable"
        "github.com/codenotary/immudb/embedded/cache"
)

// refCountedApp wraps an appendable so that cache eviction's Close()
// is *deferred* until no goroutine still holds the handle. Without
// this, a reader returned by appendableCache.Get() can race with a
// concurrent Put() that evicts and Close()s its handle, surfacing as
// `singleapp: already closed` mid-read. Refcounting makes that race
// impossible: Get/Pop return a handle with an extra ref; the read
// path must Release() when done; Close marks evicted and only
// performs the underlying Close when refs reach zero.
//
// Implements appendable.Appendable by delegating, so callers see no
// interface change beyond the new Release().
type refCountedApp struct {
        appendable.Appendable

        mu      sync.Mutex
        refs    int
        evicted bool
}

// Acquire bumps the refcount. Caller must Release exactly once.
// Returns false if the underlying app has already been closed (i.e.
// evicted with refs==0 since this handle was obtained); the handle
// is unusable in that case.
func (r *refCountedApp) Acquire() bool {
        r.mu.Lock()
        defer r.mu.Unlock()
        if r.evicted && r.refs == 0 {
                return false
        }
        r.refs++
        return true
}

// Release drops one ref. If the cache previously asked us to close
// (Close called by eviction) and we're now the last holder, perform
// the underlying Close.
func (r *refCountedApp) Release() error {
        r.mu.Lock()
        r.refs--
        doClose := r.evicted && r.refs == 0
        r.mu.Unlock()
        if doClose {
                return r.Appendable.Close()
        }
        return nil
}

// Close is what the cache's eviction path calls. We don't actually
// close the file here unless no reader still holds the handle —
// otherwise we mark it evicted and the last Release will perform the
// real close. On a double-Close we forward to the underlying so the
// caller sees the underlying ErrAlreadyClosed (preserves the
// behaviour expected by TestMultiAppClosedAndDeletedFiles etc).
func (r *refCountedApp) Close() error {
        r.mu.Lock()
        if r.evicted {
                r.mu.Unlock()
                return r.Appendable.Close()
        }
        r.evicted = true
        doClose := r.refs == 0
        r.mu.Unlock()
        if doClose {
                return r.Appendable.Close()
        }
        return nil
}

type appendableCache struct {
        cache *cache.Cache
}

// Put stores value under key. Returns the (possibly nil) evicted
// entry. The evicted entry is a *refCountedApp; the caller closes it
// (which is now eviction-safe via the refcount).
func (c appendableCache) Put(key int64, value appendable.Appendable) (int64, appendable.Appendable, error) {
        wrapped := &refCountedApp{Appendable: value}
        k, v, err := c.cache.Put(key, wrapped)
        rkey, _ := k.(int64)
        rvalue, _ := v.(appendable.Appendable)
        return rkey, rvalue, err
}

// Get returns the cached entry with one extra ref held. Caller MUST
// type-assert to *refCountedApp and call .Release() when done.
// Returns ErrKeyNotFound (via the underlying cache) on miss.
func (c appendableCache) Get(key int64) (appendable.Appendable, error) {
        v, err := c.cache.Get(key)
        if err != nil {
                return nil, err
        }
        r := v.(*refCountedApp)
        if !r.Acquire() {
                // race: between cache.Get and Acquire the entry was evicted
                // and its underlying file already closed. Return as miss so
                // the caller refetches.
                return nil, cache.ErrKeyNotFound
        }
        return r, nil
}

// Pop removes the entry from the cache and returns it. Treated as a
// transfer of ownership: caller controls the (still ref-counted)
// handle and is responsible for closing it.
func (c appendableCache) Pop(key int64) (appendable.Appendable, error) {
        v, err := c.cache.Pop(key)
        rvalue, _ := v.(appendable.Appendable)
        return rvalue, err
}

// Replace swaps the entry for a new one. The old entry's refcount
// continues to govern when its file is actually closed.
func (c appendableCache) Replace(key int64, value appendable.Appendable) (appendable.Appendable, error) {
        wrapped := &refCountedApp{Appendable: value}
        v, err := c.cache.Replace(key, wrapped)
        rvalue, _ := v.(appendable.Appendable)
        return rvalue, err
}

// Apply iterates entries. The callback sees the underlying
// appendable.Appendable (via the wrapper). No refs are taken — the
// callback runs while the cache holds the entry, which is safe for
// short-lived inspection (callers like Sync/Close that already hold
// the multiapp mutex prevent concurrent eviction).
func (c appendableCache) Apply(fun func(k int64, v appendable.Appendable) error) error {
        return c.cache.Apply(func(k, v interface{}) error {
                return fun(k.(int64), v.(appendable.Appendable))
        })
}

1	/*
2	Copyright 2026 Codenotary Inc. All rights reserved.
3
4	SPDX-License-Identifier: BUSL-1.1
5	you may not use this file except in compliance with the License.
6	You may obtain a copy of the License at
7
8	https://mariadb.com/bsl11/
9
10	Unless required by applicable law or agreed to in writing, software
11	distributed under the License is distributed on an "AS IS" BASIS,
12	WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13	See the License for the specific language governing permissions and
14	limitations under the License.
15	*/
16
17	package multiapp
18
19	import (
20	"sync"
21
22	"github.com/codenotary/immudb/embedded/appendable"
23	"github.com/codenotary/immudb/embedded/cache"
24	)
25
26	// refCountedApp wraps an appendable so that cache eviction's Close()
27	// is deferred until no goroutine still holds the handle. Without
28	// this, a reader returned by appendableCache.Get() can race with a
29	// concurrent Put() that evicts and Close()s its handle, surfacing as
30	// `singleapp: already closed` mid-read. Refcounting makes that race
31	// impossible: Get/Pop return a handle with an extra ref; the read
32	// path must Release() when done; Close marks evicted and only
33	// performs the underlying Close when refs reach zero.
34	//
35	// Implements appendable.Appendable by delegating, so callers see no
36	// interface change beyond the new Release().
37	type refCountedApp struct {
38	appendable.Appendable
39
40	mu sync.Mutex
41	refs int
42	evicted bool
43	}
44
45	// Acquire bumps the refcount. Caller must Release exactly once.
46	// Returns false if the underlying app has already been closed (i.e.
47	// evicted with refs==0 since this handle was obtained); the handle
48	// is unusable in that case.
49	func (r *refCountedApp) Acquire() bool {	26,017✔
50	r.mu.Lock()	26,017✔
51	defer r.mu.Unlock()	26,017✔
52	if r.evicted && r.refs == 0 {	26,017✔
NEW 53	return false	×
NEW 54	}	×
55	r.refs++	26,017✔
56	return true	26,017✔
57	}
58
59	// Release drops one ref. If the cache previously asked us to close
60	// (Close called by eviction) and we're now the last holder, perform
61	// the underlying Close.
62	func (r *refCountedApp) Release() error {	26,016✔
63	r.mu.Lock()	26,016✔
64	r.refs--	26,016✔
65	doClose := r.evicted && r.refs == 0	26,016✔
66	r.mu.Unlock()	26,016✔
67	if doClose {	26,016✔
NEW 68	return r.Appendable.Close()	×
NEW 69	}	×
70	return nil	26,016✔
71	}
72
73	// Close is what the cache's eviction path calls. We don't actually
74	// close the file here unless no reader still holds the handle —
75	// otherwise we mark it evicted and the last Release will perform the
76	// real close. On a double-Close we forward to the underlying so the
77	// caller sees the underlying ErrAlreadyClosed (preserves the
78	// behaviour expected by TestMultiAppClosedAndDeletedFiles etc).
79	func (r *refCountedApp) Close() error {	5,785✔
80	r.mu.Lock()	5,785✔
81	if r.evicted {	5,793✔
82	r.mu.Unlock()	8✔
83	return r.Appendable.Close()	8✔
84	}	8✔
85	r.evicted = true	5,777✔
86	doClose := r.refs == 0	5,777✔
87	r.mu.Unlock()	5,777✔
88	if doClose {	11,554✔
89	return r.Appendable.Close()	5,777✔
90	}	5,777✔
NEW 91	return nil	×
92	}
93
94	type appendableCache struct {
95	cache *cache.Cache
96	}
97
98	// Put stores value under key. Returns the (possibly nil) evicted
99	// entry. The evicted entry is a *refCountedApp; the caller closes it
100	// (which is now eviction-safe via the refcount).
101	func (c appendableCache) Put(key int64, value appendable.Appendable) (int64, appendable.Appendable, error) {	5,770✔
102	wrapped := &refCountedApp{Appendable: value}	5,770✔
103	k, v, err := c.cache.Put(key, wrapped)	5,770✔
104	rkey, _ := k.(int64)	5,770✔
105	rvalue, _ := v.(appendable.Appendable)	5,770✔
106	return rkey, rvalue, err	5,770✔
107	}	5,770✔
108
109	// Get returns the cached entry with one extra ref held. Caller MUST
110	// type-assert to *refCountedApp and call .Release() when done.
111	// Returns ErrKeyNotFound (via the underlying cache) on miss.
112	func (c appendableCache) Get(key int64) (appendable.Appendable, error) {	27,189✔
113	v, err := c.cache.Get(key)	27,189✔
114	if err != nil {	28,361✔
115	return nil, err	1,172✔
116	}	1,172✔
117	r := v.(*refCountedApp)	26,017✔
118	if !r.Acquire() {	26,017✔
NEW 119	// race: between cache.Get and Acquire the entry was evicted	×
NEW 120	// and its underlying file already closed. Return as miss so	×
NEW 121	// the caller refetches.	×
NEW 122	return nil, cache.ErrKeyNotFound	×
NEW 123	}	×
124	return r, nil	26,017✔
125	}
126
127	// Pop removes the entry from the cache and returns it. Treated as a
128	// transfer of ownership: caller controls the (still ref-counted)
129	// handle and is responsible for closing it.
130	func (c appendableCache) Pop(key int64) (appendable.Appendable, error) {	249✔
131	v, err := c.cache.Pop(key)	249✔
132	rvalue, _ := v.(appendable.Appendable)	249✔
133	return rvalue, err	249✔
134	}	249✔
135
136	// Replace swaps the entry for a new one. The old entry's refcount
137	// continues to govern when its file is actually closed.
138	func (c appendableCache) Replace(key int64, value appendable.Appendable) (appendable.Appendable, error) {	114✔
139	wrapped := &refCountedApp{Appendable: value}	114✔
140	v, err := c.cache.Replace(key, wrapped)	114✔
141	rvalue, _ := v.(appendable.Appendable)	114✔
142	return rvalue, err	114✔
143	}	114✔
144
145	// Apply iterates entries. The callback sees the underlying
146	// appendable.Appendable (via the wrapper). No refs are taken — the
147	// callback runs while the cache holds the entry, which is safe for
148	// short-lived inspection (callers like Sync/Close that already hold
149	// the multiapp mutex prevent concurrent eviction).
150	func (c appendableCache) Apply(fun func(k int64, v appendable.Appendable) error) error {	18,915✔
151	return c.cache.Apply(func(k, v interface{}) error {	19,556✔
152	return fun(k.(int64), v.(appendable.Appendable))	641✔
153	})	641✔
154	}

codenotary / immudb / 24841571249

Source File Press 'n' to go to next uncovered line, 'b' for previous

Source File
Press 'n' to go to next uncovered line, 'b' for previous