swanlake-io / swanlake / 22250770620

72%

Build Type

push

github

Committed by web-flow

Commit Message

tpch on s3 and r2 (#92)

* Add TPCH BenchBase workflow and generalize YCSB runner

- Add .github/workflows/performance.yml for TPCH performance runs
- Rename scripts/run-benchbase-ycsb.sh to run-benchbase.sh, supporting BENCHMARK param
- Add tests/benchbase/tpch-flight-sql.xml config for TPCH
- Update CI to use new runner and support coverage toggling
- Refactor artifact/cache

* Run performance workflow on push and pull_request events

* Fix performance workflow trigger and make DuckDB env sourcing optional

The performance workflow now only triggers on pull requests. The benchbase
script no longer fails if the DuckDB env file is missing, improving
robustness for local and CI runs.

* Set CARGO_TARGET_DIR in CI to ensure consistent build paths

* Add TPCH DDL and debug logging for BenchBase performance runs

- Add `tests/benchbase/tpch-ddl-ducklake.sql` for TPCH schema setup
- Add debug logging configs for JUL and Log4j
- Parameterize DDL path and logging in `run-benchbase.sh`
- Update workflow to cache and upload TPCH DDL and logs
- Set `reconnectOnConnectionFailure` to false in TPCH config
- Support debug logging and DDL path injection for TPCH runs

* Add BenchBase concurrency and timing controls to perf workflow

- Expose terminals, warmup, and benchmark time as workflow inputs
- Update run-benchbase.sh to override these in config XML
- Set tpch-flight-sql.xml defaults: 4 terminals, 30s warmup, 180s run
- Attach S3 secret in DuckLake init SQL for S3-backed runs

* Tune BenchBase TPCH scale factor handling and patch Q15

- Allow per-storage TPCH scale_factor override in workflow matrix
- Default scale_factor input is now empty (uses storage default)
- Patch Q15 to use TEMP VIEW and DROP VIEW IF EXISTS for dialect compat
- Update TPCH dialect patch version to v7 in run-benchbase.sh

* minor change

* c

* concurrent issue

* fix race

* tune performance

* add more scripts

Run Details

84 of 151 new or added lines in 7 files covered. (55.63%)

85 existing lines in 2 files now uncovered.

2846 of 3939 relevant lines covered (72.25%)

344.77 hits per line