Alan-Jowett / ebpf-verifier / 29279923487
87%

DEFAULT BRANCH: main
Ran
Jobs 2
Files 79
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 86.649% (+0.3%) from 86.386%
29279923487

push

github

web-flow 
Drive sample verification tests from the inventory; parallelize CI (#1195)

Full-program verification of the ebpf-samples corpus was 17 C++ files
generated by scripts/generate_verify_project_tests.py from
test-data/elf_inventory.json, then compiled into the test binary. The
generated files duplicated the inventory and could drift from it (there
was no check keeping them in sync), and the whole corpus ran in one
single-threaded Catch2 process.

Data-driven tests (dedup)
-------------------------
- Add src/test/test_verify_samples.cpp, which reads elf_inventory.json at
  runtime (via yaml-cpp; JSON is a subset of YAML) and registers one
  Catch2 DYNAMIC_SECTION per (object, section[, program]). Adding or
  retagging a sample is now a JSON edit: no code generation, no recompile.
- Delete the 17 generated test_verify_<project>.cpp files and
  scripts/generate_verify_project_tests.py. The inventory is the single
  source of truth. A coverage-guard test asserts the registered projects
  match the inventory exactly, so a new project cannot be left untested.
- Trim test_verify.hpp to the ELF-parse cache and the VerifyIssueKind
  taxonomy the driver and the multithreading test still use.

  Catch2's [!shouldfail] is a compile-time per-TEST_CASE tag and cannot be
  applied per data-driven entry, so an expected_failure (a safe program
  the verifier is currently too imprecise to accept) is asserted as
  *still rejected* -- an xfail/golden marker. A verifier improvement that
  starts accepting it fails the case, prompting an inventory update; a
  regression on a passing program fails it too.

Parallel CI (ctest)
-------------------
- Register ctest entries: one per project (sharded by tag) plus a "unit"
  entry for everything else. The shard list is derived from the inventory
  via string(JSON), so it cannot drift from the corpus.
- CI runs `ctest -j` instead of a single `bin/tests` process. Locally, a
  full run drops from ~104s to ~42s on 16 cores (bounded by t... (continued)

9313 of 10748 relevant lines covered (86.65%)

6391896.2 hits per line

Coverage Regressions

Lines Coverage ∆ File
57
91.77
 -0.1% src/crab/ebpf_transformer.cpp
43
84.43
 -0.02% src/io/elf_reader.cpp
41
78.43
 2.56% src/crab/interval.cpp
33
89.29
 0.09% src/crab/ebpf_checker.cpp
28
91.53
 1.23% src/crab/type_domain.cpp
19
84.62
 0.0% src/arith/num_big.hpp
18
88.65
 0.11% src/ir/cfg_builder.cpp
12
85.34
 0.0% src/crab/array_domain.cpp
9
88.97
 -2.7% src/cfg/wto.cpp
8
90.99
 0.0% src/ir/assertions.cpp
5
92.0
 1.38% src/arith/num_extended.hpp
5
95.86
 0.09% src/crab/splitdbm/adapt_sgraph.hpp
3
91.67
 3.43% src/cfg/wto.hpp
2
87.06
 0.31% src/crab/type_to_num.cpp
1
93.18
 0.16% src/crab/type_to_num.hpp
1
91.06
 -0.19% src/crab/zone_domain.cpp
1
75.0
 -25.0% src/spec/type_descriptors.hpp
Jobs
ID Job ID Ran Files Coverage
1 run-Debug - 29279923487.1 77
84.91
 GitHub Action Run
2 run-Release - 29279923487.2 79
85.76
 GitHub Action Run
Source Files on build 29279923487