jallum / flatbuffer / 3eccf87e8d8b414e62bf539342fe0173dadbe74c

100%

push

github

Batch inline vector writes (#11)

## Why

The writer treats inline vectors (numbers, enums, and structs) like
vectors of tables or strings. It wraps every element, saves the wrappers
in reverse order, and rebuilds the writer state once per element. Inline
values do not need that offset bookkeeping because their bytes are
already final.

## What changed

- Build inline vector elements in their original order and add the whole
iolist to the writer state once.
- Keep reference vectors on the existing offset-first path.
- Remove the now-unneeded per-element tags from reference-vector
bookkeeping.
- Add a Benchee workload for scalar and struct vectors at 10, 100, and
1,000 elements.

## How it works (ELI5)

The old code boxed every item, stacked the boxes backward, then unpacked
them one by one. Inline values can be put in one ordered tray and handed
to the writer together. Tables and strings still use the careful
one-by-one offset path because they need directions to where their data
lives.

## Benchee results

Apple M4 Pro, Elixir 1.19.4, Erlang/OTP 28.

| API / vector size | Before | After | Change | Memory before | Memory
after |
|---|---:|---:|---:|---:|---:|
| `to_binary`, 100 | 82.03 K ops/s | 107.32 K ops/s | 1.31x faster |
72.42 KB | 47.60 KB |
| `to_binary`, 1,000 | 5.41 K ops/s | 9.26 K ops/s | 1.71x faster |
705.26 KB | 455.44 KB |
| `to_iolist`, 100 | 96.87 K ops/s | 139.33 K ops/s | 1.44x faster |
72.41 KB | 47.54 KB |
| `to_iolist`, 1,000 | 6.38 K ops/s | 11.34 K ops/s | 1.78x faster |
705.20 KB | 455.35 KB |

I also tested flattening the batched payload to a binary inside the
writer. It was effectively tied for `to_binary`, used slightly more
memory, and forced `to_iolist` to copy, so this PR keeps the payload as
iodata.

Run with:

```sh
mix run bench/writer.exs
```

## Verification

- `mix format --check-formatted`
- `MIX_ENV=test mix test --include flatc` (113 tests, 0 failures)

19 of 19 new or added lines in 1 file covered. (100.0%)

439 of 446 relevant lines covered (98.43%)

51.98 hits per line