databendlabs / openraft / 20222136904

push

github

feat: add pipeline mode for streaming replication

Implement pipeline replication where the leader continuously streams log
entries to followers after finding the match point, replacing the
request-response pattern with efficient pipelined I/O.

## Replication Phases

Replication to a follower has two phases:

1. **Binary search phase**: The leader runs a binary search to find the exact
   matching position of log entries on the follower.

2. **Pipeline mode**: After finding the match point, the leader calls the
   `stream_append` method on the network and continuously generates
   AppendEntries requests. The network implementation should pipeline all
   requests to the follower and yield responses. Note that responses and
   requests don't have to be 1-to-1 mapped - the number of responses can be
   smaller than the number of requests.

`stream_append` provides a default implementation that calls the existing
`append_entries` method to emulate streaming replication. A mature
implementation should run in real pipeline mode instead of request-response
manner. When a request is received by `stream_append`, it is responsible for
sending all content of the request to the follower - partial success is not
allowed.

## I/O Progress Synchronization

Add watch channels for replication tasks to synchronize with leader I/O
progress:

- `io_accepted_tx`: Notifies observers before I/O operations are submitted to
  storage, enabling preparation for upcoming I/O events
- `io_submitted_tx`: Notifies replication tasks when log entries have been
  submitted to storage and are safe to read

The replication stream monitors these channels to detect leader changes and
wait for log availability without polling.

## Replication Stream Reuse

When rebuilding replication streams after a membership change, reuse existing
streams instead of destroying all and recreating. This avoids unnecessary
stream teardown and maintains in-flight replication state.

- Add `close_old_streams... (continued)

512 of 536 new or added lines in 29 files covered. (95.52%)

14664 of 15817 relevant lines covered (92.71%)

154361.45 hits per line