zentralopensource / zentral / 39612962903

91%

push

github

turbo: multipart, and the server closes it

A large artifact goes up in parts. The mint answers mode: "multipart" with the
geometry and one signed URL per part — the geometry comes back WITH the URLs,
which is the whole reason it is not published or cached anywhere. The agent sends
the same body either way and reads the mode off the answer, so there is nothing
for it to predict, no part size to cache, and no stale-configuration two-step.

Multipart is offered only where file_storage_has_multipart_uploads() says so,
which is S3. Elsewhere every upload is a single PUT, under the ceiling of the
deployment.

The threshold is TWO part sizes. At one there is no resumability to buy: a
part_size + 1 artifact is a two-part upload whose second part is one byte, so
losing the first part costs the retransmission that restarting a single PUT
would have. The whole multipart bill would be paid to protect a one-byte tail.
At two, the first failure loses at most half the transfer.

A multipart artifact is refused without crc64nvme. sha256 is composite-only on a
multipart upload, so the whole-object CRC is the only thing that can validate the
assembled object; without it completion would accept whatever arrived. Nothing
uses the CRC on a single PUT, and a single PUT is not refused for want of it.

Starting the upload is a storage call on a device endpoint, so it answers like
one: a StorageError is a 503 storage_unavailable, not the HTML page
Django renders for an unhandled exception on the one endpoint contracted never to
serve one. ATOMIC_REQUESTS would also roll the row back while the storage may
already hold the upload id — an orphan for the lifecycle rule. The agent mints
again, which is the right thing to do about a storage that was busy.

Completion is the server's. The agent posts the mint's run block and the
artifact name, and gets a 202; the server lists the parts from the storage — never from
the wire — and completes with its own credentials. That is the one ... (continued)

228 of 228 new or added lines in 8 files covered. (100.0%)

54271 of 59516 relevant lines covered (91.19%)

0.91 hits per line