How to Test Large File Uploads: 1GB Files, Boundaries, and Resumable Transfers

Small files make an upload feature look finished. A 200 KB photo hides everything that actually breaks in production: gateway timeouts, half-written files, disk limits, retry logic, and size checks that fall over past 2 GB. This guide covers generating exact-size test files on every OS, the boundary values worth automating, and how to exercise resumable uploads.

Generate Exact-Size Files Anywhere

Choose deliberately, because the tools are not interchangeable:

• Sparse files (truncate, mkfile -n) prove your size logic — but anything that hashes or scans the file still processes a full gigabyte, so they are free to create, not free to use.
• Zeros vs random matters. If anything in the path compresses or deduplicates, a zero file flies through and your "1 GB upload" finishes suspiciously fast. Random bytes cannot be compressed, so throughput numbers stay honest. An already-compressed format like an MP4 sample behaves the same way and doubles as a real-world check.
• Exact size with a real header: append random bytes to a real PDF sample until it hits your target — cat pad.bin >> sample.pdf — and you can test signature validation on large files too. For raw payloads, a binary sample padded the same way works fine.

Test Boundaries, Not Just "Big"

If your limit is 10 MiB (10,485,760 bytes), three files tell you more than one giant one:

For sub-10 MB cases, a file generator saves you the arithmetic.

Then automate the 32-bit cliff:

Parsers, older SQL INT columns, and legacy code paths that store sizes in 32-bit integers silently wrap 2 GiB files into negative values, and the failure mode is never pretty: absurd buffer allocations or universal rejection.

Know Every Limit in the Chain

• nginx: client_max_body_size defaults to 1 MB — anything bigger gets 413 Request Entity Too Large before your app sees a byte
• Express / body-parser: the limit option defaults to '100kb'
• S3 multipart: parts must be at least 5 MiB (except the last), max 10,000 parts, max object 5 TiB — a 1 TiB upload forces roughly 105 MB parts

Map the whole path — browser, CDN/WAF, reverse proxy, app, storage — because each layer rejects at its own threshold and users only see whichever error surfaces first.

Test Resumable and Chunked Uploads

Large uploads fail mid-transfer constantly: mobile networks drop, laptops sleep. Chunked PUT with Content-Range is the classic shape:

# Send the first 1 MiB (bytes 0-1048575) of a 3 MiB upload
curl -X PUT \
  -H "Content-Range: bytes 0-1048575/3145728" \
  -H "Content-Type: application/octet-stream" \
  --data-binary @chunk0.bin \
  https://example.com/upload/session-123/big.bin

Beyond the happy path, assert the ugly cases:

• Out-of-order chunks — does the server accept byte ranges arriving in any order?
• Duplicate chunk — re-sending a range must be idempotent, not append twice
• Overlapping ranges — the server should reject or truncate, never concatenate

Abort mid-transfer

Start a 1 GB upload, kill it around halfway (Ctrl+C or kill -9), then:

1. Resume from the offset the server reports — is the byte count exact?
2. Compare hashes end to end: sha256sum big.bin on Linux, shasum -a 256 big.bin on macOS, certutil -hashfile big.bin SHA256 on Windows
3. Check orphan cleanup — aborted uploads should not live on disk forever

The same matrix applies to the tus resumable-upload protocol and S3 multipart: abort, resume, duplicate part, out-of-order part.

Watch for Expect: 100-continue

For request bodies larger than about 1 MB, curl sends Expect: 100-continue and waits — one second by default — for the server's blessing before transmitting. A proxy that swallows the interim response adds a flat 1s delay to every large upload. Test with --expect100-timeout 10s, and with the handshake disabled via -H 'Expect:', to see both behaviors.

A Minimal Test Matrix

Zeros for size logic, random data for throughput, real formats for correctness, boundaries for the off-by-ones, and aborts for the code nobody tests until production. That's the whole game.