The File Upload API Mistakes I Made So You Do Not Have To

Five mistakes, five production incidents, and the one-line rule that came out of each.

Every file upload API tutorial shows the happy path. My production incidents were all in the footnotes nobody wrote. The first one was a 400 that appeared only in production, never locally, never in staging, and it was caused by a single header I had helpfully set myself. 

────────────────────────────────────────────────────────────

Upload bugs are a rite of passage. Here are five you can skip.

Key Takeaways

• Setting the multipart Content-Type by hand drops the boundary parameter and produces a 400 nobody can reproduce locally.
• Buffering uploads in memory makes peak usage a function of concurrency, so streaming is the only safe default.
• A file extension is a claim about the bytes; the byte signature is the fact, so validate the bytes.
• Retries without idempotency keys create duplicates on exactly the flakiest connections.
• Office WiFi hides every network failure mode your users will actually hit.

Mistake 1, The Hand-Set Boundary

I set Content-Type to multipart/form-data because that is what the request was, and it seemed wrong to leave a header the server would need unset. That single line produced 400s in production, on the same code that worked on my laptop. The multipart/form-data specification explains why. The header is not just a type; it carries a boundary parameter that names the delimiter separating the parts, and that delimiter is generated per request. Set the header yourself and the boundary goes missing, so the server sees a body it cannot split.

It only surfaced in production because a proxy in that environment normalised the request in a way my local setup did not, which is the worst possible way to learn this, and I have since met four other people who learned it the same way.

// Wrong. The boundary parameter is lost, and the body cannot be parsed.

await fetch('/api/files', {

method: 'POST',

headers: { 'Content-Type': 'multipart/form-data' },

body: form,

});

// Right. Let the client generate the header, boundary and all.

const form = new FormData();

form.append('file', file);

await fetch('/api/files', { method: 'POST', body: form });

Ask how to implement a file upload feature using a REST API and every answer will show you the second version. None of them will tell you why the first version fails, which is why it keeps being written.

Mistake 2, The Memory Buffer

Read the request body, hold it, write it to storage. It is the obvious shape and it works perfectly until two people upload at the same time. Peak memory is concurrency multiplied by file size, and concurrency is not a number you control. Ours was a 2 GB container, our files were around 250 MB, and the arithmetic took about eight simultaneous uploads to become an out-of-memory kill.

// Wrong. The whole file lands in memory before anything is written.

const body = await readEntireBody(req);

await storage.put(key, body);

// Right. Pipe it through. Memory stays flat whatever the file size.

await pipeline(req, storage.createWriteStream(key));

Understanding how multipart upload works in a web application helps here, because once you see the body as a stream of parts rather than a file that arrives all at once, buffering stops looking like the obvious shape and starts looking like a choice you have to justify.

Mistake 3, Trusting Extensions

We validated uploads by extension. Someone renamed an executable to .pdf, our check passed it, and we stored and later served it as a document. Nothing bad came of it, and that was luck rather than design. An extension is a claim made by whoever named the file. The byte signature at the head of the file is a fact about its contents, and the two agree only when nobody has a reason to make them disagree.

The rule is one line: read the leading bytes, match them against the types you accept, and treat the filename as a label for humans rather than as evidence. Reject anything whose signature is absent from your allow-list, rather than trying to enumerate what to block.

Mistake 4, Non-Idempotent Retries

Our client retried failed uploads, which is correct. Our server treated every request as new, which is also correct in isolation. Together they produced duplicate files, and they produced them worst on the connections that dropped most, so our least fortunate users got the messiest libraries.

The fix is an idempotency key generated once per upload attempt and sent with every retry of that attempt. The server records the key with the stored object and returns the existing result when it sees the key again. This is a few lines on each side, and it turns retry from something that might duplicate work into something that cannot.

Mistake 5, The WiFi-Only Test Matrix

Every test we wrote ran on an office connection that never dropped, never throttled, and never handed the socket from one network to another. So every failure mode our users actually had went unobserved until the feature was live, at which point it arrived as support tickets rather than as test failures.

Throttling and interruption belong in the matrix beside the happy path. Cut the bandwidth to a tenth. Kill the connection mid-transfer and restore it. Background the tab or the app. None of it is exotic tooling, and all of it is closer to what a user on a train experiences than anything a green build was telling us.

The first afternoon we ran the throttled version, it failed in three places we had never seen: a progress bar that stalled without timing out, a retry that fired before the first attempt had properly given up, and an error message that said the file was invalid when the connection was the problem. All three had been live for months. None of them had ever produced a failing test, because no test had ever been slow.

Which Mistakes a Platform Absorbs

Half this list is structural on a managed file upload API; the other half is still on you. That split is worth being precise about, because it is the useful part of this whole confession.

Buffering and retry semantics are the two that stop being possible. When transfer is chunked and handled outside your process, there is no request body sitting in your heap waiting to be written, so the concurrency arithmetic that killed our container never runs. When retry happens at the chunk level with the service tracking what it has accepted, a duplicate file is not a thing your code can accidentally create.

The other three stay yours on any stack ever built. What your product accepts is a product decision, so byte validation is your job. Which network conditions you test under is your discipline, and no service can impose it. And the multipart header belongs to whichever client you write, forever.

Five incidents, five rules, and which of them somebody else can take off your hands.

Alt text for web publication: “Five file upload api mistakes and the incidents they caused.”

Read the Footnotes Nobody Wrote

Five rules, compiled the expensive way. Let the client set the multipart header. Stream rather than buffer. Validate bytes, not names. Give every upload attempt an idempotency key. Put a bad network in the test matrix.

What strikes me looking back is that not one of these was difficult. Each fix was small, each rule fits on a line, and each incident happened because the tutorial I learned from stopped at the point where the happy path worked. That is the real category error. A working upload and a reliable upload are separated almost entirely by things nobody bothers to write down.

Frequently Asked Questions

Why does my multipart upload return 400?

Most often because Content-Type was set by hand, which drops the boundary parameter the parser needs. Remove the header and let the client generate it with the boundary included.

Should uploads be validated by extension?

Extensions are claims and byte signatures are facts. Read the leading bytes, match them against an allow-list of accepted types, and treat the filename as a label rather than as evidence.

How do you prevent duplicate uploads on retry?

An idempotency key generated once per upload attempt and sent with every retry of it. The server stores the key alongside the object and returns the existing result rather than creating a second one.

Which of these does a managed service handle for you?

Buffering and retry semantics stop being possible once transfer is chunked and tracked outside your process. Byte validation, network testing and the multipart header on your own client remain yours.