What Actually Happens When You Upload a File

It’s 8:58 AM, you're presenting at the sprint review, and you drag the final slide deck into the shared folder just as the clock turns. You watch the progress bar hit 100% and join the call, never doubting that the file actually made it through.

But underneath that simple UI, a distributed system is doing the work: chunking algorithms, parallel transfers, integrity checks, and state recovery. Building a 50 GB uploader meant understanding each of these pieces. Here's what they do, and why they exist.

Why Simple Uploads Fail#

What if that slide deck went up as a simple upload, the whole file in a single request? The connection would need to hold until the last byte arrived. If it dropped just before the bar filled, the server would have nothing to resume from, and the upload would start over from zero. And a large file would also have to fit within what one request accepts.

Breaking Files into Chunks#

The solution is to treat large files as a series of independent chunks. On APIs like S3, each part can be uploaded and retried without affecting the others. Each platform sets its own chunk-size rules, and even its own name for chunks: S3 and B2 call them parts; Dropbox and Azure call them blocks.

• Google Drive: Chunks must be multiples of 256 KiB (except the last one).
• AWS S3: Enforces a 5 MiB minimum for every part except the last, and allows up to 10,000 parts.
• Dropbox: Internally splits files into 4 MB blocks, each identified by its SHA-256 hash.
• Azure Blob Storage: Supports blocks up to 4000 MiB, with up to 50,000 blocks per blob.
• Backblaze B2: Enforces a 5 MB minimum, but recommends 100 MB parts to balance throughput and parallelism.

On S3, 10,000 parts at the 5 MiB minimum add up to about 48.8 GiB, so bigger files need bigger parts.

Resumable Uploads and Session IDs#

Chunking alone doesn't make an upload resumable. The server needs to remember your upload across requests, browser refreshes, and network changes. That's what a Session ID is for: a unique identifier for your upload in progress. Think of it as a bookmark: when you reconnect, you present this ID, and the server knows exactly where you left off. Google's API returns it as a full URL in the Location header: the session URI that every request below uses.

1. INITIATE SESSION (Create state)
   POST /upload/drive/v3/files
     ?uploadType=resumable
   Response: 200 OK
   Location: {sessionUri}
     = https://www.googleapis.com/upload/
       drive/v3/files?uploadType=resumable
       &upload_id=xa298sd_sdlkj2

2. UPLOAD CHUNKS
   PUT {sessionUri}
   Content-Range: bytes 0-524287/2097152
   Response: 308 Resume Incomplete

3. CONNECTION DROPS
   (Client switches network or restarts)

4. QUERY PROGRESS (Query server state)
   PUT {sessionUri}
   Content-Range: bytes */2097152
   Response: 308, Range: bytes=0-524287
   (404 Not Found: session expired,
    restart from the beginning)

5. RESUME
   PUT {sessionUri}
   Content-Range: bytes 524288-2097151/2097152
   Response: 200 OK (or 201 Created)

The session URI is stateful and durable: Google keeps sessions alive for a week. So you can close your browser, board a flight, and resume the upload on hotel Wi-Fi hours later, once you pick the same file again. tus-js-client, for example, stores these URLs in localStorage by default, so the upload can pick up again after a page refresh. On S3, the same role is played by the UploadId that CreateMultipartUpload returns.

Parallel Uploads and Throughput#

Uploading chunks one at a time is like driving a fleet of trucks down a single highway lane. You're wasting capacity.

Each TCP connection is a lane, and thanks to TCP slow start (opens in new tab), every new connection begins at a crawl before ramping to full speed. On high-latency or lossy paths, a single TCP connection rarely saturates modern bandwidth: its throughput is bounded by round-trip time and loss. Multiple lanes working at once get you closer to peak throughput.

HTTP/1.1 allows multiple connections (lanes) per host, giving you a raw speed multiplier. HTTP/2 multiplexes everything onto one connection; it's more efficient but shares a single congestion window. Parallel chunk uploads help on both. Not every API allows it: S3, Azure, B2 and GCS's XML API take parts in parallel; OneDrive requires them in order, and Drive resumes from a single byte offset.

A common starting point is 4 to 6 concurrent requests: desktop browsers typically open up to 6 HTTP/1.1 connections per host. Going higher buys little: on HTTP/1.1 extra requests just wait in the browser's queue, and every chunk in flight holds its buffer in memory.

Turning Chunks into a File#

S3 and Azure accept parts in any order, and B2 numbers parts you upload in parallel, so chunks can arrive out of order. Some requests might still be in flight. On those APIs, the server needs an explicit signal: "I'm done—assemble this thing." Without it, your upload sits in limbo.

On S3-style multipart APIs (Amazon S3, Google Cloud Storage's XML API, Backblaze B2), skip the commit step and your chunks become orphaned data: ghost files that haunt your bucket and keep costing you money.

Some platforms require this final call: S3 has CompleteMultipartUpload, and Box has a commit that carries the file's SHA-1. Others complete the upload on their own once the last byte of the declared total arrives: Google's resumable protocol does, and so does OneDrive by default. Either way, that is the atomic moment where loose chunks become a valid file.

Deduplication: Never Upload Twice#

Dropbox's sync engine is built on block-level deduplication: the idea that identical data should never be stored or uploaded twice. Files are split into 4 MB blocks and hashed with SHA-256. Before uploading, the client sends these hashes to the server, which checks its block index and responds with only the blocks it still needs. If every block already exists in your account, the client has nothing left to upload.

Delta Sync: Uploading Only What Changed#

While Dropbox deduplicates at the block level, Microsoft OneDrive optimizes for files you edit repeatedly. Their differential sync answers a simple question: if you change one slide in a 50 MB PowerPoint, why re-upload all 50 MB?

Microsoft doesn't publish OneDrive's algorithm; it only says the sync app uploads the parts of large files that changed. Published designs like rsync's and Dropbox's show how this kind of sync can work: divide files into blocks, compare compact fingerprints rather than raw bytes, and transfer only the blocks that differ. So for that one-slide edit, only the changed parts travel—a fraction of the total file size.

Once limited mainly to Office files, OneDrive's differential sync was extended in 2020 (opens in new tab) to all file types (opens in new tab), including images, PDFs and audio recordings. For large files with small, frequent edits, the bandwidth savings can be large.

There is a trade-off: CPU vs. Bandwidth. Calculating block signatures takes processing time. On a slow device with a fast connection, the time spent hashing can outweigh the time saved by sending fewer bytes. Some sync engines skip delta checks for very small files: if the whole file fits in one block, comparing signatures saves nothing.

Presigned URLs and CORS#

Here's the big security challenge: you want browsers to upload directly to cloud storage (bypassing your server for performance), but cloud storage requires credentials.

The solution is presigned URLs. Your frontend asks your backend for an upload URL. The backend, which holds the credentials, generates a URL that carries a cryptographic signature made with your secret key. The frontend then uploads directly to cloud storage using this signed URL. There's just one catch: CORS (Cross-Origin Resource Sharing). The bucket is a different origin from your website, so the browser blocks the request by default. The bucket's CORS configuration needs to allow your site's origin, or the upload will never start.

The URL grants temporary permission, scoped to one specific operation on one object. The signature proves authorization without revealing your secret key. Anyone who has the URL can use it until it expires, which is why presigned URLs should be treated as bearer tokens.

// Backend: one signed URL per part
const s3 = new S3Client({
  // Don't sign an empty-body checksum
  requestChecksumCalculation: "WHEN_REQUIRED",
});
const url = await getSignedUrl(
  s3,
  new UploadPartCommand({
    Bucket: "my-bucket",
    Key: "user-123/video.mp4",
    UploadId: uploadId,
    PartNumber: partNumber,
  }),
  { expiresIn: 900 }, // 15 minutes
);

// Browser: upload the part, keep its ETag
const res = await fetch(url, {
  method: "PUT",
  body: chunk,
});
if (!res.ok) {
  throw new Error(`HTTP ${res.status}`);
}
const etag = res.headers.get("ETag");
if (!etag) {
  throw new Error("ETag hidden by CORS");
}
parts.push({
  PartNumber: partNumber,
  ETag: etag,
});

Each part's response carries an ETag, and you need every one of them, sorted by part number, to complete the upload. The browser can only read it if the bucket's CORS configuration exposes the ETag header.

Verifying Integrity with Checksums#

With chunks flying around in parallel across unreliable networks, how do you make sure nothing got corrupted? Major storage APIs (S3, GCS, Azure Blob, Box, B2) verify integrity with checksums (CRCs or cryptographic hashes).

The client calculates a checksum before sending, includes it in the request, and the server verifies that the data it received matches. Mismatch? The chunk is rejected and must be re-sent. The approach varies by provider:

• AWS S3: supports several checksum algorithms, including CRC-64/NVME, CRC-32, CRC-32C, SHA-1, and SHA-256.
• Box: requires a digest with every chunk (SHA-1 or SHA-512) and a SHA-1 digest of the whole file at the final commit.
• Backblaze B2: requires the X-Bz-Content-Sha1 header on uploads.

HTTP/3 and QUIC#

TCP suffers from Head-of-Line Blocking. Imagine a one-lane road where a car crash (packet loss) stops all traffic behind it. That's TCP. On shaky mobile networks, one lost packet stalls everything on that connection.

HTTP/3 (QUIC) keeps a single connection but gives every stream its own lane inside it. Each stream is independent at the transport layer, so a dropped packet on one stream does not stall the others. In 2026, Chrome, Edge, Firefox and Safari ship with HTTP/3 enabled. Storage APIs vary: Cloud Storage supports HTTP/3, Azure Storage accepts only HTTP/1.1, and S3 doesn't document HTTP/2 or HTTP/3. Still, the protocol's benefits for mobile networks—connection migration, faster handshakes, and independent streams—make it increasingly relevant for upload-heavy applications.

Retrying with Backoff and Jitter#

When a server crashes, thousands of clients might retry at the exact same moment, causing a second crash. This is the Thundering Herd problem. The standard defense is Exponential Backoff with Jitter, plus a cap on how many times each request retries.

Instead of retrying immediately, clients wait. And crucially, they add a random delay (jitter) so they don't all come back in sync. Only transient failures are worth retrying, such as network errors, 429 and 5xx. An expired URL needs a new signature, and a 404 from Drive means starting over.

// After a failed attempt (inside catch):
if (!isRetryable(error)) throw error;
if (attempt >= MAX_RETRIES) throw error;
const ceiling = Math.min(
  BASE_DELAY * 2 ** attempt,
  MAX_DELAY
);
await sleep(Math.random() * ceiling);

Developer Takeaways#

Having built a 50 GB uploader, my advice is that most teams shouldn't build this plumbing themselves. The right layer depends on how much of the pipeline you want to own:

• tus.io (opens in new tab): You run the server; the protocol handles the rest. An open protocol for resumable uploads, with open-source clients and servers, that gives you resumable uploads and retries out of the box.
• Cloud SDKs: The cloud stores the file; you own the flow. The official libraries for AWS, Azure, and Google Cloud include high-level upload classes that split files and upload parts in parallel for you. In the browser, they should run with short-lived, scoped credentials, never your secret keys.
• Managed Services: You own none of it. If uploads aren't your core product, platforms like Cloudinary, Uploadcare, and Transloadit handle upload, storage and delivery for you.

An upload should be the one thing users never have to think about. Everything in this article is what makes that possible.