Hello there
I’ve been writing Go backends for a living since 2016. I enjoy it most when I can make a hard problem simple, usually by removing complexity that shouldn’t be there and finding abstractions that earn their keep.
A strict request body limit is a sensible default for an API, until a file upload endpoint needs more room. Adding a larger limit to that route looks like an easy fix, but uploads still hit the global limit. In this follow-up to the timeout middleware post, we’ll build MaxBytes middleware that supports per-route overrides.
Processing items concurrently in Go is easy enough: have a pool of goroutines read from one channel and write to another. Results arrive in completion order, which is fine until we need input order instead. Let’s build and benchmark three ways to make that happen with infinite stream support and no per-item allocations.
After a decade of Go backends, I wanted to build something where design, animation, and user experience came first. The result is a daily number game with a vintage cartoon style: pick the lowest number, but if someone else picks it too, you’re out.
Most API routes can use a short default timeout, but file uploads often need more time. Wrapping an upload route in another timeout middleware seems straightforward, yet the shorter global deadline still applies. Here’s how to build middleware that lets each route override the default.
Thousands of tiny database updates per second can become a few bulk queries without touching the call sites. Each call still takes a single item, blocks until it is done, reports errors, and respects context cancellation. Behind the scenes, a worker batches the updates as they arrive.
File upload handlers often stream the request body straight to S3. A small reader wrapper can sniff and validate the content type without buffering the entire file, while letting the existing upload code handle validation failures like any other I/O error.
I kept writing the same progress-tracking code for one-off concurrent Go scripts: a few atomic counters, a printing loop, and increasingly cluttered output. That led to dspc, a small lock-free package that tracks named counters across goroutines and prints progress in place without overwriting the script’s output.
While cleaning up a large GCS bucket, I tried to speed things up by deleting files concurrently, only to discover that listing them had become the bottleneck. Parallelizing the listing is less straightforward, but it can turn hours of scanning into minutes with surprisingly little Go code. Examples cover both GCS and S3.