#rill posts

Preserving Order in Concurrent Go Apps

Processing items concurrently in Go is easy enough: have a pool of goroutines read from one channel and write to another. Results arrive in completion order, which is fine until we need input order instead. Let’s build and benchmark three ways to make that happen with infinite stream support and no per-item allocations.

Real-Time Batching in Go

Thousands of tiny database updates per second can become a few bulk queries without touching the call sites. Each call still takes a single item, blocks until it is done, reports errors, and respects context cancellation. Behind the scenes, a worker batches the updates as they arrive.

From Hours to Minutes: Listing S3 and GCS Buckets Faster in Go

While cleaning up a large GCS bucket, I tried to speed things up by deleting files concurrently, only to discover that listing them had become the bottleneck. Parallelizing the listing is less straightforward, but it can turn hours of scanning into minutes with surprisingly little Go code. Examples cover both GCS and S3.