· 6 min read · #go

MaxBytes Middleware in Go: Same Trap, Different Escape

In the previous post, I needed a longer timeout for a single file upload endpoint, and the obvious fix – wrapping the route in another timeout middleware – silently did nothing. A child context can never extend its parent’s deadline, so I had to build a timeout middleware that could be chained.

That endpoint had a second requirement which I left out of the story: a larger request body limit. Most of my API accepts small JSON payloads, so a strict global limit is a sane default. File uploads need a couple of orders of magnitude more. And the obvious solution – overriding the limit with another middleware on that route – fails in exactly the same way. Silently.

Limiting the Request Body Size

Out of the box, the HTTP server caps header size via MaxHeaderBytes, but a body size limit is something every application has to add itself. It’s the same DoS-protection story as timeouts: without it, anyone can stream gigabytes into your JSON decoder.

The standard tool for this is http.MaxBytesReader. It wraps the request body and makes reads fail once they go past the limit. Turning it into a middleware is a few lines, and this is essentially what chi ships as middleware.RequestSize:

func MaxBytes(limit int64) func(next http.Handler) http.Handler {
	return func(next http.Handler) http.Handler {
		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			r.Body = http.MaxBytesReader(w, r.Body, limit)
			next.ServeHTTP(w, r)
		})
	}
}

Downstream code doesn’t need to know anything about the limit. The handler reads the body as usual, and when the limit is exceeded, the read returns an *http.MaxBytesError, which the handler turns into an HTTP 413.

The Problem

Now, the same situation as last time: a strict default limit for the whole API, and a single route that needs a much larger one.

router.Use(MaxBytes(1 * 1024 * 1024)) // 1 MB default

router.Get("/messages", messagesHandler.List)
router.Post("/messages", messagesHandler.Create)
//... many other routes

// Large route
router.
	With(MaxBytes(100 * 1024 * 1024)). // 100MB override for /files
	Post("/files", filesHandler.Upload)

And again, this code looks reasonable but doesn’t work. Each MaxBytes middleware wraps whatever body it finds, so the upload handler ends up reading through two nested limiters: the inner 100 MB one, which in turn reads from the outer 1 MB one. The outer limiter fails first, and the upload still dies at 1 MB.

And as before, route groups don’t solve this in any non-trivial case: routes needing the larger limit are already scattered across groups that exist for other reasons – auth, features, or API versions.

No WithoutCancel This Time

For timeouts, the escape hatch was context.WithoutCancel. It derives a new context that keeps the parent’s values while ignoring its cancellation signals.

There’s no such operation for the request body. By the time the inner middleware runs, r.Body is just an opaque io.ReadCloser. Our limiter may be sitting right on top, or buried under wrappers added by other middlewares. Simply put, it’s not possible to unwrap this io.ReadCloser to get the original body.

So if we can’t unwrap, let’s not wrap twice in the first place.

One Limiter, Shared Through the Context

The idea: the first MaxBytes middleware in the chain wraps the body with a limiting reader and stores a pointer to it in the request context. Every MaxBytes middleware deeper in the chain finds that pointer and simply overwrites the limit, instead of wrapping again.

var limiterCtxKey = new(int)

// MaxBytes returns a middleware that limits the number of bytes read from the request body.
// Inner middlewares can override the limit set by an outer one, both extending and shrinking it.
func MaxBytes(size int64) func(next http.Handler) http.Handler {
	return func(next http.Handler) http.Handler {
		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			ctx := r.Context()
			limiter, _ := ctx.Value(limiterCtxKey).(*limitReader)

			// The body is already wrapped by an outer middleware.
			// Just update the limit.
			if limiter != nil {
				limiter.limit = size
				next.ServeHTTP(w, r)
				return
			}

			// First MaxBytes in the chain: wrap the body
			// and share the limiter via the context.
			limiter = &limitReader{
				Reader: r.Body,
				Closer: r.Body,
				limit:  size,
			}

			ctx = context.WithValue(ctx, limiterCtxKey, limiter)
			r = r.WithContext(ctx)
			r.Body = limiter

			next.ServeHTTP(w, r)
		})
	}
}

Since middlewares run from the outermost to the innermost, the MaxBytes closest to the handler wins – exactly the override semantics we want, whether it shrinks the limit or extends it.

The limiting reader is the part we don’t have to invent. http.MaxBytesReader already gets the read semantics exactly right; the only thing it lacks is a way to change the limit after creation. So the implementation below openly borrows its logic and returns the same *http.MaxBytesError.

type limitReader struct {
	io.Reader
	io.Closer
	limit int64
	n     int64 // bytes read so far
	err   error // sticky error
}

func (r *limitReader) Read(p []byte) (n int, err error) {
	if r.err != nil {
		return 0, r.err
	}
	if len(p) == 0 {
		return 0, nil
	}

	// Ask the body for at most one byte past the limit.
	if remaining := r.limit - r.n + 1; int64(len(p)) > remaining {
		p = p[:remaining]
	}

	n, err = r.Reader.Read(p)
	r.n += int64(n)

	// Within the limit: hand out everything and remember the error.
	if r.n <= r.limit {
		r.err = err
		return n, err
	}

	// Past the limit: hand out only the bytes within the limit and fail.
	n -= int(r.n - r.limit)
	r.err = &http.MaxBytesError{Limit: r.limit}
	return n, r.err
}

The hardest case is a body of exactly limit bytes. It’s perfectly valid, but after consuming limit bytes the reader can’t tell whether the body ends there or keeps going. So the reader asks for one extra byte to distinguish between these two cases. The extra byte itself is never visible to downstream code.

The other inherited details are smaller. The error is sticky: after any failure, reads keep returning the same error without touching the body again. And the over-limit check trusts the byte count, not the error value: the final read of an oversized body may carry the extra byte together with io.EOF, and the limit must win.

Cooperative by Design

Both the Timeout and MaxBytes middlewares are cooperative: they don’t terminate anything by force. One makes the context expire, the other makes body reads fail. The rest is the handler’s side of the contract: notice the error, stop the work, and return.

The error surfaces wherever the body is read – typically inside json.Decode or io.Copy. In Go applications handler errors are usually passed up the stack and handled centrally, and that’s the place to turn http.MaxBytesError into an HTTP 413:

if _, ok := errors.AsType[*http.MaxBytesError](err); ok {
	http.Error(w, "request body too large", http.StatusRequestEntityTooLarge)
	return
}

Once the handler returns, net/http takes care of everything else: the response goes out, the request context gets canceled, and if the client is still streaming the oversized body, the server doesn’t drain it endlessly – it eventually closes the connection. That’s the same protection http.MaxBytesReader provides by flagging the connection explicitly.

And in practice this contract costs nothing. A typical Go application already works exactly this way: every error is checked, handlers return early on failure, and errors bubble up to one place that maps them to status codes. The limits enforce themselves through ordinary error handling.

Conclusion

The routing code from the beginning of the post now works exactly as it reads: a strict default, and a single route that genuinely gets the larger limit. You can try it yourself in the Go playground.

The broader lesson is the same one as last time, just one level lower. Middleware that constrains a request-scoped resource by wrapping it – a context, a body reader – composes one way only: inner layers can tighten the constraint, but can’t loosen it. For deadlines, the standard library provides an escape hatch in context.WithoutCancel. For the request body, there isn’t one, but it’s easy to build your own: wrap once, share the single mutable limiter through the context, and let the innermost middleware have the last word.

subscribe

Join the newsletter

Thanks for reading. If you want a heads-up when I publish something new, drop your email below – about one post per month, no other emails.

log – more entries