The code works until the data grows

Most backend performance bugs don’t look broken when you write them.

The query returns the right rows. The Lambda passes its tests. The code review is clean. Then the table grows, one customer uploads far more data than everyone else, or a queue gets a burst you never saw in development. Same code starts timing out.

A DynamoDB Scan reads thousands of items to return a handful. A search endpoint fetches and discards every earlier page before showing page 91. A stream handler turns one database update into forty writes somewhere else.

These aren’t really AI bugs. We wrote them long before code generators showed up. AI just ships plausible code faster, before anyone stops to explain what production actually looks like.

The useful question isn’t “does this code work?” It’s: what grows, who controls that growth, and what happens once for every item?

The useful part of Big-O

Big-O describes how work grows as the input grows. It does not tell you how many milliseconds something takes.

You only need a small vocabulary for most backend reviews:

• O(1): the amount of work stays roughly constant. A key lookup is the usual example.
• O(log n): work grows slowly because each step removes a large part of the search space.
• O(n): double the input and you roughly double the work.
• O(n log n): common when sorting a growing result set.
• O(n²): each item causes work across another growing collection.

The notation is useful, but it drops constants. That matters in backend code. A thousand in-memory comparisons and a thousand network round-trips are both O(n), but only one is likely to ruin your afternoon.

Three questions before approving the code

I use three questions when a database call, API request, or queue consumer touches a collection.

1. What is n?

Name the thing that grows. Not “the items.” Be specific:

• documents per customer
• IDs accepted by an endpoint
• rows in a table
• records in a queue batch
• related entities loaded for each event

Then ask who controls it. If a user or data pipeline can grow it and nothing enforces a limit, it is unbounded until proven otherwise.

2. What happens per item?

Count the expensive unit.

A loop that compares strings is not the same as a loop that calls DynamoDB. CPU work may be cheap enough to ignore. A database, Elasticsearch, or HTTP call has network latency, capacity limits, and failure modes.

3. Is there another loop hiding inside?

.find(), .includes(), .filter(), and .some() all scan something. A repository method inside a loop might hide a network request. A GraphQL field resolver might run once per parent result.

A function call inside a loop is worth opening before you approve it.

Name what grows, count what happens once per item, and check whether that work crosses a network boundary. If the input is bounded and small, move on. If it can grow without another deployment, test the 10× case.

The loop inside the loop

Before the database examples, the cheapest bug to write. Two collections, one lookup per outer item.

Problematic shape:

// n outer items × n inner scans = O(n²) in memory
const enriched = orders.map(order => ({
  ...order,
  customer: customers.find(c => c.id === order.customerId)
}));

Ten orders and ten customers, 100 comparisons. Invisible. Ten thousand orders and ten thousand customers, one hundred million. The Lambda that finished in 40 ms in staging times out at 30 seconds in production, and the diff that caused it added one line.

Fixed shape:

// Build the lookup once, then O(n) reads.
const customerById = new Map(customers.map(c => [c.id, c]));

const enriched = orders.map(order => ({
  ...order,
  customer: customerById.get(order.customerId)
}));

Same output. Different growth curve. n here is the larger of the two collections, and whoever produced it decides how big it gets — often a query result nobody bounded.

The rule that catches this in review: any .find(), .includes(), .filter(), or .some() inside a map, forEach, or for loop is a nested scan until proven otherwise.

DynamoDB Scan: filtering happens too late

n here is rows in the table, controlled by every write that ever landed. Every scan pays for it, whether the filter matches or not.

A DynamoDB Scan reads a page first and applies its FilterExpression afterwards. One request reads up to 1 MB; a full-table scan keeps following LastEvaluatedKey until every page has been read.

Problematic shape: this looks selective, but the filter does not reduce what DynamoDB reads.

const result = await ddb.send(new ScanCommand({
  TableName: 'orders',
  FilterExpression: '#status = :status',
  ExpressionAttributeNames: { '#status': 'status' },
  ExpressionAttributeValues: { ':status': 'pending' }
}));

If that page contains 10,000 orders and only 50 are pending, the response tells the story:

ScannedCount: 10,000
Count: 50

Fifty items came back. Ten thousand were examined, and you paid for all of them.

The fix is not automatically “add a GSI.” The real fix is to model the access pattern so DynamoDB can target a bounded partition.

A GSI might be part of the answer, but a low-cardinality key such as status = pending can create a hot partition. Sometimes the useful key is status plus a time bucket or shard.

A more targeted shape:

const result = await ddb.send(new QueryCommand({
  TableName: 'orders',
  IndexName: 'status-createdAt-index',
  KeyConditionExpression: '#status = :status',
  ExpressionAttributeNames: { '#status': 'status' },
  ExpressionAttributeValues: { ':status': 'pending' }
}));

A Query targets the selected partition and charges for the data it reads there. It doesn’t touch the rest of the table. The trade-off is that every GSI adds storage, write cost, and another data model you have to maintain.

One network call per item

n here is the size of the incoming ID list, controlled by whoever calls the endpoint. Each item costs a network round-trip. This is N+1 at its most obvious.

Problematic shape:

const users = [];

for (const userId of userIds) {
  const response = await ddb.send(new GetCommand({
    TableName: 'users',
    Key: { userId }
  }));

  users.push(response.Item);
}

If userIds came from an unbounded request, latency grows with every ID. Two thousand IDs means two thousand sequential round-trips.

Promise.all() is not automatically the fix. Firing two thousand requests at once changes a latency problem into a throttling problem.

For DynamoDB keys, BatchGetItem can retrieve up to 100 items per call. Production code needs to retry UnprocessedKeys with backoff, and shouldn’t assume results come back in request order.

Safer shape: batch, retry the unprocessed keys, cap concurrency.

const users: User[] = [];

for (const ids of chunks(userIds, 100)) {
  let requestItems: BatchGetCommandInput['RequestItems'] = {
    users: { Keys: ids.map(userId => ({ userId })) }
  };

  // Retry only the keys DynamoDB throttled, with bounded backoff.
  for (let attempt = 0; attempt < 5 && requestItems; attempt++) {
    if (attempt > 0) {
      await sleep(2 ** attempt * 50 + Math.random() * 50);
    }

    const response = await ddb.send(new BatchGetCommand({ RequestItems: requestItems }));
    users.push(...(response.Responses?.users ?? []));

    const unprocessed = response.UnprocessedKeys?.users?.Keys;
    requestItems = unprocessed?.length ? { users: { Keys: unprocessed } } : undefined;
  }

  if (requestItems) {
    throw new Error(`BatchGet exhausted retries for ${requestItems.users.Keys.length} keys`);
  }
}

That is still work proportional to the number of users. The improvement is fewer network round-trips with controlled pressure on DynamoDB, and a clear failure mode when the table cannot keep up.

Two details the type system won’t remind you about: BatchGetItem doesn’t preserve request order, so re-key the results by ID if the caller expects them aligned; and 100 is the item-count cap, but the response is also capped at 16 MB. A batch of large items comes back partially even without throttling.

The same idea applies outside DynamoDB: use a bulk endpoint when one exists, cap concurrency when it does not, and validate input bounds before the work starts.

Deep pagination is a product question

n here is the requested offset, controlled by the client. Every earlier page is work the server does and throws away.

Elasticsearch’s from + size re-collects and discards the pages before yours on every request. By default it also refuses to page past 10,000 hits. That ceiling is index.max_result_window, and raising it makes the same shape more expensive rather than fixing it.

// Server does O(from + size) work per request.
const results = await es.search({
  index: 'documents',
  from: page * pageSize,
  size: pageSize,
  query: { match_all: {} },
  sort: [{ createdAt: 'desc' }]
});

The technical answer is search_after: sort on a stable tiebreaker, pass the last hit’s sort array as the next request’s cursor. It removes the offset cost but keeps the per-page work, and it is inherently sequential. Good for “next page” UIs and exports. Useless for “jump to page 91.”

The more useful answer is upstream. If the product genuinely needs random-page access over millions of documents, that is a product decision with an infrastructure bill attached. If it doesn’t, replace the page number in the UI with a cursor and keep moving. Most “deep pagination” bugs are really “we shipped a page-number UI over an unbounded result set” bugs.

Fan-out: multiplication after the loop

n here is a product, not a sum: batch size times related documents. Two knobs, both controlled elsewhere, multiplied inside your handler.

Suppose a DynamoDB Streams Lambda receives b records. For each record, it loads d related documents and indexes them one by one.

The multiplier hiding in the handler:

for (const record of event.Records) {
  const documents = await loadRelatedDocuments(record);

  for (const document of documents) {
    await es.index({ id: document.id, document });
  }
}

The external write count is b × d. Neither factor is obvious in the code, and both can grow without anyone touching this file.

Retries then multiply whatever inefficiency was already there. For example:

10stream records

× 40related documents

× 3delivery attempts

= 1,200external writes

The narrowest fix depends on why the fan-out exists:

• Filter stream records that do not affect the search document.
• Update only the changed document when the whole group does not need rebuilding.
• Use Elasticsearch’s Bulk API to replace many sequential writes with bounded batches.
• Configure batch size, partial-batch failure handling, and retry limits for the actual event source.
• Build explicit coalescing only when repeated updates genuinely need to collapse into one later refresh.

Fixed shape for the common case: one batched bulk write per Lambda invocation, checked for per-item failures.

const operations = event.Records.flatMap(record => {
  const documents = extractChangedDocuments(record);

  return documents.flatMap(doc => [
    { index: { _index: 'documents', _id: doc.id } },
    doc
  ]);
});

if (operations.length === 0) return;

const response = await es.bulk({ operations, refresh: false });

if (response.errors) {
  const failed = response.items
    .map((item, i) => ({ item: item.index ?? item.update, doc: operations[i * 2 + 1] }))
    .filter(({ item }) => item?.error);

  // Fail the batch so the event source retries only the records that need it,
  // or route the failed docs to a DLQ — do not swallow silently.
  throw new Error(`Bulk indexing failed for ${failed.length} of ${response.items.length} docs`);
}

Two changes worth naming. First, one network call per invocation instead of b × d. Second, the handler now has an observable failure mode. response.errors is the signal the previous shape did not produce.

Do not use SQS FIFO deduplication as a generic debounce mechanism. It prevents duplicate message delivery within its deduplication window. It doesn’t wait for a burst to finish and then deliver the latest state.

Also, name the retry system. A DynamoDB Streams trigger is retried by Lambda’s event source mapping. An EventBridge rule has a different delivery policy: by default, EventBridge retries for up to 24 hours and 185 attempts with exponential backoff. Mixing those two models leads to confident but incorrect architecture diagrams.

Memory grows too

n here is items across all pages, controlled by the table. What each item costs is bytes, not milliseconds, and Lambda has a fixed ceiling.

A single DynamoDB Scan response is capped at 1 MB. The memory problem appears when application code follows every page and stores all of them.

Problematic shape:

const allItems = [];
let lastKey;

do {
  const page = await ddb.send(new ScanCommand({
    TableName: 'items',
    ExclusiveStartKey: lastKey
  }));

  allItems.push(...(page.Items ?? []));
  lastKey = page.LastEvaluatedKey;
} while (lastKey);

The array grows with the table. In a Lambda, that eventually meets a fixed memory ceiling.

Process each page and discard it instead.

Bounded-memory shape:

let lastKey;

do {
  const page = await ddb.send(new ScanCommand({
    TableName: 'items',
    Limit: 500,
    ExclusiveStartKey: lastKey
  }));

  await processPage(page.Items ?? []);
  lastKey = page.LastEvaluatedKey;
} while (lastKey);

Memory is now bounded by the page size rather than the full table. You still pay the read cost of scanning, so pagination fixes memory pressure, not the access pattern.

Measure the shape before fixing it

A complexity label is a hypothesis. Production evidence tells you whether it matters.

• Compare Lambda duration with the number of records processed. A rising p99 with a flat p50 often means larger tenants are crossing a boundary first.
• Log DynamoDB Count beside ScannedCount for suspicious reads, and request consumed capacity when cost is the question.
• Count database, Elasticsearch, and HTTP calls per invocation.
• Compare Elasticsearch page depth with latency and memory pressure.
• Test with a realistic n. Ten fixture records can make almost any approach look good.
• Identify the retry owner before multiplying attempts into the estimate.

Complexity bugs often arrive one tenant at a time. The average request can look healthy while one customer’s data has already crossed the cliff.

From code smell to architecture question

The code-level fix is only half the review. Someone reviewing at the architecture level should ask who owns the boundary that let the work grow.

Code smell	Coding question	Architecture question
Nested scan in memory	Can this be a Map lookup?	Should this join happen in the datastore instead?
Scan plus filter	Can this be a targeted Query?	Who owns the access pattern and index cost?
Remote call inside a loop	Can this be batched or bounded?	Should the service expose a bulk interface?
Deep pagination	Can this use a cursor?	Does the product require random page access?
Stream fan-out	Can writes be filtered or bulked?	Where does backpressure live?
Retried batch	Is each side effect idempotent?	Which retry system owns exhaustion and recovery?
Full accumulation	Can pages be processed and discarded?	Should the workload be split across invocations?

“Replace the loop” is a code review. “Why does this boundary require one call per item?” is an architecture review.

What to tell the coding tool before it writes the loop

A coding tool sees the types and the function body. It usually can’t see next year’s table size, the product’s pagination requirement, or the downstream throttle limit unless you provide them.

Before generating a collection-heavy path, state:

• the maximum IDs accepted by the endpoint
• expected and worst-case rows per tenant
• allowed network calls per request
• pagination depth the product must support
• Lambda timeout and memory budget
• the event source and retry behavior

Constraint brief:

Implement this enrichment path. userIds is user-controlled and capped at 500.
Do not issue one database request per ID or use unbounded concurrency.
Prefer the existing batch API, handle partial results, and preserve input ordering.

The prompt is not the guardrail. Validation, tests, and review still are. But giving the tool real limits is better than asking it to “make this scalable” and hoping it guesses what that means.

When not to optimize

This does not imply that every loop should be rewritten. In the case where a list has only two values, replacing includes() with a Set is merely showing off. When a maintenance task runs only once a month on a small table, a Scan is the sensible option. Before making any changes to the code, it is necessary to verify that n can actually increase in a production environment, that the operation is indeed taking place on a real request path, and that the proposed fix justifies the additional infrastructure required. If it doesn’t, then don’t proceed. Big-O is a tool for evaluation, not a justification for redesigning every loop.

The habit worth keeping

When I review collection-heavy backend code, I write down one sentence:

n is ___, it is controlled by ___, and each item causes ___ remote calls or ___ memory.

That sentence catches more than writing O(n) in a comment. It forces the production constraint into the review.

AI can help replace the loop once the problem is clear. It can’t infer the customer limit, the table growth, the queue burst, or the retry policy you never gave it. Those are still engineering decisions.

References

• DynamoDB Scan
• DynamoDB BatchGetItem
• Elasticsearch pagination
• EventBridge retry policy
• SQS FIFO deduplication