I work on a platform where pretty much everything runs on AWS Lambda. DynamoDB for storage, EventBridge and SNS for messaging, SQS when things need to queue. TypeScript everywhere. It’s been a few years of this, and most of what I know came from production incidents, not tutorials.
Stop blaming cold starts
Cold starts get blamed for everything. Slow API? Cold start. Timeout? Cold start. Bad day? Probably a cold start.
In practice, the thing making your Lambda slow is usually inside your handler. I spent a whole afternoon convinced I had a cold start problem. Turned out I was initializing the DynamoDB client inside the handler function instead of outside it — creating a new connection on every single invocation. Moved it to module scope, response times dropped by 200ms. Nothing to do with cold starts.
If your Lambda chain takes three seconds and you’re blaming cold starts, check your code first. It’s almost always the code.
DynamoDB single-table design is a trust fall
Coming from SQL, DynamoDB makes no sense. Your first instinct is to create a table per entity like you would in Postgres. Then someone tells you to put users, orders, and invoices in the same table and you think they’ve lost it.
I resisted single-table design for months. Then I tried it on one service and realized the whole point: you model around your queries, not your entities. One query() call gets you everything you need for a page render. No joins. No multiple round-trips. The cost is the same whether you have 10 rows or 10 million.
The tradeoff is real though: you need to know your access patterns before you write a single line of code. Add a new query pattern six months later and you’re either adding a GSI or doing a data migration. Plan upfront or pay later.
Event schemas will haunt you
We use EventBridge and SNS heavily. Services publish events, other services consume them. Works great until someone changes a field name in a publisher and doesn’t realize three other services depend on that exact shape.
I’ve debugged issues where data was silently wrong for days because an event schema changed and the consumers didn’t error — they just parsed the missing field as undefined and wrote garbage to the database. No alerts. No failures. Just bad data spreading quietly.
Now I treat event schemas like database migrations. Schema validation before publish. Breaking changes get the same review as a table migration. It’s not glamorous work but it prevents the kind of bugs that make you question your career choices.
Idempotency is not optional and you will learn this the hard way
SQS can deliver the same message twice. Lambda can retry on failure. EventBridge doesn’t guarantee exactly-once delivery. This isn’t a theoretical concern. It happens in production regularly.
I found out when a handler processed a payment event twice and a customer got double-charged. The handler worked perfectly. It just wasn’t designed to be called twice with the same input. We added conditional writes with DynamoDB (check if the idempotency key exists before processing) and the problem went away. Should have done it from day one.
Every handler that writes data needs to be safe to run twice. No exceptions.
Your logs are your only friend at 2 AM
There’s no SSH. No debugger. No “let me check the server.” When something breaks at 2 AM, you open CloudWatch and you read logs. That’s it.
The difference between a 10-minute fix and a 2-hour fix is almost always log quality. Early on I was logging like console.log('processing event') — totally useless when you’re looking at thousands of invocations. Now everything is structured JSON with a correlation ID that follows a request across services. When something fails, I can trace the entire chain in one query.
It’s boring work to set up. It’s the most valuable boring work you’ll do.
Why I’m still on serverless
People always ask about the tradeoffs. And yes, cold starts exist, DynamoDB has a learning curve, and debugging distributed event-driven systems is genuinely harder than debugging a monolith.
But nobody pages me because a server ran out of disk. I don’t maintain EC2 instances. I don’t think about capacity planning. The constraints push you toward better architecture whether you want them to or not.
I’ve been on both sides. I’ll take the constraints.