SQL
Ask a database the right questions: joins, aggregates, window functions, indexes and transactions.
In 30 seconds
SQL is the language for asking relational databases questions and changing their data. You describe what you want (these columns, from these tables, matching these rules), and the database figures out how to get it. Knowing SQL well, including indexes and transactions, is often the difference between a page that loads in 50 milliseconds and one that times out.
Why I use it
- I use SQL every day: almost every product I build stores its core data in MySQL, MariaDB or PostgreSQL, and ORMs only hide SQL, they do not replace it.
- I write reporting and analytics directly in SQL (sales, cohorts, stock valuation), where aggregates and window functions do in one query what would take hundreds of lines of application code.
- I read query plans because performance problems in web apps are most often database problems: a missing index or an N+1 loop.
- I rely on transactions to keep money, stock and loyalty balances correct when many requests touch the same rows.
How it works
01The core: SELECT, WHERE, ORDER BY and the logical order
A relational database stores data in tables: rows are records, columns are fields, and a primary key uniquely identifies each row. A SELECT query reads data. It helps to know the logical order in which the parts are evaluated, because it explains many errors: FROM and JOIN first (which rows exist), then WHERE (filter rows), GROUP BY (form groups), HAVING (filter groups), SELECT (compute columns), ORDER BY (sort) and finally LIMIT. That is why you cannot use a SELECT alias inside WHERE. NULL means 'unknown', not zero or empty: compare with IS NULL, because NULL = NULL is not true. Always pass user input as parameters (placeholders) rather than concatenating strings, which prevents SQL injection.
-- Ten most recent paid orders above 50.00
SELECT id, customer_id, total, created_at
FROM orders
WHERE status = 'paid'
AND total > 50.00
AND cancelled_at IS NULL
ORDER BY created_at DESC
LIMIT 10;
-- From application code, always use parameters:
-- SELECT * FROM users WHERE email = ? -- (MySQL)
-- SELECT * FROM users WHERE email = $1 -- (PostgreSQL)
02JOINs, GROUP BY and subqueries
Data is split across tables to avoid duplication, and JOIN puts it back together. INNER JOIN keeps only rows that match on both sides; LEFT JOIN keeps every row from the left table and fills missing matches with NULL, which is how you find 'customers with no orders'. GROUP BY collapses rows into groups so aggregate functions (COUNT, SUM, AVG, MIN, MAX) work per group; every selected column must be grouped or aggregated. HAVING filters after grouping, WHERE before. A subquery is a query inside another; EXISTS is a clear way to ask 'is there at least one related row?'. Common table expressions (WITH name AS (...)) give subqueries readable names and make long reports far easier to review.
-- Revenue per customer this year, only customers above 1,000
WITH paid AS (
SELECT customer_id, total
FROM orders
WHERE status = 'paid' AND created_at >= '2026-01-01'
)
SELECT c.id, c.name, COUNT(*) AS orders, SUM(p.total) AS revenue
FROM customers c
JOIN paid p ON p.customer_id = c.id
GROUP BY c.id, c.name
HAVING SUM(p.total) > 1000
ORDER BY revenue DESC;
-- Customers who never ordered
SELECT c.id, c.email FROM customers c
WHERE NOT EXISTS (SELECT 1 FROM orders o WHERE o.customer_id = c.id);
03Window functions
Window functions calculate across a set of related rows without collapsing them, unlike GROUP BY. You keep every order row, and next to it you can show the customer's running total, the order's rank, or the previous order's date. The OVER clause defines the window: PARTITION BY splits rows into groups (per customer), ORDER BY defines the order inside each group. Common functions are ROW_NUMBER (a unique sequence, perfect for 'latest row per group'), RANK and DENSE_RANK, LAG and LEAD (previous or next row), and running SUM or AVG. They are supported in PostgreSQL, MySQL 8+, MariaDB 10.2+ and SQLite, and they replace many slow self-joins and loops in application code.
-- Each customer's latest order, plus days since their previous one
SELECT customer_id, id, total, created_at, days_since_previous
FROM (
SELECT
o.*,
ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY created_at DESC) AS rn,
created_at - LAG(created_at) OVER (
PARTITION BY customer_id ORDER BY created_at
) AS days_since_previous
FROM orders o
WHERE status = 'paid'
) ranked
WHERE rn = 1;
-- (date arithmetic shown in PostgreSQL syntax)
04Indexes and how a B-tree works
Without an index, the database must read every row to find matches (a full table scan), like searching a book with no index page. A B-tree index is a sorted, balanced tree of key values with pointers to the rows. Because it is sorted and shallow (usually three or four levels even for millions of rows), the database can jump to a value, or to the start of a range, in a few page reads and then walk forward. That also helps ORDER BY and LIMIT. A composite index on (customer_id, created_at) is sorted by the first column, then the second, so it serves queries that filter on customer_id, or customer_id and created_at, but not created_at alone: the leftmost-prefix rule. Put equality columns first and range or sort columns last. Indexes are not free: every INSERT and UPDATE must maintain them, so index for your real queries, not every column.
-- Serves: WHERE customer_id = ? ORDER BY created_at DESC LIMIT 20
CREATE INDEX idx_orders_customer_created
ON orders (customer_id, created_at);
-- Unique index also enforces a business rule
CREATE UNIQUE INDEX idx_users_email ON users (email);
-- Not helped by the composite index above (no leftmost column):
-- SELECT * FROM orders WHERE created_at > '2026-10-01';
-- Wrapping the column in a function also prevents index use:
-- WHERE DATE(created_at) = '2026-11-05' -- slow
-- WHERE created_at >= '2026-11-05'
-- AND created_at < '2026-11-06' -- uses the index
05Reading EXPLAIN and fixing N+1 queries
EXPLAIN shows the plan the database chose: which tables it reads, in what order, and whether it uses an index or scans everything. EXPLAIN ANALYZE (PostgreSQL, and MySQL 8.0.18+) actually runs the query and shows real timings and row counts. Look for full scans on large tables (Seq Scan in PostgreSQL, type ALL in MySQL), big gaps between estimated and actual rows, and expensive sorts. The most common performance bug in web apps, though, is not one slow query but N+1: you load 50 orders with one query, then loop and run one more query per order to fetch its customer, so 51 queries instead of 2. ORMs make this easy to write by accident. The fix is eager loading (Laravel with(), Prisma include) or a single JOIN, and a query counter in development or tests to catch regressions.
EXPLAIN ANALYZE
SELECT id, total FROM orders
WHERE customer_id = 42
ORDER BY created_at DESC
LIMIT 20;
-- Good: Index Scan using idx_orders_customer_created
-- Bad: Seq Scan on orders + Sort (missing or unusable index)
-- N+1 in an ORM (pseudo-code):
-- orders = Order.limit(50) -- 1 query
-- for o in orders: o.customer.name -- +50 queries
-- Fix: load customers in one go
SELECT o.id, o.total, c.name
FROM orders o JOIN customers c ON c.id = o.customer_id
ORDER BY o.created_at DESC LIMIT 50;
06Transactions and isolation levels
A transaction groups statements into one all-or-nothing unit: either every change is saved (COMMIT) or none is (ROLLBACK). This is the A in ACID (atomicity, consistency, isolation, durability). Isolation controls what concurrent transactions can see of each other. READ COMMITTED (the PostgreSQL default) sees only committed data, but re-reading a row can return a newer value. REPEATABLE READ (the MySQL InnoDB default) gives a stable snapshot for the whole transaction. SERIALIZABLE behaves as if transactions ran one at a time, but may abort some, so the application must retry. For counters, stock and balances, avoid the read-modify-write race: either update atomically (SET stock = stock - 1 WHERE stock >= 1) or lock the row with SELECT ... FOR UPDATE. Keep transactions short, and never wait for an external HTTP call while holding locks.
BEGIN;
-- Lock the wallet row so concurrent requests wait their turn
SELECT balance FROM wallets WHERE customer_id = 42 FOR UPDATE;
-- Atomic, guarded update: fails safely if funds are insufficient
UPDATE wallets
SET balance = balance - 1500
WHERE customer_id = 42 AND balance >= 1500;
INSERT INTO wallet_transactions (customer_id, amount, reason)
VALUES (42, -1500, 'order_1001');
COMMIT; -- or ROLLBACK if the UPDATE affected 0 rows
Use it when
- Any data with relationships and rules: users, orders, products, payments, bookings.
- Reporting and analytics, where aggregates and window functions answer questions in one query.
- Anything that needs correctness under concurrency: balances, stock levels, unique constraints.
- Debugging slow pages, where EXPLAIN and the query log show what the ORM really does.
Skip it when
- Very high-volume, simple key-value lookups or caching, where Redis is faster and simpler.
- Full-text search with typo tolerance and relevance ranking at scale, where a search engine (Meilisearch, Algolia, Elasticsearch) fits better.
- Huge analytical workloads over billions of events, where a columnar warehouse (ClickHouse, BigQuery) is the right tool.
Alternatives
- ORMs and query builders
Eloquent, Prisma, Drizzle or Doctrine generate SQL for everyday CRUD; still learn SQL to understand and tune what they produce.
- Document databases (MongoDB, Firestore)
Flexible JSON documents and easy scaling, but joins and multi-document transactions are harder.
- Columnar warehouses (ClickHouse, BigQuery)
Built for analytics over massive datasets; they still speak SQL.
Best practices
- Always use parameterised queries; never concatenate user input into SQL.
- Index for real query patterns: equality columns first, range or sort columns last, and check with EXPLAIN.
- Select only the columns you need, and paginate large lists (keyset pagination for deep pages).
- Wrap multi-step changes in transactions, keep them short, and use atomic updates or FOR UPDATE for balances and stock.
- Watch the query count per request in development to catch N+1 problems early.
- Store money as integers in minor units or DECIMAL, never FLOAT.
- Enforce rules in the database too: NOT NULL, foreign keys and unique constraints.
Common mistakes
- N+1 queries hidden behind an ORM loop.
- Wrapping indexed columns in functions (DATE(created_at), LOWER(email)) so the index cannot be used.
- Using OFFSET 100000 for deep pagination, which still reads and discards all the skipped rows.
- Read-then-write updates of balances or stock without a lock or an atomic UPDATE, causing lost updates under load.
- Comparing with = NULL instead of IS NULL, and forgetting that NULL makes NOT IN behave unexpectedly.
Deep dive
07MVCC: how databases let readers and writers work at once
Both PostgreSQL and MySQL InnoDB use multi-version concurrency control (MVCC): instead of making readers wait for writers, the database keeps several versions of a row and shows each transaction the version that matches its snapshot. The two engines store old versions differently, and that shapes how they behave in production. PostgreSQL writes a new row version on every UPDATE and leaves the old one in the table as a dead tuple; autovacuum later reclaims the space. If autovacuum falls behind, tables and indexes bloat and queries slow down. InnoDB updates rows in place and keeps old versions in the undo log; a long-running transaction forces it to keep every version since that snapshot, growing the history list. In both engines the lesson is the same: long transactions are expensive even if they only read, because they pin old versions. Find them with pg_stat_activity or information_schema.innodb_trx and give background jobs a statement or idle-in-transaction timeout.
-- PostgreSQL: long or idle transactions holding old snapshots
SELECT pid, state, now() - xact_start AS age, left(query, 60) AS query
FROM pg_stat_activity
WHERE xact_start IS NOT NULL
ORDER BY age DESC
LIMIT 10;
-- PostgreSQL: tables with many dead tuples (vacuum falling behind)
SELECT relname, n_dead_tup, last_autovacuum
FROM pg_stat_user_tables ORDER BY n_dead_tup DESC LIMIT 10;
-- MySQL InnoDB: open transactions and their age
SELECT trx_id, trx_started, trx_rows_locked
FROM information_schema.innodb_trx ORDER BY trx_started;
08Advanced indexing: covering, partial, expression and skip scans
A covering index contains every column a query needs, so the database can answer from the index alone without visiting the table: an index-only scan in PostgreSQL, 'Using index' in MySQL's EXPLAIN. PostgreSQL's INCLUDE clause adds extra payload columns without making them part of the sort key. A partial index (PostgreSQL) indexes only rows matching a condition, such as unshipped orders, so it stays small and fast when most rows are irrelevant. An expression index indexes the result of a function, which fixes the 'WHERE LOWER(email) = ?' problem; MySQL supports functional key parts since 8.0.13. The leftmost-prefix rule also has a modern nuance: MySQL 8.0 and PostgreSQL 18 can perform a skip scan on a composite index when the leading column has few distinct values, treating it as several smaller range scans. It helps, but it is a fallback, not a reason to design indexes in the wrong order.
-- Covering index (PostgreSQL): answer the list query from the index
CREATE INDEX idx_orders_cust_created_inc
ON orders (customer_id, created_at DESC) INCLUDE (total, status);
-- Partial index: only the rows the dashboard actually reads
CREATE INDEX idx_orders_unshipped
ON orders (created_at) WHERE shipped_at IS NULL;
-- Expression index for case-insensitive lookups
CREATE UNIQUE INDEX idx_users_email_lower ON users (lower(email));
-- MySQL 8.0.13+: CREATE INDEX idx_email_lower ON users ((lower(email)));
-- Query must match the expression to use it
SELECT id FROM users WHERE lower(email) = lower($1);
09Keyset pagination and large result sets
OFFSET pagination looks simple but gets slower with every page, because the database must produce and throw away all skipped rows; page 5,000 of a 50-row list reads 250,000 rows. It is also unstable: if a new row is inserted while a user pages, items shift and appear twice or not at all. Keyset (cursor) pagination remembers the last row the user saw and asks for rows after it, using an index to jump there directly, so every page costs the same. Order by a unique combination, typically a timestamp plus the primary key as a tiebreaker, and compare with a row value (created_at, id) < (?, ?), which PostgreSQL and MySQL both support. The trade-off is that you cannot jump to an arbitrary page number, which is usually fine for feeds, APIs and infinite scroll. For exports of millions of rows, stream in keyset batches rather than loading everything into memory, and run them on a read replica if one exists.
-- First page
SELECT id, created_at, total
FROM orders
WHERE customer_id = 42
ORDER BY created_at DESC, id DESC
LIMIT 50;
-- Next page: pass the last row's (created_at, id) as the cursor
SELECT id, created_at, total
FROM orders
WHERE customer_id = 42
AND (created_at, id) < ('2026-10-31 18:20:00', 98123)
ORDER BY created_at DESC, id DESC
LIMIT 50;
-- Supporting index
CREATE INDEX idx_orders_cust_keyset ON orders (customer_id, created_at, id);
10Locks, deadlocks and zero-downtime schema changes
Every UPDATE takes row locks until the transaction ends. When two transactions each hold a lock the other needs, neither can continue: a deadlock. Databases detect this and abort one of them with an error (PostgreSQL checks after deadlock_timeout, one second by default; InnoDB detects it immediately), so the application must be ready to retry the whole transaction. Prevent most deadlocks by touching rows in a consistent order, for example always locking accounts by ascending id, and by keeping transactions short. InnoDB under REPEATABLE READ also takes gap locks on ranges, which can block inserts unexpectedly. Schema changes are another source of locks. In PostgreSQL, CREATE INDEX CONCURRENTLY builds an index without blocking writes, and adding a nullable column or one with a constant default is instant. In MySQL, many changes support ALGORITHM=INSTANT or INPLACE with LOCK=NONE. Always set a lock_timeout for migrations so a blocked ALTER fails fast instead of queuing every query behind it.
-- PostgreSQL: safe migration settings
SET lock_timeout = '5s';
SET statement_timeout = '15min';
CREATE INDEX CONCURRENTLY idx_orders_status ON orders (status);
-- (cannot run inside a transaction block; re-run if it fails: drop the INVALID index first)
-- Add a NOT NULL constraint without a long lock
ALTER TABLE orders ADD CONSTRAINT orders_currency_nn
CHECK (currency IS NOT NULL) NOT VALID;
ALTER TABLE orders VALIDATE CONSTRAINT orders_currency_nn;
-- MySQL 8: request an online change, fail if impossible
ALTER TABLE orders ADD COLUMN source VARCHAR(20) NULL,
ALGORITHM=INSTANT;
Real-world scenarios
Two customers bought the last item in stock
Problem
A flash sale ended with more orders than units in stock. The code read the stock, checked that it was above zero, then wrote stock minus one, and under load several requests read the same value.
How to solve it
Replace the read-then-write with one atomic, guarded statement inside the order transaction: UPDATE products SET stock = stock - 1 WHERE id = ? AND stock >= 1. If it affects zero rows, the item is sold out, so roll back and tell the customer. For multi-item carts, lock the product rows with SELECT ... FOR UPDATE in ascending id order to avoid deadlocks, verify all quantities, then update. Add a CHECK (stock >= 0) constraint so the database rejects negative stock even if a future code path forgets the guard. Retry on deadlock or serialization errors a few times with a short backoff.
A report query brings the production database to its knees
Problem
A monthly sales report joins orders, order items and products over two years of data. When the finance team runs it, the database CPU hits 100% and checkout slows down for everyone.
How to solve it
First, separate the workloads: run reports on a read replica so analytics cannot starve transactional queries. Then optimise the query: run EXPLAIN ANALYZE, add a composite index matching the date filter and join keys, and select only the needed columns. For reports over closed periods, precompute: a nightly job writes daily totals per product into a summary table, and the report aggregates a few thousand summary rows instead of millions of items. Set a statement_timeout for the reporting user. If reporting needs keep growing, move them to a columnar store such as ClickHouse, fed from the main database.
Adding an index to a 200-million-row table without downtime
Problem
A slow query needs a new index on a very large, busy orders table. A previous attempt with a normal CREATE INDEX blocked writes for twenty minutes and caused an outage.
How to solve it
In PostgreSQL, use CREATE INDEX CONCURRENTLY, which builds the index while allowing reads and writes; it takes longer and cannot run in a transaction, so run it outside the migration framework's transaction wrapper. Set lock_timeout so the brief locks at start and end fail fast rather than queueing traffic. If it fails, an INVALID index remains: drop it and retry. In MySQL 8, use ALGORITHM=INPLACE, LOCK=NONE, or an online schema change tool such as gh-ost for very large tables. Schedule it at low traffic, watch replication lag on replicas, and confirm with EXPLAIN that the planner actually uses the new index afterwards.
Cheat sheet
- Real plan with timings and buffers (PostgreSQL)
EXPLAIN (ANALYZE, BUFFERS) SELECT ...;- Real plan as a tree (MySQL 8)
EXPLAIN ANALYZE SELECT ...; -- or EXPLAIN FORMAT=TREE- Top queries by total time (PostgreSQL)
SELECT query, calls, total_exec_time FROM pg_stat_statements ORDER BY total_exec_time DESC LIMIT 10;- Build an index without blocking writes
CREATE INDEX CONCURRENTLY idx_name ON tbl (col);- Fail fast instead of queuing behind locks
SET lock_timeout = '5s';- Keyset next page
WHERE (created_at, id) < ($1, $2) ORDER BY created_at DESC, id DESC LIMIT 50- Atomic guarded decrement
UPDATE products SET stock = stock - 1 WHERE id = $1 AND stock >= 1;- Insert or update in one statement (PostgreSQL)
INSERT ... ON CONFLICT (sku) DO UPDATE SET price = EXCLUDED.price;- Insert or update in one statement (MySQL)
INSERT ... AS new ON DUPLICATE KEY UPDATE price = new.price;- Enable MySQL slow query log
SET GLOBAL slow_query_log = ON; SET GLOBAL long_query_time = 0.5;
In my real projects
- I work with MySQL and MariaDB on the Laravel platforms and PostgreSQL (through Prisma) on the multi-tenant POS SaaS, whose schema has 181 models.
- In that SaaS, every query carries an explicit tenant id, and a Prisma extension throws if the tenant scope is missing, so data cannot leak between companies.
- On the restaurant ordering platform I built around 30 reporting controllers and about 40 Excel exports, from sales and promotions to an order-timing heatmap.
- The platforms I work on rely on aggregate reporting: daily-refreshed customer segments on the restaurant platform, and reorder, stock-valuation and dead-stock reports on the pet-supplies platform.
Questions & answers
Q1What is the difference between WHERE and HAVING?
They filter at different stages. WHERE runs before grouping and filters individual rows, so it cannot use aggregate functions. HAVING runs after GROUP BY and filters whole groups, so it can use COUNT, SUM and similar. For example, to find customers with more than five paid orders, you filter status = 'paid' in WHERE, group by customer, then keep groups with HAVING COUNT(*) > 5. Putting row-level conditions in WHERE rather than HAVING is also faster, because the database discards rows before doing the expensive grouping work. Knowing the logical order of a query (FROM, WHERE, GROUP BY, HAVING, SELECT, ORDER BY) answers most questions like this.
Q2How does an index speed up a query, and what does it cost?
A typical index is a B-tree: a sorted, balanced tree of key values pointing to rows. Instead of scanning every row, the database descends a few levels of the tree to find a value or the start of a range, then reads forward in order, which also helps ORDER BY and LIMIT. For a composite index, the leftmost-prefix rule applies: an index on (customer_id, created_at) helps filters on customer_id but not on created_at alone. The costs are disk space and slower writes, because every INSERT, UPDATE and DELETE must update each index. So I index for actual query patterns, verified with EXPLAIN, rather than every column.
Q3What is an N+1 query problem and how do you fix it?
It happens when you run one query to load a list and then one extra query per item to load related data: 1 query for 50 orders plus 50 queries for their customers. Each query is fast, so it looks fine in development, but in production the round-trips add up and the database gets hammered. ORMs make it easy because accessing order.customer silently triggers a query. The fix is eager loading, such as Laravel's with('customer') or Prisma's include, which loads all related rows in one or two queries, or an explicit JOIN. I also enable lazy-loading warnings or a query counter in development so regressions are caught early.
Q4Explain transaction isolation levels and when you would change the default.
Isolation decides what one transaction sees of others running at the same time. READ COMMITTED, PostgreSQL's default, sees only committed data, but a second read can return a newer value. REPEATABLE READ, MySQL InnoDB's default, gives a consistent snapshot for the whole transaction. SERIALIZABLE guarantees the result is the same as running transactions one after another, at the cost of aborting some, which the application must retry. I rarely change the default globally. Instead, I protect specific critical sections, like wallet balances or stock, with atomic conditional updates or SELECT ... FOR UPDATE, and use SERIALIZABLE with a retry loop only for complex invariants.
Q5When would you use a window function instead of GROUP BY?
When you need an aggregate or ranking but still want to keep every row. GROUP BY collapses rows: you get one row per customer with their total. A window function adds a calculated column while keeping the detail: every order alongside the customer's running total, its rank, or the date of the previous order. Typical uses are 'latest record per group' with ROW_NUMBER, top-N per category, running totals, period-over-period comparisons with LAG, and percent of total. Before window functions, these needed self-joins or correlated subqueries, which were harder to read and often slower. All major databases support them now.
Q6A page is slow and you suspect the database. How do you investigate?
First measure instead of guessing: enable the query log or use a debugging tool to see every query the request runs, with timings and the total count. If there are dozens or hundreds of similar queries, it is N+1, fixed with eager loading. If one query dominates, run EXPLAIN ANALYZE on it and look for full table scans on big tables, sorts on large sets and poor row estimates. The usual fixes are a suitable composite index, rewriting a condition so it can use the index, selecting fewer columns, or paginating. Then measure again, and keep slow-query logging on in production to catch the next one.
Senior-level questions
S1How do you scale a relational database when one server is no longer enough?
In steps, from cheapest to most complex. First make each query efficient: indexes, fewer round-trips, no N+1, connection pooling with PgBouncer or ProxySQL. Then cache hot, rarely changing reads in Redis. Next, add read replicas and send reporting and read-heavy endpoints there, accepting replication lag; reads that must see the user's own write stay on the primary. Vertical scaling, a bigger machine, is underrated and often buys years. Partitioning large tables by time keeps indexes small and makes archiving cheap. Only after all that do I consider sharding by tenant or customer, because it complicates transactions, joins, migrations and operations enormously. A multi-tenant design with a tenant id on every row keeps that option open without paying for it early.
S2UUIDs or auto-increment integers as primary keys: what are the trade-offs?
Auto-increment integers are small, fast to compare and insert in order, which keeps B-tree indexes compact; but they reveal record counts, are guessable in URLs, and need coordination when several systems generate ids. Random UUIDs (v4) can be created anywhere, including offline clients, and do not leak business volume, but they are 16 bytes and insert at random positions, causing page splits and poor cache locality in clustered indexes like InnoDB's. Time-ordered UUIDv7 solves most of that: globally unique, generated anywhere, and roughly sequential, so inserts stay at the end of the index. PostgreSQL 18 ships a uuidv7() function. My usual choice is an integer or UUIDv7 primary key internally, with a separate opaque public id where URLs must not be guessable.
S3How do you keep data consistent between the database and other systems, such as a queue or a payment provider?
You cannot put an external system inside a database transaction, so you design for partial failure. For events, I use the transactional outbox: the business change and an outbox row are written in the same transaction, and a separate worker publishes outbox rows to the queue and marks them sent. Consumers must be idempotent, because at-least-once delivery means duplicates. For payments, I record the intent first with a unique idempotency key, call the provider with that key, and reconcile through webhooks and a periodic job that checks pending records, so a timeout never leaves money in an unknown state. Unique constraints on external ids make duplicate webhooks harmless. Distributed transactions like two-phase commit exist, but they are rarely worth their operational cost in web systems.
Glossary
- MVCC
- Multi-version concurrency control: keeping several row versions so readers see a consistent snapshot without blocking writers.
- Dead tuple
- An old PostgreSQL row version no transaction needs any more, waiting for vacuum to reclaim its space.
- Covering index
- An index containing every column a query needs, so the table itself is never read.
- Partial index
- An index on only the rows matching a condition, keeping it small for selective queries.
- Selectivity
- How few rows a condition matches; highly selective conditions benefit most from indexes.
- Keyset pagination
- Paging by remembering the last row seen and fetching rows after it, instead of using OFFSET.
- Deadlock
- Two transactions each waiting for a lock the other holds; the database aborts one to break the cycle.
- Gap lock
- An InnoDB lock on a range between index records that prevents inserts into that range.
- Read replica
- A copy of the database that receives changes from the primary and serves read-only queries.
- Planner statistics
- Collected data about value distribution that the optimiser uses to estimate rows and choose a plan.
Keep learning
Have a product in mind?Let's build it.
Tell me about your idea, your product or your team. I usually reply within one working day.