All lessons
DevOps & cloudIntermediate15 min readWorth knowing

AWS

The biggest cloud: hundreds of services, a few that matter, and a bill to watch.

  • EC2
  • S3
  • RDS
  • Lambda
  • CloudFront
  • VPC & IAM
Step 1/5
One VPS: cheap and simple, but you're also the admin, the backup guy and the 3 a.m. alarm clock.

In 30 seconds

Amazon Web Services rents you computing building blocks by the second: virtual servers, storage, databases, networking, queues and much more. You pay only for what you use and can grow from one server to thousands. The power comes with complexity: you must design networking and permissions yourself, and costs can surprise you if nobody watches them.

Not in my daily stack — I know the concepts and would learn the details when a project needs it.

Why it matters

  • AWS matters because it is the most widely used cloud, so many companies, job descriptions and integrations assume it.
  • A team needs it when it outgrows single servers: managed databases with automatic failover, global CDNs, autoscaling and many regions are hard to build alone.
  • Its managed services trade server maintenance for configuration, which can free a small team from patching and backups.
  • For my own projects I mostly run VPS servers with Docker Compose, which is cheaper and simpler at that scale; knowing AWS well is what lets me judge when that trade-off flips.

How it works

01Regions, availability zones and shared responsibility

AWS is split into regions, separate geographic areas such as Frankfurt or Ireland, each with its own copy of most services. A region contains several availability zones (AZs): one or more data centers with independent power, cooling and networking, close enough for fast connections. Running your app in at least two AZs protects you from one data center failing; using several regions protects against a whole region failing, at much higher cost and complexity. Choose a region close to your users and compliant with data rules. Security is shared: AWS is responsible for security of the cloud (hardware, data centers, the hypervisor), and you are responsible for security in the cloud: your IAM permissions, network rules, patching your EC2 servers, encrypting and not exposing your data.

  • Region = geographic area; AZ = isolated data center(s) inside it.
  • Highly available = spread across at least two AZs.
  • Misconfiguration on your side is the most common cause of cloud breaches.

02Core services and when to use them

You do not need hundreds of services; a dozen cover most web applications. Compute comes in three flavours: EC2 virtual machines you manage, containers on ECS (simpler, AWS-native) or EKS (managed Kubernetes), optionally serverless with Fargate, and Lambda functions that run on events with no servers at all. Data lives in S3 for files and RDS or Aurora for managed relational databases. CloudFront is the CDN in front, Route 53 is DNS and health-checked routing. VPC is your private network, IAM controls who may do what, and SQS is a managed queue for decoupling work.

  • EC2: full control, long-running servers, any software.
  • ECS / EKS (+ Fargate): containers without managing schedulers yourself.
  • Lambda: short event-driven tasks (up to 15 minutes), pay per invocation.
  • S3: object storage for uploads, backups, static sites; versioning and lifecycle rules.
  • RDS / Aurora: managed PostgreSQL or MySQL with backups, Multi-AZ failover, replicas.
  • CloudFront + Route 53: CDN and DNS; SQS: queues; VPC + IAM: network and permissions.

03Networking: a sensible VPC layout

A VPC (virtual private cloud) is your isolated network inside a region. You divide it into subnets, each in one availability zone. Public subnets have a route to an internet gateway and hold only things that must face the internet, typically the load balancer. Private subnets hold application servers and databases; they cannot be reached from the internet directly. Security groups act as stateful firewalls attached to each resource: the load balancer accepts 443 from anywhere, the app accepts traffic only from the load balancer's security group, the database only from the app's security group. Private resources that need outbound internet, for package updates or external APIs, use a NAT gateway, which is convenient but one of the classic cost surprises, so VPC endpoints for S3 and other AWS services are worth adding.

  • Public subnet: load balancer only.
  • Private subnet: app and database, reached via security-group references.
  • Reference security groups, not IP ranges, between your own tiers.

04IAM and least privilege

IAM (Identity and Access Management) decides who can call which AWS API on which resource. The principle of least privilege means every identity gets only the permissions its job needs. Protect the root account with MFA and never use it day to day. People should sign in through IAM Identity Center with temporary credentials, not long-lived access keys. Applications should use roles: an EC2 instance profile, an ECS task role or a Lambda execution role gives the code short-lived credentials automatically, so no keys sit in .env files. Write narrow policies like the example, which lets an app read and write only one bucket prefix. Use IAM Access Analyzer to find unused or overly broad permissions, and enable CloudTrail to record every API call.

json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject"],
      "Resource": "arn:aws:s3:::example-app-uploads/uploads/*"
    }
  ]
}

05Cost traps and how to avoid them

AWS bills are rarely high because of one big server; they grow from small items nobody watches. NAT gateways charge per hour and per gigabyte processed. Data transfer out to the internet and between availability zones adds up quietly. Every public IPv4 address is billed hourly. Unattached EBS volumes, old snapshots, idle load balancers and forgotten test environments keep charging. CloudWatch log ingestion can cost more than the servers that write the logs. RDS Multi-AZ roughly doubles database cost, which is often right, but should be a conscious choice. Defend yourself from the first day: set AWS Budgets alerts, tag every resource with project and environment, review Cost Explorer monthly, set log retention, and use Savings Plans or reserved capacity for stable workloads.

bash
aws budgets create-budget --account-id 111122223333 \
  --budget '{"BudgetName":"monthly","BudgetLimit":{"Amount":"200","Unit":"USD"},"TimeUnit":"MONTHLY","BudgetType":"COST"}' \
  --notifications-with-subscribers '[{"Notification":{"NotificationType":"ACTUAL","ComparisonOperator":"GREATER_THAN","Threshold":80},"Subscribers":[{"SubscriptionType":"EMAIL","Address":"[email protected]"}]}]'

06AWS or a VPS?

A VPS from DigitalOcean, Hetzner or similar gives you a fixed monthly price, generous included bandwidth and one simple control panel. For a single application, a well-run VPS with Docker Compose, off-site backups and monitoring is often several times cheaper than the equivalent on AWS and easier for a small team to understand. AWS wins when you need what is hard to build yourself: managed databases with automatic failover and point-in-time recovery, autoscaling for spiky traffic, many regions, compliance certifications that enterprise clients ask for, or deep integration with other AWS services. A common middle path is a VPS for compute plus a few targeted managed services, such as S3-compatible storage for uploads and a CDN. Decide by requirements and team skill, not by fashion.

Use it when

  • Traffic that changes a lot, where autoscaling and pay-per-use save real money.
  • When you need managed high availability: Multi-AZ databases, load balancers, failover DNS.
  • Enterprise or regulated clients that require specific compliance programmes or regions.
  • Event-driven workloads that fit Lambda, SQS and S3 triggers well.

Skip it when

  • A single small or medium app with steady traffic; a VPS is cheaper and simpler.
  • A team without anyone to own IAM, networking and cost; misconfiguration is the main risk.
  • Bandwidth-heavy workloads where data transfer pricing dominates the bill.

Alternatives

  • VPS providers (DigitalOcean, Hetzner)

    Predictable pricing, simple networking, ideal for one to a few servers.

  • Google Cloud

    Strong in Kubernetes (GKE), data and analytics; similar building blocks.

  • Microsoft Azure

    Popular with enterprises already on Microsoft identity and tooling.

  • Lightsail

    AWS's own VPS-style product with bundled pricing, a gentle entry point.

Best practices

  1. Lock the root account with MFA and use IAM Identity Center for people.
  2. Give applications IAM roles, never long-lived access keys in config files.
  3. Keep databases in private subnets and allow access only from the app's security group.
  4. Block public access on S3 buckets unless serving through CloudFront with origin access control.
  5. Set budgets and alerts on day one and tag every resource.
  6. Enable CloudTrail and keep infrastructure in code (Terraform or CloudFormation).

Common mistakes

  • Committing access keys to Git, where bots find and abuse them within minutes.
  • Making a bucket public to fix a permissions error quickly.
  • Putting the database in a public subnet with 0.0.0.0/0 in its security group.
  • Leaving test environments and NAT gateways running for months.
  • Using one AWS account for everything instead of separating production and experiments.

Deep dive

07How IAM decides allow or deny

Every AWS API call is evaluated against all policies that apply to it, and the logic is strict. Everything starts as an implicit deny. An explicit Deny in any applicable policy always wins. Within an account, a request is allowed only if some identity-based or resource-based policy allows it and nothing denies it. On top of that, guardrails can only take permissions away, never grant them: service control policies from AWS Organizations, permission boundaries on a role, and session policies must all allow the action as well. Conditions make policies precise: restrict by source VPC endpoint, require encryption in transit, require MFA, or limit to resources tagged for one project. When access fails, the error message and CloudTrail usually say which policy type denied it, and the IAM policy simulator can replay the decision.

json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyInsecureTransport",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": ["arn:aws:s3:::example-app-uploads", "arn:aws:s3:::example-app-uploads/*"],
      "Condition": { "Bool": { "aws:SecureTransport": "false" } }
    }
  ]
}

08Multi-account structure with AWS Organizations

An AWS account is the strongest isolation boundary AWS offers: separate permissions, separate quotas and a separate bill line. So mature setups use many accounts, managed through AWS Organizations, instead of one account for everything. A typical layout has a management account used only for billing and organisation settings, a log archive account that receives CloudTrail and config logs from every other account and that application teams cannot modify, a security account for audit tools, and separate workload accounts for production, staging and sandboxes. Service control policies at the organisation level enforce rules nobody can override inside an account, such as deny leaving the organisation, deny disabling CloudTrail, or deny any region outside the ones you use. People sign in once through IAM Identity Center and switch into roles per account. AWS Control Tower can set up this landing zone with sensible defaults.

  • Blast radius: a leaked staging credential cannot touch production.
  • Quotas and costs are separated per account and per team.
  • SCPs are guardrails: they limit, but never grant, permissions.

09EC2 access done right: IMDSv2 and Session Manager

Every EC2 instance can read its own metadata, including the temporary credentials of its instance role, from a local address, 169.254.169.254. In the original IMDSv1, any simple GET request worked, so a server-side request forgery bug in the app, one that makes the server fetch a URL an attacker supplies, could leak those credentials. That is how several well-known breaches happened. IMDSv2 requires a session token obtained with a PUT request and a hop limit, which blocks most SSRF paths; require it on every instance, and new instance types and AMIs increasingly default to it. For shell access, AWS Systems Manager Session Manager opens a session through the SSM agent, authenticated by IAM, logged to CloudTrail, with no inbound port 22, no bastion host and no SSH keys to rotate. Port forwarding through it also reaches private databases for debugging.

bash
# require IMDSv2 on an existing instance
aws ec2 modify-instance-metadata-options --instance-id i-0123456789abcdef0 \
  --http-tokens required --http-endpoint enabled
 
# shell without SSH or open ports
aws ssm start-session --target i-0123456789abcdef0
 
# tunnel to a private RDS instance through that host
aws ssm start-session --target i-0123456789abcdef0 \
  --document-name AWS-StartPortForwardingSessionToRemoteHost \
  --parameters '{"host":["db.internal.example.com"],"portNumber":["5432"],"localPortNumber":["5432"]}'

10Where network money goes, and how to keep it

Networking is where AWS bills surprise people, because traffic is priced by path. Data from the internet into AWS is free. Data out to the internet is charged per gigabyte, and CloudFront's rates are usually lower than serving directly from EC2 or S3, with a free monthly allowance, so a CDN often saves money as well as latency. Traffic between availability zones is charged in both directions, so chatty services in different zones, or replicas in other zones, have a real cost. A NAT gateway charges per hour and per gigabyte processed, on top of normal transfer, so private servers pulling large images or backups through NAT pay twice. Gateway VPC endpoints for S3 and DynamoDB are free and remove that traffic from the NAT path; interface endpoints for other services cost per hour but can still be cheaper. Every public IPv4 address is billed hourly too.

  • Add the free S3 gateway endpoint to every VPC with private subnets.
  • Serve public assets through CloudFront, not straight from S3 or EC2.
  • Use Cost Explorer grouped by usage type to find NAT and transfer costs.

Real-world scenarios

01

An access key was pushed to a public repository

Problem

GitHub sends an alert that an AWS access key appeared in a public commit twenty minutes ago. The key belongs to an IAM user with broad permissions that a script used for deployments.

How to solve it

Deactivate the key immediately in IAM, then delete it once nothing legitimate depends on it. Assume it was used: search CloudTrail for the key ID across all regions, looking for new IAM users or keys, instances launched in unusual regions (crypto mining is typical), changed security groups and S3 reads. Remove anything the attacker created, including new access keys or roles. Check the bill and any AWS Health notification about the exposure. Then fix the cause: replace the IAM user with a role, for CI through OIDC, enable secret scanning with push protection, and add an SCP that denies regions you never use.

02

The monthly bill doubled without new features

Problem

Finance asks why the AWS bill went from about 900 to 1,800 dollars. Traffic grew only slightly and nobody remembers launching anything big.

How to solve it

Open Cost Explorer, group by service, then by usage type, and compare with the previous month. Common culprits: NAT gateway bytes processed after a job started pulling data from S3 through NAT, cross-AZ transfer from a new replica, CloudWatch Logs ingestion after debug logging was enabled, or a forgotten test environment. Fix each with the right tool: a free S3 gateway endpoint, colocating chatty services, log levels and retention, deleting idle resources. Then prevent recurrence: Cost Anomaly Detection with alerts, budgets per account, mandatory project and environment tags, and a short monthly cost review owned by a named person.

03

Moving an app from a VPS to AWS step by step

Problem

A growing SaaS runs on one VPS. An enterprise client now requires a managed database with automatic failover and a documented recovery process, but a big-bang migration feels risky.

How to solve it

Move one layer at a time, each with its own rollback. First, uploads: copy files to S3 with aws s3 sync, switch the app's storage driver, and serve them through CloudFront. Second, the database: create RDS Multi-AZ in a private subnet, replicate from the VPS with logical replication or AWS DMS, then cut over in a short, announced window. Third, compute: containerise if needed and run the app on ECS with Fargate across two zones behind an ALB, while DNS can still point back to the VPS. Define infrastructure in Terraform from the start, and keep the VPS until a full month of stable operation.

Cheat sheet

Which identity am I using?
aws sts get-caller-identity
Sign in with IAM Identity Center
aws sso login --profile prod-admin
Temporary download link for a private file
aws s3 presign s3://example-app-uploads/report.pdf --expires-in 600
Follow a Lambda's logs live
aws logs tail /aws/lambda/process-upload --follow
List running instances compactly
aws ec2 describe-instances --filters Name=instance-state-name,Values=running --query "Reservations[].Instances[].[InstanceId,InstanceType,Placement.AvailabilityZone]" --output table
Require IMDSv2 on an instance
aws ec2 modify-instance-metadata-options --instance-id i-0123456789abcdef0 --http-tokens required
Shell into an instance without SSH
aws ssm start-session --target i-0123456789abcdef0
Set log retention so logs stop growing
aws logs put-retention-policy --log-group-name /app/prod --retention-in-days 30
Check bucket public access block
aws s3api get-public-access-block --bucket example-app-uploads

Questions & answers

Q1Explain the AWS shared responsibility model.

AWS secures the cloud itself: physical data centers, hardware, the network backbone and the virtualisation layer, plus the underlying infrastructure of managed services. The customer secures everything they put in the cloud: IAM users and roles, security groups, encryption choices, data classification, and the operating system and software on EC2 instances. The split moves with the service: on EC2 you patch the OS; on RDS AWS patches the database engine but you still control network access and credentials; on Lambda you only own your code and its permissions. Most real breaches come from the customer side, typically public buckets or leaked keys.

Q2When would you choose EC2, ECS with Fargate, or Lambda?

EC2 when I need full control of the machine, long-running processes, special software or the lowest cost for steady load, accepting that I patch and manage it. ECS with Fargate when the app is already containerised and I want AWS to run the containers without managing hosts; it suits APIs, workers and web apps with steady or moderately variable traffic. Lambda for event-driven, short tasks: image processing on S3 upload, webhooks, scheduled jobs, spiky APIs. Lambda's per-request pricing is excellent at low or bursty volume, but long-running jobs, heavy constant traffic and cold-start-sensitive paths are usually cheaper or better on containers.

Q3How would you secure an S3 bucket that stores user uploads?

Keep Block Public Access enabled at account and bucket level, and leave the default bucket-owner-enforced object ownership so ACLs are disabled. The application gets an IAM role with a policy limited to the needed actions on that bucket or prefix. Users upload and download through short-lived presigned URLs generated by the backend, or through CloudFront with origin access control for public assets. Encryption at rest is on by default; use KMS keys if you need audit or key control. Enable versioning and lifecycle rules, so accidental deletes are recoverable and old versions expire, and log access with CloudTrail data events if required.

Q4IAM user or IAM role: what is the difference?

An IAM user is a long-lived identity with permanent credentials, a password or access keys. A role has no permanent credentials; a trusted principal, such as an EC2 instance, a Lambda function, a user from another account or a federated person, assumes it and receives temporary credentials that expire automatically. Best practice today is to avoid IAM users almost entirely. People sign in through IAM Identity Center or an external identity provider and assume roles; workloads use instance profiles, task roles or execution roles. That removes long-lived keys, which are the most common source of leaks, and makes access easy to audit and revoke.

Q5How do you make an application highly available on AWS?

Remove single points of failure inside a region first. Run at least two app instances or tasks in different availability zones behind an Application Load Balancer, with health checks so unhealthy targets are replaced. Use RDS or Aurora with Multi-AZ so the database fails over automatically, and keep sessions and cache in a managed Redis-compatible service rather than on the instances. Store files in S3, not local disk. Put CloudFront in front for static content. Autoscaling keeps capacity matched to load. Multi-region setups add Route 53 failover and data replication, but they cost far more, so they are justified only by real business requirements.

Q6Why might a company pay more on AWS than on a VPS, and how do you control it?

Because AWS charges for many small things separately: NAT gateway hours and processed data, data transfer out and across zones, public IPv4 addresses, load balancers, log ingestion, snapshots and idle volumes. Over-provisioned instances and forgotten environments add more. A VPS bundles compute and generous bandwidth into one price. To control AWS cost: set budgets and anomaly alerts, tag resources by project and environment, right-size instances using real metrics, use Graviton instances and Savings Plans for stable load, add VPC endpoints to cut NAT traffic, set log retention, and delete what nobody owns. Review the bill monthly as a team habit.

Senior-level questions

S1Design a highly available web application on AWS for a mid-sized budget. Where do you compromise?

Two availability zones, not three, which covers a data center failure at lower cost. An Application Load Balancer in public subnets, ECS on Fargate tasks in private subnets, at least two tasks spread across zones, autoscaling on CPU or request count. RDS PostgreSQL Multi-AZ for automatic failover, with automated backups and PITR, plus ElastiCache for sessions and cache. S3 with CloudFront for assets and uploads. The compromises: one NAT gateway instead of one per zone, accepting that a zone failure breaks outbound calls from the other zone until fixed, mitigated by S3 gateway endpoints; Graviton instances and a Compute Savings Plan; a single region, with cross-region backup copies for disaster recovery rather than active-active. Each compromise is documented with its risk so the business can accept it explicitly.

S2SQS, SNS, EventBridge or Kinesis: how do you choose?

SQS is a queue: one message is processed by one consumer, with retries, visibility timeouts and dead-letter queues. Use it to buffer work for a worker pool. FIFO queues add ordering per group and deduplication at lower throughput. SNS is publish-subscribe fan-out: one message is pushed to many subscribers, often SQS queues, so each consumer gets its own durable copy. EventBridge is an event bus with content-based routing rules, schema registry, archive and replay, plus integrations with SaaS and AWS service events; it suits event-driven architectures where many teams react to domain events, at higher latency and per-event cost. Kinesis Data Streams is an ordered, replayable log partitioned by shard key, closest to Kafka, for high-volume streams like clickstreams or telemetry that several consumers read independently and at their own pace.

S3Explain the AWS disaster recovery strategies and how you would pick one.

AWS describes four, in increasing cost and decreasing recovery time. Backup and restore: data is backed up to another region and infrastructure is recreated from code when needed; RTO in hours, lowest cost. Pilot light: core data is replicated continuously, for example a cross-region database replica, while compute is defined but switched off; RTO in tens of minutes. Warm standby: a scaled-down but fully working copy runs in the second region and is scaled up during failover; RTO in minutes. Multi-site active-active: both regions serve traffic, with near-zero RTO but the hardest problems, especially data consistency and conflict handling. The choice follows the business's RPO and RTO and the cost of downtime. Most companies are well served by backup and restore or pilot light, provided the failover is rehearsed and infrastructure is in code.

S4At what point is Lambda more expensive than containers?

Lambda charges per request and per gigabyte-second of execution, with nothing for idle time. For spiky or low traffic, that is hard to beat: a function invoked a few thousand times a day costs almost nothing, scales instantly and needs no capacity planning. As traffic becomes high and steady, you pay for every millisecond of every request, while a container on Fargate or EC2 serves many concurrent requests on capacity you pay for once, especially with Savings Plans. The crossover depends on duration, memory and concurrency, so model it with real numbers. Other factors matter too: long tasks over fifteen minutes, cold-start-sensitive latency, connection limits to relational databases (often needing RDS Proxy), and the operational cost of running containers. Many systems mix both: containers for the core API, Lambda for events and glue.

Glossary

ARN
Amazon Resource Name: the unique identifier of any AWS resource, used in policies to say exactly what is allowed.
Principal
Who makes a request: a user, role, AWS service or federated identity that policies allow or deny.
SCP
Service control policy: an organisation-level guardrail limiting the maximum permissions available in member accounts.
Instance profile
The container that attaches an IAM role to an EC2 instance, giving its software temporary credentials automatically.
IMDSv2
The token-based version of the EC2 instance metadata service, which blocks most credential theft through SSRF bugs.
NAT gateway
A managed service letting private-subnet resources reach the internet outbound; billed per hour and per gigabyte.
VPC endpoint
A private connection from your VPC to an AWS service, so traffic does not go over the internet or NAT.
Multi-AZ
A deployment with a standby copy in another availability zone that takes over automatically if the primary fails.
Savings Plan
A one- or three-year commitment to a set hourly spend in exchange for a large discount on compute.
Egress
Data leaving a network, such as AWS to the internet; usually the transfer direction that costs money.

Have a product in mind?Let's build it.

Tell me about your idea, your product or your team. I usually reply within one working day.