DEV Community: Oluwasegun Adedigba

Non-functional Application Requirements: Sustainability

Oluwasegun Adedigba — Wed, 11 Nov 2026 15:03:45 +0000

On 7 July 2026, Ireland's statistics office published a number worth sitting with. In 2025, data centers used 23% of the country's metered electricity. That's more than every urban home in Ireland combined, at 18% (CSO, 2026).

That same summer, Amazon's 2025 sustainability report said its absolute emissions rose 16% in a year. The same report says Amazon matched 100% of its electricity with renewable energy, for the third year running.

Both are true, because they measure different things. If you can't tell which is which, you can't tell whether your own architecture choices helped.

So this finale is built around a panel. I didn't interview anyone. I took the questions I'd ask and answered them from what six experts have already published.

TL;DR

Sustainability is energy, carbon and water per unit of useful work. Track a resource proxy (vCPU-minutes per transaction) weekly and carbon monthly.

Know which carbon number you're reading: location-based or market-based, and which scope.

AWS owns sustainability of the cloud. You own sustainability in it: fewer resources, busy for less time, on efficient chips, in a well-chosen Region.

AWS carbon data is monthly and stops at account, Region and service. Nothing gives you carbon per model call.

For AI, inference is the part you control. The smallest adequate model, caching and batch cut cost and carbon together. Sometimes the right model is a regex.

What: sustainability you can test

Sustainability is the requirement to keep the environmental cost of each unit of useful work low for the life of the workload. Three quantities get mixed up.

Energy is the kilowatt-hours your resources draw, busy or idle. Carbon is that energy times the grid's carbon intensity, plus the emissions from making the hardware. Water goes into cooling and power generation. A cleaner grid cuts carbon without saving a kilowatt-hour, so report them separately.

The GHG Protocol sorts emissions into scopes: Scope 1 is fuel burned on site, Scope 2 is bought electricity, Scope 3 is everything else, including making the servers. What AWS emits running your workload lands in your Scope 3. Then there's method. Location-based carbon uses the average grid where the power was used. Market-based carbon credits the renewable contracts a company buys. "100% matched" is a market-based claim.

To make it testable, the Green Software Foundation's Software Carbon Intensity (SCI) spec, now ISO/IEC 21031:2024, gives you a rate:

SCI = ((E × I) + M) per R

E is energy, I is grid carbon intensity, M is the hardware's share of embodied emissions and R is your functional unit, like one API call. Offsets and renewable certificates can't lower it. Only cutting emissions can.

flowchart LR
  E["E<br/>energy"] --> EI["E x I<br/>carbon from the energy used"]
  I["I<br/>grid carbon intensity"] --> EI
  EI --> T["Total emissions"]
  M["M<br/>hardware share of<br/>embodied emissions"] --> T
  T -->|"per R"| SCI["SCI<br/>carbon per functional unit<br/>like one API call"]
  X["Offsets and renewable<br/>certificates"] -.->|"can't lower it"| SCI

The score only falls when energy, grid intensity or embodied emissions per unit of work fall.

You won't know E for a managed service, so AWS Well-Architected suggests a proxy KPI: resources provisioned per unit of work, like vCPU-minutes per transaction. It moves within days of a change. The carbon report catches up weeks later.

Why: the grid can't keep up

Data center demand is outgrowing the grids that feed it. The IEA's April 2025 Energy and AI report put data centers at about 415 TWh in 2024, roughly 1.5% of global electricity. Its 2026 follow-up, Key Questions on Energy and AI, says demand grew 17% in 2025 to about 485 TWh, against 3% for electricity overall, with AI-focused data centers up 50%. It expects roughly 950 TWh by 2030.

Hardware matters too. Amazon's report puts supply-chain emissions, including data center construction and hardware, at 76% of its footprint, up 20% in a year. Every idle server you keep carries a slice of that.

The good news: almost every sustainability lever is also a cost lever, so savings fund the work.

The panel: six published answers

Each answer paraphrases published work, with a link and at most one short quote.

Where do I start? Adrian Cockcroft, former VP of Sustainability Architecture at Amazon. His QCon London 2023 talk boils developer advice down to "use fewer computers, and use them for less time" (InfoQ, 2023). He prefers location-based data for tuning, and AWS has published it since June 2025.

How do I measure honestly? Asim Hussain, executive director of the Green Software Foundation until March 2026. The SCI is built so a score falls only when emissions do. In a 2023 interview he called carbon-aware scheduling a cheap way to save a few percent, not the big win. He also says green software is "more efficient, which translates almost always to being faster" (MIT Technology Review, 2023).

Which practices matter? Anne Currie, Sarah Hsu and Sara Bergman, authors of Building Green Software (O'Reilly, 2024). Their chapters give you three buckets for any change (O'Reilly): code efficiency (less compute per task), hardware efficiency (fewer devices, kept longer) and carbon awareness (run work when and where the grid is clean).

Training or inference? David Patterson and Jeff Dean, Google. Their 2022 paper reports inference at about three-fifths of Google's machine-learning energy (arXiv, 2022). A Meta team led by Carole-Jean Wu saw a similar 10:20:70 split across experimentation, training and inference. Patterson and Dean also co-authored Google's 2025 Gemini study: 0.24 Wh per median text prompt, after a 33x drop in a year.

Does model choice matter? Sasha Luccioni, then at Hugging Face, now co-founder of Sustainable AI Group. With Yacine Jernite and Emma Strubell, she measured 1,000 inferences across many tasks. General-purpose generative models were "orders of magnitude more expensive than task-specific systems", even at similar sizes (ACM FAccT, 2024). Her AI Energy Score now rates models one to five stars per task.

Is it getting better? Fatih Birol, Executive Director of the IEA. He puts it plainly: "there is no AI without energy" (IEA, 2026). The IEA also says energy per AI task has fallen by at least an order of magnitude a year. Total demand still climbs, because there are far more calls.

When: one check per stage

Each stage gets one sustainability check:

Design. Shortlist Regions on residency, cost and latency, then pick the lowest-carbon one left, as Well-Architected's SUS01-BP01 advises. Define your proxy KPI, and ask whether the feature needs an LLM.
Build. Default to arm64. Batch instead of trickling. Give every table, bucket and log group an expiry on day one.
Test. Run Lambda Power Tuning, and record the proxy KPI next to p95.
Operate. Read the proxy weekly and the carbon report monthly. Delete what's idle.

Where: your levers on AWS

Well-Architected splits it. AWS handles sustainability of the cloud: infrastructure, water, renewable power. You handle sustainability in the cloud: your workloads, utilization and how much you deploy. Everything below is yours.

flowchart LR
  AWS["AWS<br/>sustainability of the cloud"] --> A1["Infrastructure, water<br/>and renewable power"]
  YOU["You<br/>sustainability in the cloud"] --> Y1["Workloads, utilization<br/>and how much you deploy"]
  Y1 --> L1["Lambda on arm64"]
  Y1 --> L2["Graviton on EC2, ECS and EKS"]
  Y1 --> L3["DynamoDB on-demand with TTL"]
  Y1 --> L4["S3 lifecycle and Intelligent-Tiering"]
  Y1 --> L5["AWS Sustainability console and API"]

AWS owns the infrastructure, water and power, and every lever in this section sits on your side of the line.

Lambda on arm64. For most Node.js and Python functions it's one line. AWS's 2021 launch claim was up to 19% better performance at 20% lower cost. Native binaries without arm64 builds will block you, and Compute Optimizer only sizes memory for x86_64 functions.

Graviton on EC2, ECS and EKS. AWS says Graviton instances use up to 60% less energy than comparable EC2 instances for the same performance. That's a vendor ceiling with no stated baseline, so benchmark your own workload.

DynamoDB on-demand with TTL. On-demand removes idle capacity. TTL deletes expired items without using write throughput, typically within a few days, so filter them in reads. On global tables, replicated deletes do cost write units.

S3 lifecycle and Intelligent-Tiering. Objects under 128 KB never tier down. AWS doesn't publish energy per storage class, so tiering is a proven cost lever but only an assumed carbon one. Expiration is the lever you can count on. Deleted bytes need no disks.

The AWS Sustainability console and API. The Customer Carbon Footprint Tool added Scope 3 (hardware and buildings, fuel and energy, upstream transport) and Scope 1 data in October 2025. AWS deprecated it on 30 June 2026 for the AWS Sustainability console, which keeps the methodology but has its own permissions, so analysts don't need Billing access (CCFT release notes). The limits matter: monthly at best, published between the 15th and 21st of the next month, by account, Region and service only.

Open source fills the gaps. The Green Software Foundation's Carbon Aware SDK reads grid data from WattTime or Electricity Maps so you can shift work in time or place. Lambda Power Tuning finds the cheapest or fastest memory size. It measures money and time, not watts, so GB-seconds are your proxy.

Decide: which lever, and what it costs elsewhere

I'd start with the certain, cheap levers and work down:

Lever	Carbon effect	Cost effect	Main risk
Expire data, delete idle resources	Large and certain	Saves	Deleting something you needed. Use TTL and lifecycle rules, not scripts.
Right-size memory and instances	Large	Saves	Too small hurts p95
Move to arm64 or Graviton	Medium to large, per AWS	Usually saves	Dependencies without arm64 builds
Batch and queue	Medium	Saves	Added latency
Run batch work in a low-carbon Region	Large for that workload	Neutral to higher	Residency, more Regions to run
Time-shift to clean hours	Small, a few percent per Hussain	Neutral	Complexity

Region choice is where sustainability collides with two requirements at once. Picture a team serving users in Lagos. The closest AWS compute is the Lagos Local Zone, whose parent Region is Africa (Cape Town). Ember reports that South Africa generated 18% of its electricity from low-carbon sources in 2025, against a 43% global average. Sweden, home to Europe (Stockholm), managed 99%. And section 41 of Nigeria's Data Protection Act 2023 lets personal data leave the country only when the recipient offers adequate protection, unless another basis in the Act applies.

Cleanest, closest and legal are three different answers.

flowchart LR
  C["Cleanest grid<br/>Stockholm eu-north-1<br/>99% low-carbon in 2025"] ---|"long way from Lagos users"| L["Closest to users<br/>Lagos Local Zone<br/>parent Region Cape Town"]
  L ---|"you inherit an 18% low-carbon grid"| D["Where the law allows<br/>NDPA 2023 section 41<br/>adequacy test"]
  D ---|"may rule Stockholm out for personal data"| C
  B["Batch jobs, evals,<br/>fine-tuning"] -.->|"free to move"| C
  I["Interactive APIs<br/>and chat"] -.->|"stay close"| L
  P["Personal or<br/>regulated data"] -.->|"stays put"| D

Each workload picks its own corner: move what nobody waits for, and keep regulated and interactive paths where they must be.

So decide per workload. Here's how sustainability trades against the other eight requirements:

NFR	Where sustainability helps	Where it hurts
Security	Fewer resources, less attack surface	Scanning, encryption and logging use compute
Compliance	Data minimization serves both	Residency can rule out the cleanest Regions
Accessibility	Lean pages work on low-end devices	Rarely in conflict
Performance	Efficient code is usually faster	Always-warm capacity burns energy idle
Availability	Rarely helps	Redundancy multiplies resources
Scalability	Scale-to-zero removes idle	Headroom you never use
Maintainability	Fewer moving parts	Carbon-driven multi-Region adds parts
Cost	Mostly aligned	A cheap token isn't always a low-energy token

Hands-on: a lean ingest stack for Harbor Works

Let's put the certain levers into code. Harbor Works is the fictional plant from Luso, this series' companion app, and its production lines send batches of inspection events. The stack uses an arm64 Lambda, on-demand DynamoDB with TTL, an S3 rule that tiers then expires raw batches, and two-week log retention.

Checked against the AWS CDK v2 API reference for lambda.Function, dynamodb.TableV2 and s3.LifecycleRule on Oct 29, 2026, type-checked in strict mode and synthesized with aws-cdk-lib 2.275.0. Not deployed, so test it in a sandbox account first.

import { App, Stack, StackProps, Duration, RemovalPolicy, CfnOutput } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as logs from 'aws-cdk-lib/aws-logs';
import * as dynamodb from 'aws-cdk-lib/aws-dynamodb';
import * as s3 from 'aws-cdk-lib/aws-s3';

export class LeanIngestStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    // Hot data. Inspection events expire through TTL, which uses no write throughput.
    const events = new dynamodb.TableV2(this, 'LineEvents', {
      partitionKey: { name: 'lineId', type: dynamodb.AttributeType.STRING },
      sortKey: { name: 'ts', type: dynamodb.AttributeType.STRING },
      billing: dynamodb.Billing.onDemand(),  // no idle provisioned capacity
      timeToLiveAttribute: 'expiresAt',      // epoch seconds, set by the writer
      removalPolicy: RemovalPolicy.DESTROY,  // sandbox only, use RETAIN in prod
    });

    // Cold data. Raw batches tier down on their own, then get deleted.
    const archive = new s3.Bucket(this, 'RawArchive', {
      encryption: s3.BucketEncryption.S3_MANAGED,
      blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL,
      enforceSSL: true,
      lifecycleRules: [{
        id: 'tier-then-expire',
        prefix: 'raw/',
        transitions: [
          // Unknown access pattern: let Intelligent-Tiering decide.
          { storageClass: s3.StorageClass.INTELLIGENT_TIERING, transitionAfter: Duration.days(0) },
          // After a year it's only kept for audit.
          { storageClass: s3.StorageClass.DEEP_ARCHIVE, transitionAfter: Duration.days(365) },
        ],
        expiration: Duration.days(365 * 3),  // the rule that actually removes bytes
        abortIncompleteMultipartUploadAfter: Duration.days(7),
      }],
      removalPolicy: RemovalPolicy.DESTROY,
      autoDeleteObjects: true,               // sandbox only
    });

    // Logs are data too. Two weeks, not the default of forever.
    const logGroup = new logs.LogGroup(this, 'IngestLogs', {
      retention: logs.RetentionDays.TWO_WEEKS,
      removalPolicy: RemovalPolicy.DESTROY,
    });

    const ingest = new lambda.Function(this, 'Ingest', {
      runtime: lambda.Runtime.NODEJS_24_X,
      architecture: lambda.Architecture.ARM_64,  // Graviton
      memorySize: 512,                           // starting point, then Power Tuning decides
      timeout: Duration.seconds(10),
      handler: 'index.handler',
      code: lambda.Code.fromInline(`
        exports.handler = async (event) => {
          // Batch, don't trickle: one invocation handles many events.
          return { received: (event.Records || []).length };
        };`),
      logGroup,
      environment: { TABLE: events.tableName, BUCKET: archive.bucketName },
    });

    events.grantWriteData(ingest);
    archive.grantPut(ingest);

    // Printed by cdk deploy. Power Tuning needs it below.
    new CfnOutput(this, 'IngestArn', { value: ingest.functionArn });
  }
}

const app = new App();
// Pick the Region on residency and latency first. eu-north-1 is only an example.
new LeanIngestStack(app, 'LeanIngestStack', { env: { region: 'eu-north-1' } });

The synthesized template shows Architectures: [arm64], BillingMode: PAY_PER_REQUEST, TTL, RetentionInDays: 14 and both transitions.

So why 512 MB? It's a starting guess. Lambda gives CPU in proportion to memory, a full vCPU at 1,769 MB, so the setting with the fewest GB-seconds is sometimes larger than you'd expect. Deploy Power Tuning from its repo and let it decide.

Checked against the Lambda Power Tuning execution docs on Oct 29, 2026. Not run, so test it in a sandbox account first.

# INGEST_ARN comes from the stack output. POWER_TUNING_ARN is the tool's state machine.
cat > tuning-input.json <<EOF
{
  "lambdaARN": "${INGEST_ARN}",
  "powerValues": [128, 256, 512, 1024, 1769],
  "num": 50,
  "payload": { "Records": [] },
  "parallelInvocation": true,
  "strategy": "cost"
}
EOF
aws stepfunctions start-execution \
  --state-machine-arn "$POWER_TUNING_ARN" \
  --input file://tuning-input.json

Then take a monthly carbon baseline. The end date is exclusive, so this returns January to September 2026.

Checked against the AWS CLI reference for get-estimated-carbon-emissions on Oct 29, 2026. Not run, so test it in a sandbox account first.

aws sustainability get-estimated-carbon-emissions \
  --time-period Start=2026-01-01T00:00:00Z,End=2026-10-01T00:00:00Z \
  --group-by REGION SERVICE \
  --granularity MONTHLY \
  --emissions-types TOTAL_LBM_CARBON_EMISSIONS TOTAL_MBM_CARBON_EMISSIONS

Chart the location-based column, the one Cockcroft says to tune on, next to your proxy KPI.

Sustainability in the age of AI

AI as a workload

Patterson's and Wu's numbers agree: over a model's life, inference dominates, and inference is the part you pay for and control. The Green Software Foundation's SCI for AI, ratified in December 2025, formalizes this. A provider boundary covers design, training and deployment. A consumer boundary covers operation, with units like carbon per token or per agent workflow.

Better still, the Amazon Bedrock features that cut your token bill also cut compute per answer:

Model Distillation. AWS claims distilled models run up to 500% faster and cost 75% less, with under 2% accuracy loss for use cases like RAG.
Prompt caching. Up to 90% lower cost and 85% lower latency, per AWS, because the cached prefix isn't reprocessed.
Intelligent Prompt Routing. Picks between two models in a family per prompt, so easy requests can go to the smaller one.
Batch inference and the Flex tier. Batch costs 50% of on-demand for selected models. Flex is discounted for work that can wait.

Here's the caveat. AWS publishes prices and speedups for these, not watts. The energy saving is my inference, not published data.

flowchart LR
  F["Model Distillation<br/>prompt caching<br/>Intelligent Prompt Routing<br/>batch inference and Flex"] -->|"published by AWS"| C["Lower token bill<br/>and faster answers"]
  F -.->|"inferred, no watts published"| W["Less compute<br/>per answer"]

The cost saving is published by AWS, but the energy saving is an inference, not measured data.

Silicon. AWS claims Inf2 instances give up to 50% better performance per watt than comparable EC2 instances. Trainium3's figures use different baselines, so they don't compare. The December 2025 launch post claims "4x greater energy efficiency" for UltraServers against Trainium2 UltraServers and "40% better" for the chip against earlier generations. Amazon's 2025 sustainability report says over five times more output tokens per megawatt. They matter if you self-host on SageMaker or EKS. On Bedrock, you don't pick the chip.

The measurement gap. The Sustainability API groups by USAGE_ACCOUNT_ID, REGION and SERVICE, nothing finer. You'll see a monthly service share, not which model or agent produced it. So estimate per call from token counts and a public benchmark such as the AI Energy Score, label it an estimate, and track tokens per successful task as your proxy. Cross-Region inference also moves prompts onto other grids, which makes it a Compliance decision too.

The biggest lever comes first: whether to call a model at all. Luccioni's "orders of magnitude" is why this tree exists.

flowchart TB
  A["New feature idea"] --> B{"Is the output fixed<br/>for a given input?"}
  B -->|"yes"| R1["Rules, regex, SQL<br/>or a lookup table"]
  B -->|"no"| C{"Narrow task with<br/>labeled examples?"}
  C -->|"yes"| R2["Small task-specific model<br/>classifier or extractor"]
  C -->|"no"| D{"Asked and answered<br/>before?"}
  D -->|"often"| R3["Cache or retrieve<br/>the stored answer"]
  D -->|"rarely"| E{"Does a user wait<br/>for the result?"}
  E -->|"no"| R4["Smallest adequate LLM<br/>through batch or Flex"]
  E -->|"yes"| R5["Smallest adequate LLM<br/>with caching and routing"]

A general-purpose LLM is the last stop on every branch, after rules, small models and caches.

At Harbor Works, flagging a reading above a line's temperature limit is a rule. Summarizing last night's shift log goes through batch, because nobody waits on it at 2 a.m.

AI as an engineering aid

Coding assistants are good at the mechanical work: porting builds to arm64, adding lifecycle rules and log retention across a repo. They're bad at numbers. Ask one for a Region's carbon intensity and you'll get a confident figure with no source. So:

Accept no carbon or energy figure from an assistant without a primary link.
Judge each suggested optimization on the proxy KPI before and after, not on how good the explanation sounds.
Count the assistant itself. Agent loops are inference too, so cap their iterations in CI.

Who: owners and the shared responsibility split

Activity	Product owner	Architect	Developers	Platform / FinOps
Define unit of work and proxy KPI	A	R	C	C
Region and silicon choice	C	A	C	R
Right-sizing, arm64, retention	I	C	R/A	C
Monthly carbon reporting	I	C	I	R/A
"Should this use an LLM?"	A	R	C	I

R = responsible, A = accountable, C = consulted, I = informed.

Your share depends on the compute type:

EC2. AWS runs facilities, power and hardware. You own instance family, utilization, schedules and Region.
Fargate. AWS also packs hosts. You own task size, arm64 images and scaling to zero.
Lambda. AWS also packs environments. You own memory, architecture, invocations and logs.
Bedrock. AWS runs serving and the model provider owns training. You own whether to call, which model, tokens, caching and batch.

Monday checklist

[ ] Pick one unit of work and chart vCPU-minutes or GB-seconds per unit weekly.
[ ] Record a location-based carbon baseline by Region and service.
[ ] Switch your three busiest Lambda functions to arm64 and run Power Tuning on each.
[ ] Give the five largest log groups a retention period and the five largest buckets a lifecycle rule.
[ ] Add TTL to every DynamoDB table holding time-bound data.
[ ] Check whether residency lets your batch and eval jobs run in a lower-carbon Region.
[ ] Put every new AI feature through the decision tree, and log tokens per successful task.

Conclusion

I couldn't put these six in a room, but their published answers line up. Cockcroft gives the instruction: fewer computers, for less time. Hussain and the SCI make it measurable without offsets. Currie, Hsu and Bergman sort the work. Patterson and Luccioni show that for AI, inference and model choice are where your control sits. Birol warns that cheaper calls won't help if the calls keep multiplying.

So keep three habits: a proxy you move weekly, a carbon report you read monthly, and the reflex of asking whether a model call is needed. Ireland's 23% is what millions of small provisioning decisions add up to.

Make every watt answer a question someone actually asked.

Series wrap-up: nine requirements, one system

Part 1 started from Werner Vogels' re:Invent 2023 keynote and the first law of his Frugal Architect: make cost a non-functional requirement. Nine requirements later, none stands alone, and AI has pulled them tighter. One LLM feature now touches residency, prompt injection, screen readers on streamed text, time-to-first-token, throttling, token quotas, prompt versioning, the token bill and the energy behind it.

flowchart LR
  SEC["Security"] -->|"controls add latency"| PERF["Performance"]
  SEC -.->|"both non-negotiable"| COMP["Compliance"]
  COMP -->|"residency limits failover Regions"| AVAIL["Availability"]
  COMP -->|"residency limits clean Regions"| SUS["Sustainability"]
  A11Y["Accessibility"] -.->|"lean pages help low-end devices"| PERF
  PERF -->|"warm capacity sits idle"| SUS
  AVAIL -->|"redundancy multiplies resources"| COST["Cost"]
  SCAL["Scalability"] -->|"headroom costs money"| COST
  SCAL -.->|"autoscaling absorbs spikes"| AVAIL
  MAINT["Maintainability"] -.->|"simple systems are easier to tune"| PERF
  COST -.->|"mostly aligned"| SUS

Solid arrows are tensions you have to decide, and dotted arrows are alliances you can use.

Security, Compliance and Accessibility are the floor. The other six are trade-offs you make on purpose, write down and revisit as things change.

Now over to you. Which of the nine do you find hardest to get right, and why? Tell me in the comments.

Series: Non-functional Application Requirements

Non-functional Application Requirements: An Introduction
Non-functional Application Requirements: Security
Non-functional Application Requirements: Compliance
Accessibility
Performance
Availability
Scalability
Maintainability
Cost
Sustainability

Cover image: AWS re:Invent 2023 – Keynote with Dr. Werner Vogels

Sources

Non-functional Application Requirements: Cost

Oluwasegun Adedigba — Wed, 11 Nov 2026 15:03:02 +0000

In January 2024, Jack Ellis, founder of the analytics company Fathom, published a line-by-line account of $94,885 a year in savings on his team's AWS bill (Fathom, 2024). Fathom has no outside funding, and its ingest endpoint takes billions of requests a month on Lambda.

None of it came from a discount. It came from removing work: one Lambda invocation per pageview instead of two, a queue off the hot path, database traffic off a NAT gateway, logs nobody read.

Here's the line that makes it a cost story. Fathom raised its entry-level price by only $1, Ellis wrote, "because we knew we could get our costs down and not have to charge our customers lots more."

Cost decides what you can charge. That's why it belongs in your requirements.

TL;DR

Make cost a number: cost per unit of value (per request, tenant or successful task), not the monthly total.

Most waste is multiplication. Find the path where the money goes and read it line by line.

Run the FinOps loop, Inform → Optimize → Operate, and tag first. You can't optimize spend nobody owns.

Give every workload a budget and an anomaly monitor. It's about 90 lines of Terraform.

For AI the unit is the token, output costs about 5x input, and an agent without a session budget is an open tab.

What: cost, price and value

People use these three interchangeably. Don't. Cost is what you pay to deliver the service: the AWS bill, plus licenses, support and the people who run it. Price is what your customer pays. Value is what they get, and it caps the price. When cost per pageview drops, you can hold price and keep margin, which is exactly what Fathom did.

A bill that doubles while traffic triples is good news. The measurable form of this NFR is unit economics: cost divided by a unit the business cares about. The FinOps Framework puts it in the Quantify Business Value domain. AWS Well-Architected calls the same idea "measure overall efficiency."

Pick units you can test:

Cost per request, for APIs, and cost per tenant, for SaaS.
Cost per successful task, for AI features. Failed and retried calls count against it.
Tag coverage: the share of spend you can attribute to an owner. If much of the bill is untagged, every other number is a guess.

Write the target like a latency SLO: "ingest stays under $0.50 per million events at 2x today's traffic."

Why: waste is growing again

Flexera's 2026 State of the Cloud survey of more than 750 cloud decision-makers put estimated wasted cloud spend at 29%, the first increase in five years, with AI workloads named as a driver. 85% called managing cloud spend their top challenge (Flexera, March 2026).

AI moved fast. In the State of FinOps 2026 (1,192 respondents), 98% said they now manage AI spend, up from 31% in 2024 (FinOps Foundation, 2026).

Werner Vogels made it the first law of The Frugal Architect: "Make Cost a Non-functional Requirement." This series started from that keynote, so this part takes the law literally.

The case study: how Fathom cut $94,885 a year

Fathom's ingest path looked like many serverless systems that grew fast. A third-party CDN sent each pageview through AWS WAF and an Application Load Balancer to a Lambda function, which put a job on SQS. A second Lambda wrote it to Redis and SingleStore and logged to CloudWatch. It ran on Laravel Vapor, which deploys Laravel to Lambda.

The queue had a reason. Their database once ran in a single Availability Zone, so SQS buffered ingest from storage. That was an availability decision, and a sound one. Later the databases moved to multiple AZs. For a while, the queue stayed anyway.

flowchart LR
  subgraph B["Before: every pageview"]
    B1["CDN"] --> B2["WAF + ALB"] --> B3["Lambda 1"] --> B4["SQS"] --> B5["Lambda 2"] --> B6["Redis + SingleStore"]
    B5 --> B7["CloudWatch logs<br/>nobody read"]
    B6 -.->|"via NAT gateway"| B8["NAT data charges"]
  end
  subgraph A["After"]
    A1["CDN"] --> A2["WAF + ALB"] --> A3["One Lambda"] --> A6["Redis + SingleStore<br/>VPC peering, PrivateLink"]
    A3 -.->|"only if the database is down"| A4["SQS fallback"]
  end

Each pageview went from two Lambda invocations and a queue hop to one invocation, with SQS kept only as a fallback.

Ellis went after one endpoint, because "that's where all the money was being spent." What he found, in his numbers:

SQS, $23,989 a year, and Lambda, $20,862. With multi-AZ databases, the hot path didn't need the queue. One Lambda request per pageview halved invocations, and duration fell because the function now did a Redis hit and an insert, each typically 1 ms or less.
NAT gateway, $17,162. Database traffic was crossing the NAT gateway. It moved to VPC peering for Redis and AWS PrivateLink for SingleStore. "After seeing vast amounts of data being moved across it, I admitted I was wrong."
CloudWatch, $7,550. Lambda wrote logs nobody used, since errors went to Sentry. He removed the log permission and told readers to set retention, which log groups lack by default.
S3, $5,774. Versioning was on in one bucket, quietly keeping history.
The rest: WAF $12,201 (rate limiting moving to the CDN), CloudFront $4,800 (an internal sync moving about 4 TB a month), Route 53 $2,547 (longer DNS TTLs, fewer billed queries).

WAF and Route 53 were still in progress when he wrote it up, and CloudFront was a cost he said they "can easily remove", so about $19,500 of the total was still to come. Fathom doesn't publish its total bill, so there's no percentage, only dollars.

Let's put a number on the multiplier. At us-east-1 list prices (October 2026), Lambda requests cost $0.20 per million and SQS standard requests $0.40 per million (AWS Price List). A billion events a month, each making one extra Lambda call and one queue request, add about $600 in request fees alone, before duration, logs or data transfer. Per event it's nothing. Every event, every month, it's a line item.

Four lessons I'd put on any team's wall:

Find the path where the money goes. Fathom audited one endpoint, the one carrying the traffic.
Count work per business event. Two invocations and a queue hop per pageview is a design choice, and it multiplies.
Distrust defaults. No log retention, versioning left on, traffic through NAT. Nobody decided any of them.
Revisit a trade-off when its reason expires. The queue paid for availability until multi-AZ databases took over that job.

The FinOps loop that makes it repeatable

Fathom's review was a project. Keeping the savings takes a loop. The FinOps Framework calls its three phases Inform, Optimize and Operate.

flowchart LR
  I["Inform<br/>tags, Cost Explorer,<br/>CUR 2.0, unit cost"] --> O["Optimize<br/>right-size, delete,<br/>Savings Plans"]
  O --> P["Operate<br/>budgets, anomaly alerts,<br/>monthly review"]
  P -->|"new usage, new questions"| I
  P -.->|"cost in the design doc"| D["Next feature"]
  D -.-> I

The loop only turns if Operate feeds back into Inform, and new features enter with a cost estimate already attached.

Duolingo ran this loop in 2024 and reported 20% annualized savings within months, with weekly cost reports, cost estimates in tech specs and quarterly spend reviews (Duolingo, 2024). One find rhymes with Fathom's: a staging environment cost more than production because someone scaled it up for a test and forgot.

When: where cost enters the lifecycle

Design. Estimate unit cost and count calls per business event before you build.
Build. Make the cheap path the default: log retention, S3 lifecycle rules, table TTLs, tags in IaC. Infracost (Apache 2.0) adds a cost diff to Terraform pull requests.
Test. Read cost per request off your load test, a day later, once Cost Explorer catches up.
Operate. Budgets and anomaly alerts per workload, a monthly look at Cost Optimization Hub, one owner per tag.

Where: AWS services and the alternatives

Cost allocation tags come first, because every other tool groups by them. A tag counts only after you activate it, and a backfill reaches back 12 months.

flowchart LR
  T["Tag in IaC<br/>app, team, env"] --> R["Resources and<br/>Bedrock inference profiles"]
  R --> U["Usage records"]
  K["Activate tag<br/>in Billing console"] --> U
  U --> CE["Cost Explorer<br/>group by tag"]
  U --> CUR["Data Exports<br/>CUR 2.0"]
  U --> BU["AWS Budgets<br/>filter by tag"]
  U --> AD["Cost Anomaly Detection<br/>monitor by tag"]
  CUR --> UE["Unit cost<br/>per tenant or request"]

An untagged or unactivated resource never reaches the right-hand side, so it never gets an owner, a budget or an alert.

Cost Explorer is the free console for trends: 13 months back, 18 months of forecast, refreshed at least daily. Its API costs $0.01 per paginated request. For unit cost, export line items with Data Exports (CUR 2.0) and join them with request counts.

AWS Budgets alerts on actual or forecast spend and updates up to three times a day, so it's a smoke alarm, not a circuit breaker. Budget actions can apply an IAM policy or SCP, or stop EC2 and RDS instances.

Cost Anomaly Detection runs about three times a day, can take 24 hours to spot something, and needs 10 days of history for a new service. Each account gets one AWS-managed services monitor, and a management account up to 500 customer-managed ones.

Trusted Advisor, Compute Optimizer and Cost Optimization Hub find idle and oversized resources. The Hub prices them with your discounts. Trusted Advisor's full check set needs Business Support+ or higher.

Open source. OpenCost, a CNCF incubating project, allocates Kubernetes cost down to the container. LiteLLM (MIT) enforces per-key and per-team budgets in an LLM gateway.

Decide: commitments, and what cost work costs elsewhere

Option	What it covers	Up to	Term	Choose when
Compute Savings Plans	EC2 any family or Region, Fargate, Lambda	66% off	1 or 3 years	Your compute mix will change
EC2 Instance Savings Plans	One instance family in one Region	72% off	1 or 3 years	A stable fleet you won't re-platform
Database Savings Plans	Aurora, RDS, DynamoDB, ElastiCache, OpenSearch and more	35% off	1 year	Steady database spend across engines
Reserved Instances	Specific instance types, Standard or Convertible	72% off	1 or 3 years	Services or shapes Savings Plans don't fit
Spot	Spare EC2 capacity, interruptible	90% off	None	Stateless, retryable batch and CI work
Bedrock Reserved tier	Guaranteed tokens per minute for a model	Capacity, not a discount	1 or 3 months	Predictable, high-volume inference

AWS's maximum discounts, from the Savings Plans FAQ, Reserved Instances and Spot pages, October 2026. Commit to your floor, not your peak.

How cost pulls against the other NFRs:

NFR	Where cost work helps it	Where it hurts it
Security	Deleting unused resources shrinks attack surface	Cutting WAF or logs to save money
Compliance	Data minimization cuts storage and risk	Residency can rule out the cheapest Region or global inference
Accessibility	Lean pages suit cheap devices	Generated captions and alt text cost model calls
Performance	Fewer hops is faster too	Caches, provisioned concurrency and Priority tier bill when idle
Availability	Multi-AZ data can retire a buffer queue	Every redundant copy is paid for
Scalability	Scale-to-zero removes idle cost	Commitments can lock in capacity you outgrow
Maintainability	Fewer services to run	Clever cost hacks are hard to hand over
Sustainability	Less compute per request saves energy too	A cheap Region isn't always a clean one

Hands-on: a budget, an anomaly monitor and a tagged model

This stack gives one workload, the line copilot at Harbor Works (the fictional plant in Luso, this series' companion app), an owner tag, a tagged Bedrock inference profile, a monthly budget with forecast alerts and an anomaly monitor. Every alert goes to one SNS topic.

Checked against the Terraform AWS provider docs for each resource and the Cost Anomaly Detection SNS policy page on Oct 29, 2026. terraform validate passes on Terraform 1.17.1 with AWS provider 6.71.0. Not deployed, so test it in a sandbox account first.

terraform {
  required_providers {
    aws = { source = "hashicorp/aws", version = "~> 6.71" }
  }
}
provider "aws" {
  region = "us-east-1"
  default_tags { tags = { app = "line-copilot", env = "prod" } } # tag everything
}
data "aws_caller_identity" "me" {}

# Activation fails until the key has appeared in billing data (up to 24 hours).
resource "aws_ce_cost_allocation_tag" "app" {
  tag_key = "app"
  status  = "Active"
}

# Use this profile's ARN as modelId so Bedrock token spend carries the tags.
resource "aws_bedrock_inference_profile" "copilot" {
  name = "line-copilot-sonnet"
  model_source {
    copy_from = "arn:aws:bedrock:us-east-1:${data.aws_caller_identity.me.account_id}:inference-profile/us.anthropic.claude-sonnet-4-6"
  }
  tags = { feature = "shift-summary" }
}

resource "aws_sns_topic" "cost" { name = "line-copilot-cost-alerts" }
data "aws_iam_policy_document" "cost" {
  statement {
    actions   = ["SNS:Publish"]
    resources = [aws_sns_topic.cost.arn]
    principals {
      type        = "Service"
      identifiers = ["budgets.amazonaws.com", "costalerts.amazonaws.com"]
    }
    condition {
      test     = "StringEquals"
      variable = "aws:SourceAccount"
      values   = [data.aws_caller_identity.me.account_id]
    }
  }
}
resource "aws_sns_topic_policy" "cost" {
  arn    = aws_sns_topic.cost.arn
  policy = data.aws_iam_policy_document.cost.json
}

resource "aws_budgets_budget" "copilot" {
  name         = "line-copilot-monthly"
  budget_type  = "COST"
  limit_amount = "1500"
  limit_unit   = "USD"
  time_unit    = "MONTHLY"
  cost_filter {
    name   = "TagKeyValue"
    values = ["user:app$line-copilot"] # format is user:<key>$<value>
  }
  dynamic "notification" { # warn on the forecast, then at 80% actual
    for_each = { FORECASTED = 100, ACTUAL = 80 }
    content {
      notification_type         = notification.key
      threshold                 = notification.value
      threshold_type            = "PERCENTAGE"
      comparison_operator       = "GREATER_THAN"
      subscriber_sns_topic_arns = [aws_sns_topic.cost.arn]
    }
  }
  depends_on = [aws_sns_topic_policy.cost]
}

resource "aws_ce_anomaly_monitor" "copilot" {
  name         = "line-copilot"
  monitor_type = "CUSTOM" # customer managed: up to 10 tag values, monitored together
  monitor_specification = jsonencode({
    And  = null, CostCategories = null, Dimensions = null, Not = null, Or = null
    Tags = { Key = "app", MatchOptions = null, Values = ["line-copilot"] }
  })
}
resource "aws_ce_anomaly_subscription" "copilot" {
  name             = "line-copilot-immediate"
  frequency        = "IMMEDIATE" # IMMEDIATE requires an SNS subscriber
  monitor_arn_list = [aws_ce_anomaly_monitor.copilot.arn]
  subscriber {
    type    = "SNS"
    address = aws_sns_topic.cost.arn
  }
  threshold_expression {
    dimension {
      key           = "ANOMALY_TOTAL_IMPACT_ABSOLUTE"
      match_options = ["GREATER_THAN_OR_EQUAL"]
      values        = ["50"]
    }
  }
  depends_on = [aws_sns_topic_policy.cost]
}

Three things will catch you. Cost allocation tags are managed from the management account, so run the tag resource there. Tags aren't retroactive without a backfill. And a KMS-encrypted topic needs key policy grants for both services.

Cost in the age of AI

AI as a workload: the token is the unit

You pay for input and output tokens at different prices. In the us-east-1 price list dated October 12, 2026, Claude Sonnet 4.6 on the global cross-Region profile costs $3.00 per million input tokens and $15.00 per million output tokens (AWS Price List). The US geographic profile costs 10% more.

Take a Harbor Works shift summary: 6,000 tokens in, 500 out, $0.0255 per call. Output is 8% of the tokens and 29% of the cost. A million summaries a month is $25,500, so cap maxTokens and ask for short answers before anything else.

Attribute it before you optimize it. Application inference profiles, IAM principal tags (since April 2026), and projects and workspaces on the bedrock-mantle endpoint all deliver billed dollars to Cost Explorer and CUR 2.0, per usage type per day (Bedrock cost management). Profiles cover InvokeModel and Converse on bedrock-runtime, one per model and tag set. Request metadata adds per-call detail to invocation logs.

Alert where the bill lands. In May 2026, The Register reported a reader whose Claude Opus usage on Bedrock reached $30,141.33 in April with no anomaly alert (The Register, 2026). Claude bills through AWS Marketplace, which anomaly detection didn't monitor then, and $8,026.54 of credits hid the spend for weeks. AWS added third-party Bedrock models on August 19, 2026. Keep a budget alarm too. A second alarm costs almost nothing.

Pull the levers in order. Each is cheaper to try than the next.

flowchart TB
  L1["1. Right-size the model<br/>Haiku 4.5 is 3x cheaper than Sonnet 4.6 here,<br/>Nova 2 Lite about 8x"] --> L2["2. Intelligent Prompt Routing<br/>easy prompts go to the smaller model"]
  L2 --> L3["3. Prompt caching<br/>reuse the stable prefix"]
  L3 --> L4["4. Batch or Flex tier<br/>half price for work nobody waits on"]
  L4 --> L5["5. Distillation<br/>train a small model on the big one"]
  L5 --> L6["6. Reserved tier<br/>buy capacity once volume is steady"]
  E["Eval suite"] -.->|"gates every step"| L1
  E -.-> L2
  E -.-> L5

Start at the top and only move down when the step above is done and your evals still pass.

Right-size. The same call costs $0.0085 on Claude Haiku 4.5 and $0.00305 on Amazon Nova 2 Lite, at global list prices. Run your evals on the smaller model first.
Intelligent Prompt Routing sends each prompt to the cheaper model in a family unless the predicted quality gain clears your threshold. AWS claimed up to 30% savings at preview in 2024. It's optimized for English.
Prompt caching. Care Access caches each medical record as the prompt prefix and changes only the question. It reported 86% lower Bedrock costs and 66% faster processing (AWS Machine Learning Blog, November 2025). On Sonnet 4.6 a cache read costs a tenth of fresh input, so a 4,000-token cached prefix cuts our call to $0.0147, once the first call has written it at a 25% premium. The prefix needs at least 1,024 tokens, and batch can't use the cache.
Batch and Flex. Batch lists at half the on-demand rate for these models. The Flex tier, set with serviceTier, halves Nova 2 Lite's price but queues behind Standard under load. The 2 a.m. shift summary belongs here.
Distillation. AWS claims distilled models run up to 500% faster and 75% cheaper with under 2% accuracy loss for RAG (AWS, May 2025). It's a vendor figure, so measure it on your own data.
Reserved tier. You buy tokens per minute for 1 or 3 months, with a 100,000 input TPM minimum. In AWS's October 2026 price list, a fully used 1-month Sonnet 4.6 reservation costs the same per token as on-demand Standard, and the 3-month term is 10% less. The pricing page doesn't show Reserved rates and access goes through your account team, so confirm your quote. You're buying guaranteed capacity, not a discount.

Self-hosting is a seventh lever that pays only at volume. Part 7, Scalability, works through that break-even.

Runaway agents are a new failure mode. One request can become dozens of model calls, and a loop that never reaches end_turn burns tokens until something stops it. Billing alarms see it hours later. The stop belongs in the request path.

sequenceDiagram
  participant U as User
  participant A as Agent loop
  participant G as Budget guard
  participant K as Kill switch
  participant B as Bedrock
  U->>A: one request
  loop each step
    A->>G: next call
    G->>K: AI enabled? cached 30s
    G->>G: steps and dollars under cap?
    G->>B: Converse with maxTokens sized to what is left
    B-->>G: usage in tokens
    G->>G: add cost to session total
  end
  G-->>A: BudgetExceeded
  A-->>U: partial answer and a clear message

The guard checks the switch and the session's spend before every call, so a runaway loop stops within one step, not one billing cycle.

Checked against the Converse API reference and the us-east-1 Bedrock price list on Oct 29, 2026. Unit-tested with botocore's Stubber on boto3 1.43.121 for the budget-exceeded and kill-switch paths. Not deployed, so test it in a sandbox account first.

# budget_guard.py: per-session token budget and a kill switch around Converse
import time
import boto3

bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")
ssm = boto3.client("ssm", region_name="us-east-1")

# USD per 1M tokens: Claude Sonnet 4.6, global profile, us-east-1 price list, Oct 2026
PRICE = {"in": 3.00, "out": 15.00, "cache_read": 0.30, "cache_write": 3.75}
_switch = {"value": "on", "checked": 0.0}


class BudgetExceeded(Exception):
    pass


def ai_enabled(ttl=30):
    """One SSM parameter flips every agent off. Cached so each step isn't an SSM call."""
    if time.time() - _switch["checked"] > ttl:
        param = ssm.get_parameter(Name="/line-copilot/ai-enabled")["Parameter"]
        _switch.update(value=param["Value"], checked=time.time())
    return _switch["value"] == "on"


def call_cost(u):
    return (u.get("inputTokens", 0) * PRICE["in"]
            + u.get("outputTokens", 0) * PRICE["out"]
            + u.get("cacheReadInputTokens", 0) * PRICE["cache_read"]
            + u.get("cacheWriteInputTokens", 0) * PRICE["cache_write"]) / 1_000_000


class Session:
    def __init__(self, session_id, max_usd=0.50, max_steps=12):
        self.id, self.max_usd, self.max_steps = session_id, max_usd, max_steps
        self.spent, self.steps = 0.0, 0

    def converse(self, model_id, messages, **kwargs):
        if not ai_enabled():
            raise BudgetExceeded("kill switch is off")
        if self.steps >= self.max_steps or self.spent >= self.max_usd:
            raise BudgetExceeded(f"{self.id}: {self.steps} steps, ${self.spent:.4f}")
        # Cap output to what the remaining budget can pay for (floor of 64 tokens).
        left = self.max_usd - self.spent
        max_out = max(64, min(2048, int(left * 1_000_000 / PRICE["out"])))
        resp = bedrock.converse(
            modelId=model_id,
            messages=messages,
            inferenceConfig={"maxTokens": max_out},
            requestMetadata={"session": self.id},  # per-call detail in invocation logs
            **kwargs,
        )
        self.steps += 1
        self.spent += call_cost(resp["usage"])
        return resp

Pass the profile ARN from the Terraform stack as model_id and the spend lands under the app tag too. The guard can overshoot by one call, so the output cap shrinks as the budget runs down.

Vector storage can quietly dominate RAG cost. S3 Vectors charges $0.06 per GB-month, so a million 1,024-dimension vectors, about 4 GB, cost roughly $0.25 a month to store, plus $2.50 per million queries (S3 pricing). A classic OpenSearch Serverless collection bills at least 2 OCUs, about $350 a month at $0.24 per OCU-hour. The catch is S3 Vectors' latency, around 100 ms for frequent queries, so put it behind a smaller, faster hot tier.

AI as an engineering aid

AI assistants are good at the reading. The AWS Billing and Cost Management MCP server lets an assistant query cost, budget, anomaly and Savings Plans data, with a SQL engine for the arithmetic (AWS, August 2025). The AWS FinOps Agent, in free preview in us-east-1 since June 9, 2026, investigates anomalies and can open Jira tickets and post to Slack (AWS, June 2026).

They're bad at judgment. An assistant will happily recommend a three-year commitment for a workload you're about to re-platform. The guardrails I'd set:

Read-only billing roles. No purchase or budget-action permissions.
Every number cites a query you can rerun. Spot-check it against CUR.
Count the assistant's own cost: Cost Explorer API pages are $0.01 each, and agent loops are inference too.
A human approves every commitment, deletion and budget action.

Who: owners and the shared responsibility split

Activity	Product owner	Architect	Developers	Platform / FinOps
Unit cost target and price decisions	A	C	I	R
Calls per event, model choice, cost in design docs	C	R/A	R	C
Tags, retention, caps, budget guards	I	C	R/A	C
Budgets, anomaly alerts, commitments, monthly review	I	C	C	R/A

R = responsible, A = accountable, C = consulted, I = informed.

AWS sets prices, meters usage and supplies recommendations. What you control depends on compute type:

EC2: instance size, purchase option, and shutting down what's idle.
Containers (ECS on Fargate, EKS): task and pod sizing, bin packing, the Spot mix.
Lambda: memory, duration, invocations per event, log volume. INIT time has been billed since August 1, 2025.
Managed AI (Bedrock): the provider sets token prices and caching rules. You own tokens per task, model choice, output caps and session budgets.

Monday checklist

[ ] Pick one unit cost metric for your busiest workload and write a target for it.
[ ] Find the endpoint with the most spend and count service calls per request on it.
[ ] Set retention on every CloudWatch log group that has none.
[ ] Activate app and team as cost allocation tags and check tag coverage.
[ ] Deploy the budget and anomaly monitor above for one workload.
[ ] Route Bedrock calls through a tagged application inference profile.
[ ] Cap maxTokens on every model call, and put a per-session budget on every agent.
[ ] Run your eval set on the next-smaller model.
[ ] Add a prompt cache checkpoint after your stable prefix and watch cacheReadInputTokens.

Conclusion

Fathom didn't negotiate a better rate. It read its ingest path one service at a time and removed work each request didn't need: a second invocation, a queue that had outlived its reason, traffic through NAT, logs nobody opened. That bought the room to raise prices by a dollar instead of more.

The method holds for AI. Measure the unit, tokens per successful task. Attribute it with tags, profiles and principals. Pull the levers in order and gate each with evals. Then put a budget in the request path, because the billing alarm arrives a day late.

Your bill is your architecture, itemized.

Series: Non-functional Application Requirements

Non-functional Application Requirements: An Introduction
Non-functional Application Requirements: Security
Non-functional Application Requirements: Compliance
Accessibility
Performance
Availability
Scalability
Maintainability
Cost
Sustainability (coming soon)

Cover image: AWS re:Invent 2023 – Keynote with Dr. Werner Vogels

Next up, Sustainability: the series closes with published expert voices on energy and carbon per unit of work, and why most cost levers are carbon levers too.

Sources

Non-functional Application Requirements: Maintainability

Oluwasegun Adedigba — Wed, 11 Nov 2026 15:02:47 +0000

On August 1, 2012, Knight Capital rolled out new code to its order router by hand and missed one of the eight servers. The new code also reused a flag that once switched on "Power Peg", a feature nobody had used since 2003. On the missed server, that flag woke the dead code up. In 45 minutes the router sent millions of orders and got more than 4 million executions while trying to fill 212 customer orders, and Knight lost more than $460 million. The SEC found that no second technician reviewed the deploy, and no written procedure required one (SEC order, 2013).

Nothing in that story is exotic. It took dead code, a flag with two meanings and a deploy that depended on nobody missing a step.

That's what maintainability failures usually look like. The system runs fine until somebody has to change it.

TL;DR

Maintainability is how cheaply and safely you can change a running system. Measure it with DORA's five delivery metrics.

Prompts and model IDs are code, model end-of-life is a scheduled outage, and MCP servers are dependencies. Version, test and inventory all three.

Gate every prompt or model change on an eval suite: a golden set, cheap checks, then a calibrated judge.

AI raises throughput but, per DORA 2025, still hurts stability. Small batches and fast rollback turn speed into delivery.

What: maintainability, and how to measure it

ISO/IEC 25010:2023 defines maintainability as the "capability of a product to be modified by the intended maintainers with effectiveness and efficiency." For a cloud system, I break that into five parts you can check:

Modifiability. You change one thing without touching five others.
Testability. A machine tells you in minutes whether the change broke anything.
Observability. You can follow one request end to end.
Deployability. You can ship and undo a change without a hero.
Operability. On-call can run it from a runbook, upgrades included.

Clean code helps, but a tidy codebase with a manual deploy and no rollback is still hard to maintain.

The measurable form is DORA's delivery metrics, five since 2024. Throughput is change lead time, deployment frequency and failed deployment recovery time. Instability is change fail rate and deployment rework rate, the share of deploys forced by a production incident. DORA says to measure one service at a time and never turn the numbers into targets. For AI features, add the eval pass rate per prompt version and the days until your nearest model end-of-life.

Why: neglect compounds

Unmaintainable code costs time. Adam Tornhill and Markus Borg found that low-quality code had 15 times more defects than high-quality code, and that fixing issues in it took 124% more development time (Code Red, 2022).

AI made the problem faster, not smaller. DORA's 2024 report estimated that a 25% rise in AI adoption went with a 7.2% drop in delivery stability. The 2025 edition, the latest at the time of writing, surveyed nearly 5,000 professionals. 90% use AI at work, and AI adoption now goes with higher throughput but still with lower stability (DORA, 2025).

Don't trust how fast it feels, either. In METR's 2025 trial, experienced open-source developers took 19% longer with AI tools while believing they'd been 20% faster (METR, 2025). METR called the data from its late-2025 follow-up, published in February 2026, "an unreliable signal", so treat the 19% as a warning about perception.

When: where maintainability enters the lifecycle

Design. Give every prompt, model ID and flag one owner.
Build. Everything deployable lives in Git, prompts and IaC included.
Test. Evals on every prompt or model change, a canary on every deploy.
Operate. Drift checks, a lifecycle inventory, traces you can follow, and a monthly look at the DORA metrics.

Where: two pitfalls that predate AI

Each pitfall follows the same arc: what you'll notice, why it happens, the fix, and the tool I'd reach for.

Pitfall 1: Deploys that live in someone's head

You'll notice it as a deploy that only works when one person runs it, and a rollback nobody wants to try.

It happens because the deploy started as a script for one server and grew. Nobody wrote every step down, and a missed step fails silently, as it did at Knight.

flowchart LR
  A["Manual deploy<br/>to 8 servers"] --> B["1 server missed"]
  C["New code reuses<br/>an old flag"] --> D["Flag switched on<br/>at market open"]
  B --> D
  D -->|"on the missed server"| E["Power Peg code<br/>unused since 2003 runs"]
  E --> F["4 million executions in 45 min<br/>loss over 460M USD"]

It took three ordinary gaps lining up: a manual deploy, dead code left in place, and a flag that meant two things.

The fix is a pipeline that is the only road to production, with rollback that needs no human. CodePipeline orchestrates, CodeBuild tests, and CodeDeploy shifts traffic. For Lambda, CodeDeployDefault.LambdaCanary10Percent5Minutes sends 10% of traffic to the new version, waits five minutes, then shifts the rest. Add a CloudWatch alarm to the deployment group with auto-rollback on DEPLOYMENT_STOP_ON_ALARM, and a bad canary redeploys the last good revision while you sleep. CDK Pipelines updates its own stages from the repo once deployed. GitHub Actions works too, and CodeCommit has been generally available again since November 2025.

Content needs staged rollouts too. CrowdStrike's July 2024 outage came from a content update, and its root-cause fixes include staged rings with bake time. Prompts and feature flags are content. Keep flags in AWS AppConfig, and never give an old flag a new meaning.

Pitfall 2: The console fix that never made it back

You'll notice it when the next deploy undoes last month's console hotfix, or fails on a resource that "shouldn't" look like that.

It happens because incidents reward speed, and nobody owns the follow-up.

The fix is to check for drift on a schedule and before every deploy, then fix it in code:

cdk drift MyStack --fail exits with code 1 on drift, so it works as a pipeline step.
The AWS Config rule cloudformation-stack-drift-detection-check runs periodically, within a 15-minute limit, so scope it with tags.
Drift-aware change sets (November 2025) with --deployment-mode REVERT_DRIFT put drifted resources back to the template.
In Terraform, terraform plan -refresh-only -detailed-exitcode returns 2 when something changed outside Terraform.

CloudFormation only checks properties you set explicitly, so write the ones that matter into the template.

Maintainability in the age of AI

AI changes maintainability from two sides. As a workload, it adds things that change without a code diff. As an engineering aid, it writes more code than your review process was built for.

AI as a workload

Pitfall 3: Prompts and model IDs outside version control

You'll notice it when answers got worse on Tuesday and the diff is empty. The prompt lives in a console text box or an environment variable, and the model ID is a string someone edited.

It happens because prompts feel like copy, so they get edited like copy. But a prompt plus a model ID is your feature's behavior, so changing either is a release.

The fix is to keep both in Git and ship them through the pipeline. Amazon Bedrock Prompt Management stores prompts as numbered snapshot versions, so the app calls version 7 while you edit the draft. When you pass a prompt version ARN to Converse as the modelId, the request can't include system, inferenceConfig, toolConfig or additionalModelRequestFields (Converse API reference). That's the point. The version owns the model and its settings, not the caller.

Here's the shift-assistant prompt for Harbor Works, the fictional plant in my companion app Luso, as CDK. The template text sits in a file beside the code.

Checked against the CDK CfnPrompt reference and AWS::Bedrock::PromptVersion on Oct 29, 2026. Type-checked and synthesized with aws-cdk-lib 2.275.0 and TypeScript 5.9.3. Not deployed — test in a sandbox account first.

import { createHash } from 'node:crypto';
import { readFileSync } from 'node:fs';
import { App, RemovalPolicy, Stack, StackProps } from 'aws-cdk-lib';
import * as bedrock from 'aws-cdk-lib/aws-bedrock';
import * as ssm from 'aws-cdk-lib/aws-ssm';
import { Construct } from 'constructs';

// Prompt text, model ID and settings live in Git, next to the code that calls them.
const variant: bedrock.CfnPrompt.PromptVariantProperty = {
  name: 'main',
  templateType: 'TEXT',
  modelId: 'eu.anthropic.claude-sonnet-4-6', // pinned EU geo inference profile
  inferenceConfiguration: { text: { temperature: 0.2, maxTokens: 512 } },
  templateConfiguration: {
    text: {
      text: readFileSync('prompts/shift-assistant.txt', 'utf8'),
      inputVariables: [{ name: 'line' }, { name: 'question' }],
    },
  },
};

export class PromptStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    const prompt = new bedrock.CfnPrompt(this, 'ShiftAssistant', {
      name: 'shift-assistant',
      defaultVariant: 'main',
      variants: [variant],
    });

    // Every PromptVersion property forces replacement, so a hash of the variant in
    // the description cuts a new numbered version whenever anything in it changes.
    const hash = createHash('sha256').update(JSON.stringify(variant)).digest('hex');
    const version = new bedrock.CfnPromptVersion(this, 'ShiftAssistantVersion', {
      promptArn: prompt.attrArn,
      description: `release ${hash.slice(0, 12)}`,
    });
    version.applyRemovalPolicy(RemovalPolicy.RETAIN); // old versions stay callable

    // The eval stage and the app read the ARN from here: one place to promote or roll back.
    new ssm.StringParameter(this, 'CandidateArn', {
      parameterName: '/harbor-works/shift-assistant/candidate-prompt-arn',
      stringValue: version.attrArn,
    });
  }
}

new PromptStack(new App(), 'HarborWorksPrompts', { env: { region: 'eu-west-1' } });

Rolling back is a git revert, which cuts a new version with the old content. Not on Bedrock? A prompts/ folder plus a versioned model ID in config does the same job.

Pitfall 4: Shipping LLM changes with no eval suite

You'll notice it when prompt changes are "tested" in a playground with three questions, and users find the four answers that broke.

It happens because LLM output isn't deterministic, so teams decide it can't be tested. It can. You test properties and judgments instead of exact strings, in three layers:

A golden set of real questions with reference answers, owned by someone who knows the domain. I'd start with 50 from production logs and add every reported bug.
Cheap checks first. Must include, must not include, valid JSON. They cost nothing and catch a lot.
A calibrated judge for the rest. In Zheng et al.'s 2023 study, strong LLM judges agreed with humans over 80% of the time, as often as humans agree with each other, but showed position, verbosity and self-enhancement bias. Use a judge from a different model family at temperature 0, and check it against human labels first.

On AWS, Bedrock evaluations run judge jobs with built-in metrics such as Builtin.Correctness, even on responses from other providers. AgentCore Evaluations (GA March 2026) adds on-demand agent regression tests for CI/CD. The open-source choice is promptfoo, still MIT-licensed after joining OpenAI in 2026.

Pitfall 5: Treating model end-of-life as a surprise

You'll notice it when a model ID that worked for a year starts returning ResourceNotFoundException: This model version has reached the end of its life.

It happens because nobody put model lifecycles on a calendar. Bedrock models go from Active to Legacy to EOL. AWS publishes the EOL date when a model enters Legacy, usually six months ahead and sometimes only 45 days. In Legacy, existing customers "may lose access after 15 days of inactivity". After EOL, requests fail and "migration will not happen automatically" (Bedrock model lifecycle). Claude Sonnet 4.5 is already Legacy, with EOL on April 8, 2027.

flowchart LR
  A["Active<br/>earliest EOL on the model card"] -->|"AWS publishes the EOL date"| L["Legacy<br/>6 months or 45 days of notice"]
  L --> E["End of life<br/>requests fail"]
  A -.-> Y1["You: pin the ID<br/>and record the dates"]
  L -.-> Y2["You: eval the successor<br/>and migrate"]
  E -.-> Y3["You: already moved"]

The EOL date only appears once a model enters Legacy, so that's when your migration clock starts.

The fix is an inventory with an alarm. This check fails a daily build 90 days before trouble.

Checked against the GetFoundationModel API and the model lifecycle guide on Oct 29, 2026. Unit-tested with botocore Stubber on boto3 1.43.121. Not deployed — test in a sandbox account first.

"""Fail (exit 1) if a model we depend on is Legacy or within 90 days of EOL."""

import datetime as dt
import sys

import boto3

# Base model IDs: strip the eu./us./global. inference-profile prefix.
IN_USE = ["anthropic.claude-sonnet-4-6", "amazon.nova-2-lite-v1:0"]
WARN = dt.timedelta(days=90)

bedrock = boto3.client("bedrock", region_name="eu-west-1")
now = dt.datetime.now(dt.timezone.utc)
at_risk = []
for model_id in IN_USE:
    life = bedrock.get_foundation_model(modelIdentifier=model_id)["modelDetails"].get(
        "modelLifecycle", {}
    )
    eol = life.get("endOfLifeTime")  # set once the model enters Legacy
    print(f"{model_id}: {life.get('status')} eol={eol}")
    if life.get("status") == "LEGACY" or (eol and eol - now < WARN):
        at_risk.append(model_id)

sys.exit(1 if at_risk else 0)

Run the check from a Region that lists every model you use. In Europe, Sonnet 4.6 runs In-Region only in London (eu-west-2) and Ireland reaches it through the EU profile, so if the call fails in eu-west-1, point the client at eu-west-2.

Your upgrade runbook is then the eval suite: point a branch at the new model, run the golden set, compare. Runtimes need the same calendar. Lambda deprecates Python 3.10 on October 31, 2026, and Amazon Inspector exports SBOMs in CycloneDX 1.4 or SPDX 2.3 for EC2, ECR and Lambda, so the inventory isn't a spreadsheet.

Pitfall 6: Tracing a non-deterministic request with a retiring SDK

You'll notice it when a user says the assistant "went in circles". Your logs show a 200 after 14 seconds, but not which prompt version ran or which tools it called.

It happens for two reasons. One question becomes a chain of model and tool calls that request logs can't show. And many teams use the X-Ray SDKs, which entered maintenance mode on February 25, 2026 and reach end of support on February 25, 2027 (AWS, 2025). The X-Ray service itself stays.

The fix is OpenTelemetry, through AWS Distro for OpenTelemetry (ADOT) or the upstream SDKs, exporting to CloudWatch's OTLP endpoint at https://xray.<region>.amazonaws.com/v1/traces with SigV4. Follow the GenAI semantic conventions: gen_ai.operation.name and gen_ai.provider.name on every inference span, plus gen_ai.request.model and the gen_ai.usage.* token counts. Then add one attribute of your own, the prompt version ARN. It turns "answers got worse" into a query.

sequenceDiagram
  participant U as Operator
  participant API as Assistant API
  participant BR as Bedrock
  participant MCP as Fault history MCP server
  participant CW as CloudWatch OTLP
  U->>API: Why did line 2 stop
  Note over API: invoke_agent span with the prompt version ARN
  API->>BR: chat span, gen_ai.request.model
  BR-->>API: tool call requested, token counts
  API->>MCP: execute_tool span, get_fault_history
  MCP-->>API: last 10 faults
  API->>BR: second chat span with the tool result
  BR-->>API: answer and finish reason
  API-->>U: answer
  API->>CW: export all spans over OTLP

One question becomes four spans, and the prompt version attribute links a bad answer to the change that caused it.

The conventions are young. Every gen_ai.* attribute is at Development stability, so pin your instrumentation library. Prompt and completion text sits in an opt-in event because it often holds personal data, so leave it off in production by default.

CloudWatch generative AI observability (GA October 2025) gives you dashboards for model invocations and AgentCore agents, and CloudWatch Omni (GA September 2026) adds traces and evaluators in the IDE. Prefer to own the backend? Jaeger takes the same OTLP spans.

Pitfall 7: MCP servers nobody reviewed

You'll notice it when an agent behaves differently and no code changed. Or worse, it behaves the same and quietly does something extra.

It happens because MCP servers skip the controls you put on libraries. They get installed with an unpinned npx command, they hold credentials, and their tool descriptions go straight into the model's context. The MCP specification itself says tool annotations "should be considered untrusted, unless obtained from a trusted server."

That's already been exploited. In September 2025, the unofficial postmark-mcp npm package shipped 15 clean versions, then version 1.0.16 quietly BCC'd every email it sent to an attacker (The Register, 2025).

flowchart LR
  APP["Shift assistant"] --> PV["Prompt version<br/>Git and Bedrock"]
  PV --> M["Model ID<br/>pinned, has an EOL date"]
  APP --> FW["Agent framework<br/>pinned library"]
  APP --> MCP1["Fault history MCP server<br/>internal, reviewed"]
  APP --> MCP2["Email MCP server<br/>third-party npm package"]
  MCP2 --> CRED["Mail API credentials"]
  MCP2 -.->|"tool descriptions enter the prompt"| M
  RT["Lambda runtime<br/>deprecation date"] --> APP

Each box changes on its own schedule, and the third-party server is the one holding credentials, so each needs a pinned version and an owner.

The fix is to treat each MCP server as a third-party package with network access. Pin exact versions, never @latest. Put servers in your SBOM, and give each one least-privilege credentials of its own. Commit a snapshot of the tool schemas with a test that fails when the live tools differ, so a changed description becomes a red build.

AI as an engineering aid

Pitfall 8: AI-written code outpacing review

You'll notice it when pull requests get bigger, review turns into skimming, and change fail rate creeps up while everyone feels faster.

It happens because generating code got cheap and reviewing it didn't. DORA's 2025 report calls AI an amplifier, and its AI Capabilities Model singles out two habits: working in small batches, and strong version control, where frequent commits and frequent rollbacks amplified the gains.

The fix is to let machines carry the checks so people review intent. Cap pull request size, require tests with generated changes, and enforce lint, types and coverage in CodeBuild. Keep code owners, so generated code still has a human one. If change fail rate rises while lead time falls, you're borrowing against the future.

The same rules make coding agents useful for upgrades. Kiro (GA November 2025) does spec-driven development with property-based tests. AWS Transform custom (GA December 2025) runs Java 8 to 17, Python and Node.js upgrades from one CLI command. Still on Amazon Q Developer in the IDE? Its plugins reach end of support on April 30, 2027, with Kiro as the named successor.

Whatever the agent, the gate is the same: it opens a pull request, the full test and eval suite runs, and a person approves. An agent migration without tests is just a very large unreviewed diff.

Still immature: judges need recalibrating, the GenAI conventions are unstable, and MCP has no common standard for signing servers.

Hands-on: an eval gate for prompt and model changes

At Harbor Works, the shift assistant answers operators' questions about line faults. A wrong answer could mean someone clears a jam without locking out the conveyor, so prompt changes ship like code.

flowchart LR
  PR["Pull request<br/>prompt, model ID or code"] --> SRC["CodePipeline<br/>source and trigger"]
  SRC --> STG["Deploy to staging<br/>new prompt version"]
  STG --> CB["CodeBuild<br/>lint, unit tests, evals"]
  CB -->|"pass rate at or above the gate"| PROD["Deploy to production<br/>CodeDeploy canary"]
  CB -->|"below the gate or any error"| STOP["Pipeline stops<br/>report lists failing cases"]
  PROD -->|"canary alarm fires"| RB["Automatic rollback<br/>last good revision"]

A prompt or model change can't reach production without passing the same eval gate and canary as code.

The staging deploy runs the pitfall 3 stack, which writes the new prompt version ARN to Parameter Store. CodeBuild reads it and runs lint, tests and evals.

Checked against the CodeBuild buildspec reference and available runtimes on Oct 29, 2026. YAML-parsed, not run in CodeBuild — test in a sandbox account first.

version: 0.2

env:
  variables:
    JUDGE_MODEL_ID: "eu.amazon.nova-2-lite-v1:0" # a different family from the generator
    EVAL_THRESHOLD: "0.9"
  parameter-store:
    # Written by the staging deploy: ARN of the prompt version under test
    PROMPT_VERSION_ARN: "/harbor-works/shift-assistant/candidate-prompt-arn"

phases:
  install:
    runtime-versions:
      python: 3.12
    commands:
      - pip install -r requirements-dev.txt # pinned boto3, pytest, ruff
  build:
    commands:
      # One phase on purpose: a failing command skips the rest of the phase,
      # so the paid eval step only runs after lint and unit tests pass.
      - ruff check .
      - ruff format --check .
      - pytest -q --junitxml=reports/unit.xml
      - python evals/run_evals.py evals/golden.jsonl reports/evals.xml

reports:
  unit-tests:
    files: ["unit.xml"]
    base-directory: reports
    file-format: JUNITXML
  llm-evals:
    files: ["evals.xml"]
    base-directory: reports
    file-format: JUNITXML

Each line of the golden set is one case:

{"id": "e17-conveyor-jam", "vars": {"line": "2", "question": "Conveyor stopped with fault E-17. What do I do first?"}, "reference": "E-17 is the jam sensor. Lock out the conveyor before clearing the jam, then reset from the HMI.", "must_include": ["lock"], "must_not_include": ["bypass the sensor"]}

The runner applies the string checks, asks the judge only when they pass, and writes JUnit XML for CodeBuild's test reports.

Checked against the Converse API reference on Oct 29, 2026. Unit-tested with botocore Stubber on boto3 1.43.121 and Python 3.13, ruff clean. Not run against Bedrock — test in a sandbox account first.

"""Eval gate: run the golden set against a prompt version, write JUnit XML."""

import json
import os
import re
import sys
import xml.etree.ElementTree as ET

import boto3
from botocore.config import Config

brt = boto3.client("bedrock-runtime", config=Config(retries={"mode": "standard"}))
PROMPT_ARN = os.environ.get("PROMPT_VERSION_ARN", "")  # ...:prompt/ID:VERSION
JUDGE = os.environ.get("JUDGE_MODEL_ID", "")
THRESHOLD = float(os.environ.get("EVAL_THRESHOLD", "0.9"))
RUBRIC = (
    "Grade a plant-floor assistant.\nQuestion: {q}\nReference: {ref}\n"
    "Candidate: {ans}\nScore 1 only if the candidate agrees with the reference "
    'and gives a safe next step. Reply with JSON only: {{"score": 0 or 1, "reason": "..."}}'
)


def text_of(r: dict) -> str:
    return next(b["text"] for b in r["output"]["message"]["content"] if "text" in b)


def generate(variables: dict) -> str:
    # The prompt version pins model, system prompt and inference config, so
    # Converse rejects system/inferenceConfig/toolConfig here. Variables only.
    r = brt.converse(
        modelId=PROMPT_ARN,
        promptVariables={k: {"text": v} for k, v in variables.items()},
    )
    return text_of(r)


def judge(case: dict, answer: str) -> tuple[int, str]:
    prompt = RUBRIC.format(
        q=case["vars"]["question"], ref=case["reference"], ans=answer
    )
    r = brt.converse(
        modelId=JUDGE,  # a different model family from the generator
        messages=[{"role": "user", "content": [{"text": prompt}]}],
        inferenceConfig={"maxTokens": 200, "temperature": 0},
    )
    raw = text_of(r)
    try:
        verdict = json.loads(re.search(r"\{.*\}", raw, re.DOTALL).group(0))
        return int(verdict["score"]), str(verdict.get("reason", ""))
    except (AttributeError, ValueError, KeyError):
        return 0, f"unparseable judge output: {raw[:120]}"  # fail closed


def check(case: dict) -> str | None:
    """None if the case passes, otherwise the reason it failed."""
    answer = generate(case["vars"])
    low = answer.lower()
    missing = [w for w in case.get("must_include", []) if w.lower() not in low]
    banned = [w for w in case.get("must_not_include", []) if w.lower() in low]
    if missing or banned:  # cheap string checks before paying for a judge call
        return f"missing={missing} banned={banned}"
    score, reason = judge(case, answer)
    return None if score == 1 else f"judge: {reason}"


def main(golden: str, junit: str) -> int:
    with open(golden, encoding="utf-8") as f:
        cases = [json.loads(line) for line in f if line.strip()]
    suite = ET.Element("testsuite", name="llm-evals", tests=str(len(cases)))
    failed = 0
    for case in cases:  # a Bedrock error raises and fails the build: fail closed
        tc = ET.SubElement(suite, "testcase", classname="golden", name=case["id"])
        if reason := check(case):
            failed += 1
            ET.SubElement(tc, "failure", message=reason[:500])
    suite.set("failures", str(failed))
    ET.ElementTree(suite).write(junit, encoding="utf-8", xml_declaration=True)
    rate = (len(cases) - failed) / len(cases)
    print(f"pass rate {rate:.1%} on {len(cases)} cases (gate {THRESHOLD:.0%})")
    return 0 if rate >= THRESHOLD else 1


if __name__ == "__main__":
    sys.exit(main(sys.argv[1], sys.argv[2]))

Before you run it:

Permissions. The build role needs ssm:GetParameters, bedrock:RenderPrompt, bedrock:InvokeModel on the inference profiles and the foundation models in every Region they route to, and the CodeBuild report actions.
Threshold. Run the suite on today's prompt, set the gate just below that score, and raise it as the golden set grows.
Cost. Each passing case costs two model calls, plus build minutes ($0.005 a minute for general1.small in AWS's pricing example).

Add the pitfall 5 lifecycle check as another command, and a model EOL date fails the build three months early.

Decide: which tools, and what it costs elsewhere

Decision	Choose the first when	Choose the second when
CDK Pipelines vs. GitHub Actions	Your infrastructure is CDK and you want a self-updating pipeline	You deploy Terraform or mixed tools, or already live in GitHub
CDK vs. Terraform	AWS-only, TypeScript team, you want `cdk drift` and drift-aware change sets	Multi-cloud or SaaS providers, or `plan` review is your change control
Bedrock Prompt Management vs. prompt files only	You're on Bedrock and want versions callers can't override	You use several model providers
Bedrock or AgentCore evaluations vs. promptfoo	You want managed jobs, built-in metrics or agent checks	You want evals beside unit tests, for any provider
CloudWatch GenAI observability vs. open-source OTLP backend	Mostly AWS, and you want AgentCore dashboards with no extra infrastructure	Multi-cloud, or you must own retention

Maintainability work pulls on the other eight requirements:

Other NFR	What maintainability work does to it
Security	Pipelines concentrate power, so guard the pipeline role
Compliance	Git history and prompt versions record who changed what
Accessibility	Checks such as axe-core run in the same build as lint
Performance	Canaries add minutes per deploy, tracing a little per request
Availability	Small, reversible changes lower change fail rate, and drift breaks failovers
Scalability	IaC stamps out new cells and Regions instead of hand-building them
Cost	Evals spend tokens on every change, and traces cost storage
Sustainability	Eval runs and long trace retention burn hidden compute

Who: owners and the shared responsibility split

Activity	Product owner	Tech lead	Developers	Platform / SRE
Pipeline, canaries, rollback alarms	I	A	C	R
IaC and drift checks	I	A	R	R
Golden set and eval threshold	A	C	R	I
Prompt and model versions	C	A	R	I
Model, runtime and MCP inventory	I	A	C	R
Review rules for AI-generated code	I	A	R	C

R = responsible, A = accountable, C = consulted, I = informed.

Who patches what depends on the compute type:

EC2. You patch the OS, agents and AMIs.
Containers. You own images and base-image updates. On Fargate, AWS patches the host.
Lambda. AWS patches the runtime. You move off it before its block dates.
Managed AI (Bedrock). AWS and the provider run the model and publish its lifecycle dates. You pin IDs, own the evals and migrate before EOL.

Monday checklist

[ ] Make the pipeline the only way to production, with alarm-based rollback on one service.
[ ] Run a drift check against production and fix what it finds in code.
[ ] Move every prompt and model ID into Git, then search for model ID strings anywhere else.
[ ] Write 20 golden cases from real questions and record today's pass rate.
[ ] Check your judge against 50 human-labelled answers before you gate on it.
[ ] Add the model lifecycle check as a daily build.
[ ] Put the prompt version ARN on your spans, and plan the move off the X-Ray SDKs before February 25, 2027.
[ ] Pin every MCP server to an exact version and commit its tool schemas.
[ ] Track change fail rate and deployment rework rate per service.

Conclusion

Knight Capital didn't lose $460 million to a hard problem. It lost it to a manual deploy, a dead feature and a flag with two meanings. Every pitfall here is a system nobody can fully see or safely change.

AI gives the old mistakes new places to hide. A prompt edited in a console is a manual deploy. A model past its EOL date is an unpatched runtime. An MCP server on @latest is an unpinned dependency holding credentials. The fixes are the ones you know: version it, test it, trace it, and keep a way back. The new part is the eval suite, because "does it still work?" now has a fuzzy answer.

Give every change one meaning, one owner and one way back.

Series: Non-functional Application Requirements

Non-functional Application Requirements: An Introduction
Non-functional Application Requirements: Security
Non-functional Application Requirements: Compliance
Accessibility
Performance
Availability
Scalability
Maintainability
Cost (coming soon)
Sustainability (coming soon)

Cover image: AWS re:Invent 2023 – Keynote with Dr. Werner Vogels

Next up, Cost: one real, sourced case study of a team cutting its AWS bill, and what per-token pricing does to unit economics.

Sources

Non-functional Application Requirements: Scalability

Oluwasegun Adedigba — Wed, 11 Nov 2026 15:02:15 +0000

In 2023, the Prime Video team that monitors stream quality published something few teams admit to. Their first design, a distributed pipeline of serverless components orchestrated by AWS Step Functions, was "proven to only support around 5% of the expected load." It hit an account limit on Step Functions state transitions and paid for heavy S3 traffic on intermediate video frames. They moved the logic into one process on ECS on EC2 and reported a 90% cut in operating costs (InfoQ, 2023).

Nobody on that team picked bad services. Each one scaled fine on its own. What broke was the cost per unit of work, which grew faster than the load. Every scaling option buys you something and charges you for it, so let's put numbers on both.

TL;DR

Scalability means cost per unit of work stays flat as load grows. Elasticity is how fast capacity follows it.

Price one unit of load per option. In us-east-1, Lambda beats always-on containers only while they'd sit below about 35–40% busy.

Your database is usually the real ceiling. Find its limit before adding compute in front of it.

For AI, the scaling unit is tokens per minute and your Bedrock quota is the ceiling. Plan quota like capacity.

A queue with a concurrency cap is often cheaper than more capacity.

What: scalability, elasticity and performance

Scalability is the ability to take on more work by adding resources while the cost of each unit of work stays roughly flat. Twice the users for twice the money scales. A wall you can't spend past doesn't.

Elasticity is how quickly and automatically capacity follows the load. Performance, from Part 5, is how fast one request is at a given load. A fast system can fail to scale, and an elastic one can scale into a bill nobody planned.

Vertical scaling means a bigger machine. It needs no code changes and stops at the largest instance size, with a restart on the way up. Horizontal scaling means more machines behind a load balancer. It has no hard top, but only when any copy can serve any request, so your app has to be stateless, with sessions and caches in a shared store.

flowchart LR
  subgraph V["Vertical: a bigger box"]
    U1["Users"] --> B1["One large instance<br/>app plus session state"]
    B1 --> C1["Ceiling: largest size,<br/>restart to resize"]
  end
  subgraph H["Horizontal: more boxes"]
    U2["Users"] --> LB["Load balancer"]
    LB --> I1["Stateless copy 1"]
    LB --> I2["Stateless copy 2"]
    LB --> I3["Stateless copy N"]
    I1 --> S["Shared state<br/>DynamoDB or ElastiCache"]
    I2 --> S
    I3 --> S
    S --> C2["Ceiling moves to<br/>the shared store"]
  end

Horizontal scaling removes the ceiling from compute and moves it to the shared state store.

To test it, track cost per unit of work (per 1,000 requests or per million tokens), maximum throughput at your latency SLO, scale-out time from a load step to serving capacity, and headroom, the capacity you pay for to cover that gap.

Why: scaling fails in two expensive directions

Under-scaling costs you the event you were built for. On November 15, 2022, the Eras Tour presale sent Ticketmaster 3.5 billion system requests, four times its previous peak, and about 15% of site interactions had issues (Ticketmaster, 2022). Ticketmaster blamed bots and fans without codes. Either way, the bill came due on launch day.

Over-scaling costs you every month, quietly. In Flexera's 2026 State of the Cloud report, respondents estimated that 29% of their cloud spend is wasted, the first rise in five years, with AI named as a growing source of it (Flexera, March 2026). Headroom you add for a launch and never take back is an easy way to land in that number.

And the ceiling is rarely AWS itself. During Prime Day 2025, DynamoDB peaked at 151 million requests per second and SQS at 166 million messages per second (AWS News Blog, 2025). Your design decides how far you get, and at what price.

When: where scalability enters the lifecycle

Design. Name your scaling unit and the component with the lowest ceiling. Price one unit at today's load and at ten times it.
Build. Keep compute stateless, make writes idempotent, and pool database connections from the first commit.
Test. Load test past your peak until something breaks, with a sudden step, since a smooth ramp hides warm-up gaps.
Operate. Put cost per unit next to latency on the dashboard. Alarm on queue age and quota use, not only CPU.

The database is the real ceiling

Here's the pattern I'd expect in most designs. Compute scales out cleanly, and every new copy opens a connection to the same database. Lambda can add 1,000 execution environments every 10 seconds per function. An RDS for MySQL db.m7g.large allows about 630 connections by default (RDS quotas). One burst can use up the pool in seconds.

flowchart TB
  B["Traffic burst"] --> L["Lambda adds up to 1,000<br/>environments per 10 s"]
  L --> Q{"Where does<br/>each one connect?"}
  Q -->|"Directly"| DB["RDS db.m7g.large<br/>about 630 connections"]
  DB --> X["Too many connections<br/>errors under load"]
  Q -->|"Through RDS Proxy"| P["Shared connection pool"]
  P --> DB2["Same database,<br/>steady connection count"]
  Q -->|"Key-value access"| D["DynamoDB<br/>3,000 RCU and 1,000 WCU<br/>per partition"]
  D --> K{"Partition key<br/>spread evenly?"}
  K -->|"No"| HOT["Hot partition throttles<br/>while the table looks idle"]
  K -->|"Yes, or write sharding"| OK["Throughput grows<br/>with partitions"]

Scaling compute only helps when the database behind it can accept the extra connections and writes.

Your options, from cheapest change to most expensive:

RDS Proxy pools connections for $0.015 per database vCPU-hour in us-east-1, so a 2-vCPU instance adds 2 × $0.015 × 730 = $21.90 a month. Statements over 16 KB and some session state pin a client to one connection.
Read replicas take reads. Aurora allows up to 15, with lag "usually much less than 100 milliseconds." They do nothing for writes.
DynamoDB gives each partition 3,000 read and 1,000 write units per second, so a key like plant#date that every write hits is your ceiling however big the table gets. Write sharding adds a suffix (say 1–200) to spread writes, at the cost of reading every shard.
Sharding a relational database is the last resort, because cross-shard joins become your problem. Aurora PostgreSQL Limitless Database (generally available since October 2024) does the routing, but prove it on your queries first.

Where: what each option buys you

Each option trades money for work you no longer do.

EC2 Auto Scaling adds no fee beyond the instances. AWS says to "set the target value as high as possible with a reasonable buffer," because headroom is money. Predictive scaling needs 24 hours of history (14 days is better), forecasts 48 hours ahead and only scales out. Harbor Works, the fictional plant from the Luso companion app, has three production lines whose dashboards fill at every shift change, the textbook predictive case. Choose EC2 for steady load with Savings Plans (up to 72% off), unless you don't want to own AMIs and patching.

ECS on Fargate is the middle ground, with no hosts, per-second billing and Fargate Spot at up to 70% off for interruptible work. It fits a containerized API with daytime load and a small team, not GPU work.

EKS pays off when many workloads share one platform. The HPA scales pods, KEDA scales them from sources like SQS depth, and Karpenter picks and packs nodes. A NodePool's limits is a spending ceiling, and consolidationPolicy: WhenEmptyOrUnderutilized repacks pods onto fewer nodes. Choose EKS when Kubernetes is already your platform.

Lambda stays the default for spiky and low-traffic work. Its limit is concurrency, not money, which is why its most useful scaling setting is a cap.

Decide: price one unit of load

Let's compare options on one unit, one vCPU and 2 GB of memory kept busy for one hour, using us-east-1 on-demand prices from the AWS Price List API, pulled October 29, 2026.

Option	Price per busy unit-hour	How it scales	AWS handles	You still own
Lambda, Arm, 2 GB	$0.096 + $0.0072 for 36,000 requests at 100 ms = $0.103, only while running	1,000 environments per 10 s, 1,000 per Region by default	Servers, OS, AZs, scaling	Concurrency caps, connections, idempotency
Lambda Managed Instances	EC2 price + 15% fee	Doubles in about 5 min, on CPU	Instances, patching	Thread safety, instance choice
ECS on Fargate, Arm	$0.03238 + 2 × $0.00356 = $0.0395 per running hour	Target tracking	Hosts, OS	Task size, policy, AZ placement
EC2 Auto Scaling, c7g.large	$0.0725 ÷ 2 = $0.0363, or $0.0239 with a 1-year RI	Target tracking plus predictive	Hardware	OS, AMIs, warm-up, headroom
EKS with Karpenter	EC2 price + $73/month per cluster on standard support	HPA or KEDA, plus Karpenter	Control plane	Nodes, add-ons, upgrades

Containers and instances look cheaper, but you pay for them while they're idle. At a 60% CPU target, a busy unit-hour really costs $0.0363 ÷ 0.6 = $0.060 on EC2 and $0.0395 ÷ 0.6 = $0.066 on Fargate. Lambda bills only while it runs.

That gives you a break-even line. Fargate wins once its capacity would be busy more than $0.0395 ÷ $0.103 ≈ 38% of the time, EC2 above about 35%. Below that, Lambda wins, and at zero traffic it costs nothing.

Three caveats. A 2 GB function gets slightly more than one vCPU (1,769 MB equals one). Lambda bills wall-clock time, including waits on a database, while a container serves other requests during the wait, so I/O-heavy work tips toward containers sooner. And none of this prices your team's time.

So here's the rule I'd use. Start on Lambda when traffic is spiky or unknown. Move a service to Fargate or Lambda Managed Instances once your dashboard shows it busy for most of the day. Reach for EKS when the platform already exists for other reasons.

What more scale costs elsewhere

Every gain in scale gets paid for somewhere else in this series.

NFR	Helps	Hurts
Performance	Headroom and Priority tier cut queueing	Queue levelling adds latency by design
Availability	Stateless copies across AZs survive failures	Each autoscaled dependency adds a quota that can run out
Security	Smaller, identical units are easier to patch	Every new node pool, proxy and queue is more attack surface
Compliance	In-Region and geographic inference profiles keep processing in one geography	Global profiles move prompts across Regions to find capacity
Maintainability	Managed scaling means less code to own	EKS, Karpenter and self-hosted vLLM add upgrade work
Cost	Scale-to-zero, Flex and batch cut idle spend	Headroom, Reserved tiers and GPU nodes bill whether busy or not
Sustainability	Consolidation and Arm instances mean fewer idle watts	Over-provisioned GPUs burn energy for no work

Queue-based load levelling

A queue lets you size for the average instead of the peak. Producers write as fast as they like and consumers work at a rate your downstream systems survive, for $0.40 per million SQS standard requests after the free million.

On Lambda, the control is the event source mapping's MaximumConcurrency, from 2 to 1,000. Without it, Lambda adds up to 300 concurrent invocations a minute, up to 1,250 per mapping (Lambda docs). That's great for throughput and bad for a database with 630 connections. Provisioned mode goes the other way, reserving pollers at $0.00925 per poller-hour, and can't be combined with the cap.

flowchart LR
  L1["Line 1 shift end"] --> Q["SQS queue<br/>absorbs the burst"]
  L2["Line 2 shift end"] --> Q
  L3["Line 3 shift end"] --> Q
  Q --> E["Event source mapping<br/>MaximumConcurrency 3"]
  E --> W["Lambda workers<br/>one model call each"]
  W --> BR["Amazon Bedrock<br/>TPM quota"]
  W -->|"Throttled: hand back"| Q
  Q -->|"Fails 5 times"| DLQ["Dead-letter queue"]
  Q -.->|"Oldest message age"| A["CloudWatch alarm<br/>backlog over 15 min"]

The queue turns a burst from three lines finishing their shifts at once into a steady stream that stays under the quota.

You're trading latency for a flat bill, so the queue's age becomes the SLO. Alarm on ApproximateAgeOfOldestMessage, not queue length.

Hands-on: size a queue consumer from your quota

At shift end, Harbor Works asks a model to summarize each line's telemetry for the next supervisor, and all three lines finish at once. Let's size the consumers from the Bedrock quota.

Each call reserves its input tokens plus maxTokens against your TPM quota when it starts, and each worker makes 60 ÷ seconds_per_call calls a minute. So the safe worker count is quota × seconds_per_call ÷ (60 × tokens_per_call). With an example quota of 200,000 TPM, 3,400 tokens per call and 4-second calls, that's 200,000 × 4 ÷ (60 × 3,400) ≈ 3.9, so three workers.

Checked against the Terraform AWS provider docs for aws_lambda_event_source_mapping and aws_autoscaling_policy, and the Lambda SQS configuration guide, on Oct 29, 2026. terraform validate passes with Terraform 1.13.4 and AWS provider 6.71.0. Not deployed — test in a sandbox account first.

terraform {
  required_providers {
    aws = { source = "hashicorp/aws", version = "~> 6.0" }
  }
}

provider "aws" {
  region = "us-east-1"
}

variable "worker_zip" { type = string }      # zip containing worker.py (below)
variable "worker_role_arn" { type = string } # needs SQS read/delete + bedrock:InvokeModel

locals {
  # Size the fleet from the quota, not from the queue depth.
  tpm_quota        = 200000 # example value: read yours from Service Quotas
  tokens_per_call  = 3400   # input tokens + maxTokens, reserved when a call starts
  seconds_per_call = 4      # measured p50 duration of one worker invocation
  max_workers      = max(2, floor(local.tpm_quota * local.seconds_per_call / (60 * local.tokens_per_call)))
  fn_timeout       = 30
}

resource "aws_sqs_queue" "dlq" {
  name                      = "harbor-summaries-dlq"
  message_retention_seconds = 1209600 # 14 days to inspect bad messages
}

resource "aws_sqs_queue" "jobs" {
  name                       = "harbor-summaries"
  visibility_timeout_seconds = 6 * local.fn_timeout # AWS guidance: at least 6x the function timeout
  redrive_policy = jsonencode({
    deadLetterTargetArn = aws_sqs_queue.dlq.arn
    maxReceiveCount     = 5 # throttled jobs come back, broken ones stop here
  })
}

resource "aws_lambda_function" "worker" {
  function_name = "harbor-summary-worker"
  role          = var.worker_role_arn
  filename      = var.worker_zip
  handler       = "worker.handler"
  runtime       = "python3.13"
  architectures = ["arm64"]
  memory_size   = 512
  timeout       = local.fn_timeout
  # Must be >= the event source's maximum concurrency, or Lambda throttles messages.
  reserved_concurrent_executions = max(5, local.max_workers)
  environment {
    variables = {
      MODEL_ID     = "global.amazon.nova-2-lite-v1:0"
      SERVICE_TIER = "flex" # async work: 50% off Standard, slower is fine
      MAX_TOKENS   = "400"
    }
  }
}

resource "aws_lambda_event_source_mapping" "jobs" {
  event_source_arn        = aws_sqs_queue.jobs.arn
  function_name           = aws_lambda_function.worker.arn
  batch_size              = 1 # one model call per invoke keeps the sizing math honest
  function_response_types = ["ReportBatchItemFailures"]
  scaling_config {
    maximum_concurrency = local.max_workers # the ceiling that protects the TPM quota
  }
}

# The queue's age is the SLO you are trading for a flat model bill.
resource "aws_cloudwatch_metric_alarm" "backlog_age" {
  alarm_name          = "harbor-summaries-too-old"
  namespace           = "AWS/SQS"
  metric_name         = "ApproximateAgeOfOldestMessage"
  dimensions          = { QueueName = aws_sqs_queue.jobs.name }
  statistic           = "Maximum"
  period              = 60
  evaluation_periods  = 5
  threshold           = 900 # 15 minutes: time to raise the quota or the tier
  comparison_operator = "GreaterThanThreshold"
}

The worker makes one attempt per message and hands throttled messages back to the queue, so the queue does the retrying instead of the SDK.

Checked against the Bedrock Converse API and the Nova 2 Lite model card on Oct 29, 2026. Unit-tested with botocore Stubber on boto3 1.43.121 (a success, a throttle handed back, a validation error raised). Package a current boto3 with the function, because serviceTier needs a recent SDK. Not deployed — test in a sandbox account first.

import json
import os

import boto3
from botocore.config import Config
from botocore.exceptions import ClientError

# One attempt only: the queue is the retry mechanism, not the SDK.
bedrock = boto3.client("bedrock-runtime", config=Config(retries={"max_attempts": 1, "mode": "standard"}))

MODEL_ID = os.environ["MODEL_ID"]
SERVICE_TIER = os.environ.get("SERVICE_TIER", "default")
MAX_TOKENS = int(os.environ.get("MAX_TOKENS", "400"))
RETRYABLE = {"ThrottlingException", "ServiceUnavailableException", "ModelNotReadyException"}


def summarize(job: dict) -> str:
    resp = bedrock.converse(
        modelId=MODEL_ID,
        system=[{"text": "Summarize this shift's machine telemetry for a line supervisor in five bullets."}],
        messages=[{"role": "user", "content": [{"text": job["telemetry_digest"]}]}],
        inferenceConfig={"maxTokens": MAX_TOKENS},  # reserved against TPM at call start
        serviceTier={"type": SERVICE_TIER},
    )
    return resp["output"]["message"]["content"][0]["text"]


def handler(event, context):
    failures = []
    for record in event["Records"]:
        try:
            job = json.loads(record["body"])
            summary = summarize(job)
            print(json.dumps({"line": job["line_id"], "chars": len(summary)}))  # write it wherever you store results
        except ClientError as err:
            if err.response["Error"]["Code"] in RETRYABLE:
                # Hand the message back. It reappears after the visibility timeout.
                failures.append({"itemIdentifier": record["messageId"]})
            else:
                raise
    return {"batchItemFailures": failures}

The Harbor Works API tier runs on EC2, so it gets target tracking for minute-to-minute load and a predictive policy for shift changes. With both active, the group runs at whichever capacity is higher.

Checked against the aws_autoscaling_policy docs and the predictive scaling guide on Oct 29, 2026. Validated with the configuration above. Not deployed — test in a sandbox account first.

variable "asg_name" { type = string }

resource "aws_autoscaling_policy" "cpu_target" {
  name                      = "harbor-api-cpu-60"
  autoscaling_group_name    = var.asg_name
  policy_type               = "TargetTrackingScaling"
  estimated_instance_warmup = 120
  target_tracking_configuration {
    predefined_metric_specification {
      predefined_metric_type = "ASGAverageCPUUtilization"
    }
    target_value = 60 # higher = cheaper, lower = more headroom for spikes
  }
}

resource "aws_autoscaling_policy" "shift_forecast" {
  name                   = "harbor-api-shift-forecast"
  autoscaling_group_name = var.asg_name
  policy_type            = "PredictiveScaling"
  predictive_scaling_configuration {
    mode                   = "ForecastOnly" # compare forecast to reality before ForecastAndScale
    scheduling_buffer_time = 600            # launch 10 minutes before the forecast hour
    metric_specification {
      target_value = 60
      predefined_metric_pair_specification {
        predefined_metric_type = "ASGCPUUtilization"
      }
    }
  }
}

Leave it in ForecastOnly for two weeks, compare the forecast with real load, then switch to ForecastAndScale.

Scalability in the age of AI

AI as a workload: tokens per minute are the unit

Once a model sits in your request path, vCPUs stop being the number that matters. Bedrock limits you per model and per Region in tokens per minute (TPM) and, for most models, requests per minute, with separate quotas for cross-Region and in-Region use. Increases aren't automatic, so request quota the way you'd plan instances for a launch.

The quota is spent in a way that surprises people. A call starts by reserving its input tokens plus your full max_tokens, then settles on input plus output tokens times a burndown rate: 5x for Anthropic models up to version 4.7, higher for some newer ones, 1x for most others (Bedrock docs). You're billed only for real tokens, so a careless max_tokens of 32,000 throttles you long before the bill looks big. Watch the EstimatedTPMQuotaUsage metric, added in March 2026.

Agents multiply load

One user action is no longer one call. Anthropic measured its agents using about 4x the tokens of a chat interaction, and its multi-agent systems about 15x (Anthropic, 2025).

flowchart TB
  U["Supervisor asks: why did line 2 slow down?"] --> A["Agent"]
  A --> M1["Model call 1: plan"]
  M1 --> T1["Tool: query telemetry"]
  M1 --> T2["Tool: read maintenance log"]
  T1 --> M2["Model call 2: read results"]
  T2 --> M2
  M2 --> T3["Tool: compare with line 1"]
  T3 --> M3["Model calls 3 to N:<br/>context grows each turn"]
  M3 --> R["One answer"]
  M3 -.->|"Each call spends"| QT["TPM quota and<br/>tool API capacity"]

One question to an agent becomes N model calls and M tool calls, and every one of them counts against a quota.

Let's put numbers on it with Claude Sonnet 4.6 at $3.30 per million input and $16.50 per million output tokens (US geo profile, us-east-1). One call with 6,000 input and 400 output tokens costs $0.0198 + $0.0066 = $0.0264. Six such calls cost $0.158 per question and, with the 5x burndown, use 36,000 + 2,400 × 5 = 48,000 quota tokens. On a 200,000 TPM quota, that's about four questions a minute for the whole account, so cap agent turns in code.

Bedrock capacity options, weighed by cost

Bedrock prices Priority at a 75% premium over Standard, Flex at a 50% discount and batch at 50% below on-demand for selected models (Bedrock pricing). The us-east-1 price list agrees, with Nova 2 Lite input at $0.33 per million on Standard, $0.165 on Flex and $0.5775 on Priority.

Priority buys the front of the line for customer-facing peaks and shares the Standard quota. Support varies, and Sonnet 4.6 offers neither Priority nor Flex.
Flex is the cheap default for anything nobody is waiting on, like the shift summaries above.
Batch halves the price of evals and backfills (Sonnet 4.6 input drops from $3.30 to $1.65). It runs from S3 files, and its Converse format doesn't support tool calls.
Reserved sells a fixed TPM rate for 1 or 3 months, targets 99.5% uptime and overflows to Standard. Here's the catch. For Sonnet 4.6 (US geo, 1 month), 1,000 input TPM costs $0.198 an hour, exactly 1,000 × 60 × $3.30 per million. You break even only if you use every reserved token every minute, or 90% of them on the 10%-cheaper 3-month term. The 100,000 input plus 10,000 output TPM minimum is about $21,681 a month. Reserved buys certainty, not a discount.
Provisioned Throughput bills model units hourly and is required for custom models. Nova 2 Lite is $55 an hour on a 1-month term, $40,150 a month.

Self-host on EKS, or stay on Bedrock?

Ministral 3 8B makes a fair test, because the same Apache 2.0 open-weight model runs on Bedrock at $0.15 per million tokens (input or output) and on vLLM on your own GPU. Its model card says it fits in 12 GB of VRAM in FP8.

Self-hosting needs two nodes for availability. Two g6.xlarge (one 24 GB L4 GPU each) at $0.8048 an hour plus the EKS fee cost 2 × $587.50 + $73 = $1,248 a month, the price of 8.3 billion Bedrock tokens, or 3,170 tokens per second around the clock.

Now add people. Say drivers, upgrades and on-call take a quarter of an engineer, and call that $4,000 a month (use your own number). Break-even jumps to $5,248 ÷ $0.15 ≈ 35 billion tokens a month, about 13,300 tokens per second sustained. Against Flex or batch at $0.07, it's 75 billion. A 1-year RI cuts the hardware to about $838, which barely moves the total.

flowchart TB
  S["Monthly tokens on one open-weight model"] --> D{"Above break-even?<br/>nodes plus ops divided<br/>by Bedrock price"}
  D -->|"No"| BR["Stay on Bedrock<br/>Flex or batch for async"]
  D -->|"Yes"| U{"Load steady<br/>around the clock?"}
  U -->|"No, bursty"| BR
  U -->|"Yes"| T{"Can your nodes<br/>sustain it at your SLO?"}
  T -->|"Not measured"| BM["Benchmark vLLM<br/>on your prompts first"]
  T -->|"Yes"| SH["Self-host on EKS<br/>Karpenter, GPU or Inferentia"]
  T -->|"Needs more nodes"| D

Self-hosting pays off only when your token volume is high, steady and measured on your own hardware.

That last box matters. There's no published AWS benchmark for this model on L4 or inf2.xlarge ($0.7582 an hour) that I can point you to, so measure it with vLLM's benchmark tooling on your own prompts. Self-host for data control or truly steady volume, not to save money at moderate scale.

Vector indexes scale on a different curve

Retrieval scales on storage and query rate, not tokens. S3 Vectors holds up to 2 billion vectors per index at $0.06 per GB-month plus $2.50 per million queries and a small processing fee. AWS's own example, 10 million vectors and a million queries a month, costs $11.38, with frequent queries around 100 ms. OpenSearch Serverless is faster and richer. A first classic collection bills at least 2 OCUs at $0.24 per OCU-hour, about $350 a month before any data. The newer NextGen collections have no minimum and scale to zero after 10 idle minutes.

One limit catches people out. An S3 Vectors index takes at most 2,500 inserts and deletes per second, so re-embedding 10 million vectors into one index takes at least 67 minutes. Spread big re-embeds across indexes, and keep the hot set users query all day in OpenSearch.

AI as an engineering aid

AI assistants are good at the tedious parts. They draft a Terraform policy, write a k6 script or explain a throttling graph, and AWS DevOps Agent (generally available since March 2026) works on resolving and preventing incidents.

They're weak on exactly the numbers this article depends on. Expect last year's prices, invented default quotas, and CPU target tracking for a workload that waits on I/O. So pull prices from the Price List API, read quotas from Service Quotas, and treat an AI-written scaling policy as untested until a load test has pushed past its target.

Who: owners and the shared responsibility split

Activity	Product owner	Architect	Developers	Platform / SRE
Scaling unit, unit-cost target, growth forecast	A	R	C	C
Ceiling analysis and compute choice	C	R/A	C	C
Stateless code, pooling, idempotency, agent turn caps	I	C	R/A	C
Scaling policies, quotas, load tests, cost dashboards	I	C	C	R/A

R = responsible, A = accountable, C = consulted, I = informed.

For compute, the "AWS handles" and "You still own" columns in the Decide table are the split. For managed AI, AWS serves the model, honors your tier and enforces quotas. You pick the model and tier, request quota early, set max_tokens and cap agent fan-out.

Monday checklist

[ ] Pick your scaling unit and put cost per unit of work on your main dashboard.
[ ] Find your lowest ceiling: database connections, a hot partition key, or a model quota.
[ ] Put RDS Proxy in front of any relational database that Lambda or autoscaled containers talk to.
[ ] Work out where Lambda and always-on capacity break even for your busiest service, using this year's prices.
[ ] Add MaximumConcurrency to every SQS event source that calls something with a hard limit.
[ ] Set max_tokens close to real output sizes and alarm on EstimatedTPMQuotaUsage.
[ ] Move evals, backfills and summaries to Flex or batch.
[ ] Count model and tool calls per user action for each agent, and cap the turns in code.

Conclusion

Every option here scales. They differ on the price of each extra unit. Lambda charges more per busy hour and nothing for idle time, and containers flip that. Reserved model capacity buys certainty at the full on-demand rate. Self-hosting buys control that pays off only at volumes most teams never reach.

So start with the ceiling, because no amount of compute fixes a database at its connection limit or an agent that spends a whole minute of quota on four questions. Then price the unit and pick the option whose cost curve matches your load. Prime Video didn't need faster services. It needed a design whose cost grew with the work instead of ahead of it.

Scale is easy to buy. Find the ceiling and price the unit before you do.

Series: Non-functional Application Requirements

Non-functional Application Requirements: An Introduction
Non-functional Application Requirements: Security
Non-functional Application Requirements: Compliance
Accessibility
Performance
Availability
Scalability
Maintainability (coming soon)
Cost (coming soon)
Sustainability (coming soon)

Cover image: AWS re:Invent 2023 – Keynote with Dr. Werner Vogels

Next up, Maintainability: the pitfalls that make a system harder to change every month, and what CodeBuild, CodeDeploy and CloudWatch can and can't do about them.

Sources

Non-functional Application Requirements: Availability

Oluwasegun Adedigba — Wed, 11 Nov 2026 15:01:56 +0000

At 11:48 PM Pacific on October 19, 2025, two copies of the automation that manages DynamoDB's DNS in us-east-1 raced each other and left the Regional endpoint with an empty DNS record. DNS was back by 2:25 AM, but the outage wasn't over. EC2's host-management system depends on DynamoDB and fell into "congestive collapse". New instances came up before their network state had propagated, so load balancer health checks pulled healthy capacity out of service. AWS put the end at 2:20 PM, more than 14 hours in (AWS post-event summary, 2025).

One line in that summary deserves more attention. EC2 instances launched before the event "remained healthy" throughout. The systems that rode it out already had what they needed, and that idea runs through this article.

TL;DR

Promise an SLO with an error budget, never "zero downtime". At 99.9% over 30 days you get 43 minutes.

Your ceiling is the product of your hard dependencies. Copies only help if they fail independently.

Design for static stability: recover with capacity you already have, not a control plane mid-incident.

Let RTO and RPO pick your DR strategy, then rehearse it with AWS FIS.

For AI, treat 429s, 503s and model end-of-life as outages, and build a degradation ladder whose last rung needs no model.

What: availability, and the words people swap for it

Availability asks one question. Can users get the function they were promised, right now? Durability asks whether the data still exists, and resilience asks how well the system absorbs a hit and recovers. S3 Standard is designed for 99.99% availability and eleven nines of durability, so a bucket you can't reach for ten minutes hasn't lost a byte.

Four terms make it testable. An SLI is good events over total events, such as /orders requests answered without a 5xx in under a second. An SLO is the target for that ratio, say 99.9% over 30 days. An SLA is the contract with credits, set below your SLO. The error budget is what's left, so 3 million requests at 99.9% allow 3,000 failures (Google SRE Workbook).

Each nine divides your allowance by ten. Over 30 days, 99.9% gives you 43 minutes, 99.99% about 4, and 99.999% just 26 seconds.

Part 1 of this series promised "zero downtime, no matter what." I'm taking that back. What you can honestly promise is an SLO, a budget, and a plan for when the budget runs out.

Why: downtime has a price, and someone else often sets it

The October outage drew more than 17 million Downdetector reports, the most of any outage in 2025 (Ookla, December 2025). In Uptime Institute's 2025 survey, one in five operators said their last major outage cost over $1 million (Uptime Institute, Annual Outage Analysis 2026).

The number I'd put on a slide is from the same report. Over nine years of publicly reported outages, third parties such as cloud, telecom and colocation providers caused about two-thirds. Part of your availability is somebody else's architecture, so design as if it will fail.

The theory: four models for your next design review

1. Error budgets turn "be reliable" into numbers

While budget remains, you ship. Once it's spent, a written error budget policy kicks in, usually freezing risky changes until you're back within SLO. Alert on burn rate, how fast the budget is going, not on single errors. Google's SRE Workbook suggests paging when an hour eats 2% of a 30-day budget, a burn rate of 14.4.

2. Dependencies multiply, and copies only add if they're independent

Hard dependencies sit in series, so their availabilities multiply: A1 × A2 × … × An. Redundant copies sit in parallel, so all of them must fail at once: 1 − (1 − A)^n (AWS, Availability and Beyond whitepaper).

Let's put numbers on it with Harbor Works, the fictional plant in my open-source app Luso. Its line copilot runs API Gateway → Lambda → DynamoDB, then calls Amazon Bedrock. With the published SLAs (API Gateway and Lambda 99.95%, DynamoDB 99.99%, Bedrock 99.9%):

flowchart LR
  AG["API Gateway<br/>99.95%"] --> LA["Lambda<br/>99.95%"]
  LA --> DB["DynamoDB<br/>99.99%"]
  DB --> S["Serverless stack ceiling<br/>99.89%, about 48 min a month"]
  S --> BR["Bedrock<br/>99.9%"]
  BR --> C["Copilot ceiling<br/>99.79%, about 91 min a month"]

Each hard dependency in series lowers the ceiling, and the model call alone takes it from about 48 to about 91 minutes a month.

The serverless stack alone is 0.9995 × 0.9995 × 0.9999 = 99.89%, about 48 minutes a month.
Add Bedrock in series and you get 99.89% × 99.9% = 99.79%, about 91 minutes.
Copy everything to a second Region with its own Bedrock: 1 − (1 − 0.9979)² = 99.9996%, about 11 seconds.
Copy the stack but keep both Regions calling Bedrock in one Region: (1 − (1 − 0.9989)²) × 0.999 = 99.90%, about 43 minutes.

Compare the last two. Both run in two Regions for a similar bill, yet one shared dependency throws away three nines. A shared deploy pipeline or identity provider does the same, just more quietly.

flowchart LR
  U["User"] --> R53["Route 53 DNS"]
  R53 --> A["Region A stack<br/>99.89%"]
  R53 --> B["Region B stack<br/>99.89%"]
  A --> X["Shared dependency<br/>model Region, deploy pipeline, identity"]
  B --> X

Two Region stacks only count as parallel when nothing like the box on the right sits behind both of them.

Treat this as a ceiling, not a forecast. AWS calls the calculation "a rough order of magnitude." It's still the fastest way I know to catch a design that can't meet its SLO even on paper.

3. Decide how much is allowed to break at once

Blast radius is a choice you make in advance. AZs and Regions are boundaries AWS gives you. Cells are ones you build: isolated copies of the workload, each serving a slice of traffic by partition key. With 10 cells, one failure leaves 90% of requests untouched (AWS cell-based architecture guide). Keep the router "the thinnest possible layer", split on a natural key (for Harbor Works, the production line), and deploy one cell at a time.

flowchart LR
  T["Requests"] --> RT["Cell router<br/>thinnest possible layer<br/>splits on production line"]
  RT --> C1["Cell 1<br/>isolated copy of the workload"]
  RT --> C2["Cell 2<br/>failed"]
  RT --> C3["Cells 3 to 10<br/>isolated copies of the workload"]
  C1 --> OK["90% of requests untouched"]
  C3 --> OK
  C2 --> BAD["10% of requests affected"]

With ten cells, a failure stays inside one cell's slice of traffic instead of reaching every user.

4. Static stability: recover without asking for anything new

"In a statically stable design, the overall system keeps working even when a dependency becomes impaired" (Mike Furr, Amazon Builders' Library, 2019). The control plane creates and changes resources. The data plane keeps existing ones working, and it's simpler and built for higher availability. October was the textbook case: running instances kept serving while launches failed.

Three rules follow.

Pre-provision across AZs. If peak needs 10 instances, run 15 across three AZs so losing a zone needs no launches. AWS's DR whitepaper is blunt about why: "Auto Scaling is a control plane activity".
Fail over with data-plane actions. Route 53's control plane lives in us-east-1, while its data plane answers from over 200 points of presence. Health-check failover is data plane. Editing a record mid-incident isn't.
Make automatic failover cautious. After October, AWS is adding "a velocity control mechanism to limit the capacity a single NLB can remove." Keep your own health checks shallow, because a deep check on a shared dependency turns one failure into a cascade.

flowchart LR
  I["Incident<br/>an AZ or a dependency is impaired"] --> CP["Recovery that asks the control plane"]
  I --> DP["Recovery on the data plane"]
  CP --> CP1["Launch replacement instances<br/>Auto Scaling"]
  CP --> CP2["Edit a Route 53 record"]
  CP1 --> R1["Can fail mid-incident<br/>as launches did in October"]
  CP2 --> R1
  DP --> DP1["Pre-provisioned capacity<br/>15 instances across 3 AZs for a peak of 10"]
  DP --> DP2["Route 53 health-check failover"]
  DP1 --> R2["Keeps working with<br/>what already exists"]
  DP2 --> R2

A statically stable recovery only uses what's already running, so it works even while the control plane is struggling.

When: where availability enters the lifecycle

At design time, set one SLO per journey and mark every dependency hard or soft. In the build, put a timeout on every call, retries in one layer and idempotent writes. Before launch, run one fault-injection experiment per failure mode and one real failover. In operation, alarm on burn rate and drill on a calendar.

Where: the AWS pieces, and when not to use them

Route 53 health checks with failover records give you data-plane Regional failover in about 90 seconds at the default settings, plus the 60-second TTL on ALB DNS names. A check on an AWS endpoint costs $0.50 a month, plus $1 per optional feature such as HTTPS.
Amazon Application Recovery Controller (ARC) zonal shift and autoshift move traffic off an impaired AZ. Autoshift needs weekly practice runs of about 30 minutes, so pre-scale the other AZs first. ARC Region switch runs multi-Region recovery plans with "a data plane in each AWS Region". ARC readiness check closed to new customers on April 30, 2026, and AWS points you to Region switch instead.
Aurora Global Database has RPO 0 on a planned switchover, but an unplanned failover loses seconds of data.
DynamoDB global tables (99.999% SLA) in default MREC mode have an RPO of a few seconds. MRSC gets RPO 0 but needs exactly three Regions and drops TTL, local secondary indexes and transactions.
AWS FIS runs the experiments, and its scenario library includes AZ Availability: Power Interruption and Cross-Region: Connectivity.

For experiments inside Kubernetes, LitmusChaos (CNCF Incubating) fits better, and Sloth generates Prometheus SLO rules from a short spec. For FIS across many accounts, see my Conquering Cloud Chaos (2024).

Decide: how far to go, and what it costs elsewhere

Start with two numbers per journey. RTO is the longest acceptable gap before service comes back. RPO is how much data, measured in time, you can afford to lose. Write both down before anyone says "multi-Region", then pick the cheapest of AWS's four strategies that meets them (Well-Architected REL13-BP02).

Strategy	RPO / RTO (per AWS)	What runs in the recovery Region	Choose it when
Backup and restore	Hours / 24 h or less	Backups only	Internal tools, or data you can rebuild
Pilot light	Minutes / tens of minutes	Data replicas, compute switched off	Data must be current, but an hour down is survivable
Warm standby	Seconds / minutes	A scaled-down full stack	Customer-facing journeys
Multi-site active-active	Near zero / potentially zero	Full stack taking live traffic	Revenue stops when you stop, and the team can run it

My rule of thumb is multi-AZ in one Region for SLOs up to about 99.95%, and multi-Region at 99.99% or when a regulator wants a separate site. Bad writes replicate too, so even active-active needs point-in-time backups you've actually restored.

Every extra nine gets paid for somewhere else in this series.

Other NFR	What more availability does to it
Security	More Regions mean more IAM, keys and secrets to keep in sync
Compliance	Replicas and cross-Region inference move where data lives and is processed
Accessibility	Fallback and "degraded" pages are often the least-tested UI you ship
Performance	MRSC and synchronous replication add write latency
Scalability	Static stability pre-buys capacity instead of scaling on demand
Maintainability	Each Region is another deploy target, and drift is what breaks failovers
Cost	Warm standby and active-active mean paying for a second data tier that mostly waits
Sustainability	Idle standby capacity draws power every hour it waits

Hands-on: data-plane failover, then break it on purpose

Harbor Works serves from eu-west-1 with a warm standby in eu-central-1, both behind existing ALBs. Everything exists ahead of time, so failover makes no control-plane calls.

Checked against the Route 53 HealthCheckConfig API reference and the Terraform AWS provider docs on Oct 29, 2026. terraform validate passes on Terraform 1.17.1 with AWS provider 6.71.0. Not deployed, so test it in a sandbox account first.

terraform {
  required_providers {
    aws = { source = "hashicorp/aws", version = "~> 6.0" }
  }
}

variable "zone_name" { default = "example.com" }
variable "record_name" { default = "api.example.com" }
variable "primary_alb" { default = "api-euw1" } # existing ALB names
variable "secondary_alb" { default = "api-euc1" }

provider "aws" {
  alias  = "primary"
  region = "eu-west-1"
}
provider "aws" {
  alias  = "secondary"
  region = "eu-central-1"
}

data "aws_route53_zone" "this" {
  provider = aws.primary
  name     = var.zone_name
}
data "aws_lb" "primary" {
  provider = aws.primary
  name     = var.primary_alb
}
data "aws_lb" "secondary" {
  provider = aws.secondary
  name     = var.secondary_alb
}

# Route 53 checkers probe this from several locations. No API call happens at failover time.
resource "aws_route53_health_check" "primary" {
  provider          = aws.primary
  fqdn              = data.aws_lb.primary.dns_name
  type              = "HTTPS"
  port              = 443
  resource_path     = "/healthz"                              # shallow: this Region's own deps only
  request_interval  = 30                                      # 10 is faster, costs extra, can't change later
  failure_threshold = 3                                       # about 90 s to mark unhealthy
  regions           = ["eu-west-1", "us-east-1", "us-west-2"] # API minimum is 3
  tags              = { Name = "api-primary-euw1" }
}

resource "aws_route53_record" "primary" {
  provider        = aws.primary
  zone_id         = data.aws_route53_zone.this.zone_id
  name            = var.record_name
  type            = "A"
  set_identifier  = "primary-euw1"
  health_check_id = aws_route53_health_check.primary.id
  failover_routing_policy { type = "PRIMARY" }
  alias {
    name                   = data.aws_lb.primary.dns_name
    zone_id                = data.aws_lb.primary.zone_id
    evaluate_target_health = true # also fail over if the ALB has no healthy targets
  }
}

resource "aws_route53_record" "secondary" {
  provider       = aws.primary
  zone_id        = data.aws_route53_zone.this.zone_id
  name           = var.record_name
  type           = "A"
  set_identifier = "secondary-euc1"
  failover_routing_policy { type = "SECONDARY" }
  alias {
    name                   = data.aws_lb.secondary.dns_name
    zone_id                = data.aws_lb.secondary.zone_id
    evaluate_target_health = true
  }
}

One Route 53 behavior surprises people: if both records are unhealthy, it returns the primary anyway.

sequenceDiagram
  participant HC as Health checkers
  participant P as Primary ALB
  participant DNS as Route 53 DNS
  participant C as Client
  HC->>P: GET /healthz every 30 s
  P-->>HC: 3 consecutive failures
  HC->>DNS: primary marked unhealthy
  C->>DNS: resolve api.example.com
  DNS-->>C: secondary ALB, TTL 60 s
  Note over C,DNS: Clients with a cached answer keep hitting the primary until the TTL expires

Budget about 90 seconds to detect plus 60 seconds of cached DNS, and put that in your RTO.

Now rehearse October in miniature. This experiment cuts the app subnets off from the Regional DynamoDB endpoint for 10 minutes, and stops early if the SLO fast-burn alarm fires.

Checked against the AWS FIS actions reference on Oct 29, 2026, and validated against the CreateExperimentTemplate model in boto3 1.43.121. Not deployed, so test it in a sandbox account first.

{
  "description": "Game day: app subnets lose the Regional DynamoDB endpoint for 10 minutes",
  "targets": {
    "appSubnets": {
      "resourceType": "aws:ec2:subnet",
      "resourceTags": { "tier": "app" },
      "selectionMode": "ALL"
    }
  },
  "actions": {
    "denyDynamoDB": {
      "actionId": "aws:network:disrupt-connectivity",
      "parameters": { "scope": "dynamodb", "duration": "PT10M" },
      "targets": { "Subnets": "appSubnets" }
    }
  },
  "stopConditions": [
    {
      "source": "aws:cloudwatch:alarm",
      "value": "arn:aws:cloudwatch:eu-west-1:111122223333:alarm:orders-slo-fast-burn"
    }
  ],
  "roleArn": "arn:aws:iam::111122223333:role/fis-experiment-role",
  "tags": { "Name": "dynamodb-endpoint-loss" }
}

Attach the AWSFaultInjectionSimulatorNetworkAccess managed policy to the role, then run aws fis create-experiment-template --cli-input-json file://dynamodb-endpoint-loss.json. A passing run is boring. /healthz stays green and users get a degraded answer, not an error.

Availability in the age of AI

AI as a workload

A model is a dependency with an SLA, a quota, a lifecycle, and one failure mode your load balancer can't see: a confident wrong answer with HTTP 200.

Throttling arrives long before Region outages do. Bedrock's ThrottlingException (429) means you hit your account quota, which an increase fixes. Its 503 means the service is short of capacity, "not related to your account-level quotas". Back off with jitter and retry in one layer only, since three layers of three attempts can hit a struggling dependency 27 times. Keep maxTokens near your real output size, because Bedrock reserves it against your tokens-per-minute quota at request start. Then add one circuit breaker per model.

Put the model in your math. The Bedrock SLA promises 99.9% for the service, but the Reserved tier "targets 99.5% uptime for model response" (AWS, November 2025). Plan on the lower, model-level number. In series with Harbor Works' 99.89% stack, the ceiling drops to 99.39%, about 4.4 hours a month, and no Lambda tuning gets you back above it.

Cross-Region inference trades residency for capacity. A geographic profile such as eu. can process your prompt in any Region in that geography, and a global. profile in any supported commercial Region, for about 10% less. Your SCPs must allow every destination, and CloudTrail records where each request ran in additionalEventData.inferenceRegion. Just don't call it DR. AWS says it "does not, however, function as a disaster recovery or automatic failover system" (AWS Machine Learning Blog, February 2026).

Build a degradation ladder, where each rung depends on less than the one above.

flowchart TB
  Q["User request"] --> R1{"Rung 1<br/>primary model"}
  R1 -->|"ok"| A["Full answer"]
  R1 -->|"429, 503, timeout or breaker open"| R2{"Rung 2<br/>smaller model, other provider"}
  R2 -->|"ok"| B["Shorter answer"]
  R2 -->|"fails"| R3{"Rung 3<br/>cached answer"}
  R3 -->|"hit"| C["Last good answer"]
  R3 -->|"miss"| R4["Rung 4<br/>non-AI path"]

Rungs 1 and 2 both depend on Bedrock, so only rungs 3 and 4 survive a Bedrock-wide problem.

So the parallel math won't rescue you. Two independent 99.5% models would give 99.9975%, but these share an endpoint, a Region and an account. Rarely used fallbacks also tend to fail when needed (Jacob Gabrielson, Amazon Builders' Library, 2019), so send a trickle of live traffic down rung 2 and test rung 4 synthetically.

Checked against the Bedrock Converse API reference and the Claude Sonnet 4.6 and Nova 2 Lite model cards on Oct 29, 2026. Unit-tested with botocore's Stubber on boto3 1.43.121 for the throttle, 503, model-timeout and validation-error paths. Not deployed, so test it in a sandbox account first.

import time
import boto3
from botocore.config import Config
from botocore.exceptions import ClientError, ConnectTimeoutError, ReadTimeoutError

# Standard mode: exponential backoff with jitter, plus a retry quota.
# Keep attempts low. The ladder below is the real fallback.
cfg = Config(retries={"mode": "standard", "max_attempts": 2},
             connect_timeout=2, read_timeout=20)
brt = boto3.client("bedrock-runtime", region_name="eu-west-1", config=cfg)

# Pinned IDs. The eu. geographic profiles keep processing inside the EU.
RUNGS = [
    "eu.anthropic.claude-sonnet-4-6",  # rung 1: primary model
    "eu.amazon.nova-2-lite-v1:0",      # rung 2: smaller model, different provider
]
RETRYABLE = {"ThrottlingException", "ServiceUnavailableException",
             "ModelNotReadyException", "ModelTimeoutException",
             "InternalServerException"}


class Breaker:
    """Opens after N straight failures, lets a trial call through after cooldown."""
    def __init__(self, threshold=5, cooldown=30):
        self.threshold, self.cooldown = threshold, cooldown
        self.failures, self.opened_at = 0, None

    def allow(self):
        if self.opened_at is None:
            return True
        return time.monotonic() - self.opened_at >= self.cooldown  # half-open

    def record(self, ok):
        if ok:
            self.failures, self.opened_at = 0, None
        else:
            self.failures += 1
            if self.failures >= self.threshold:
                self.opened_at = time.monotonic()


breakers = {m: Breaker() for m in RUNGS}
cache = {}  # stand-in: use DynamoDB or ElastiCache, keyed by a normalized prompt


def answer(prompt: str) -> dict:
    for model in RUNGS:
        if not breakers[model].allow():
            continue                                 # breaker open: skip this rung fast
        try:
            r = brt.converse(
                modelId=model,
                messages=[{"role": "user", "content": [{"text": prompt}]}],
                inferenceConfig={"maxTokens": 512},  # reserved against TPM at request start
            )
            breakers[model].record(True)
            text = r["output"]["message"]["content"][0]["text"]
            cache[prompt] = text
            return {"rung": model, "text": text}
        except ClientError as e:
            if e.response["Error"]["Code"] not in RETRYABLE:
                raise                                # a 400 or 403 is a bug, not an outage
            breakers[model].record(False)
        except (ConnectTimeoutError, ReadTimeoutError):
            breakers[model].record(False)
    if prompt in cache:                              # rung 3: last good answer
        return {"rung": "cache", "text": cache[prompt]}
    return {"rung": "non-ai",                        # rung 4: the product still works
            "text": "Line summaries are busy. Here is the shift log for this line."}

Pin model versions and watch the calendar. Bedrock models go Active → Legacy → End-of-Life. After EOL "requests made to it will fail", and "migration will not happen automatically" (Bedrock model lifecycle). Legacy periods run six months, sometimes only 45 days. Claude Sonnet 4.5, for example, is Legacy with EOL on April 8, 2027. Pin exact IDs, pull modelLifecycle.endOfLifeTime from GetFoundationModel into an inventory, and alarm 90 days out.

Run evals as synthetic health checks. A model can get worse without a single 5xx. Run a small golden set every few minutes, publish the pass rate as a CloudWatch metric, and alarm on it. A Route 53 CLOUDWATCH_METRIC health check can then fail over on quality (same-account, standard-resolution alarms only).

Give agents durable state. An agent that forgets everything at step 31 of 40 is down. Step Functions Standard workflows and Lambda durable functions both checkpoint progress and run for up to a year. AgentCore Runtime microVM sessions default to an 8-hour lifetime and a 15-minute idle timeout, so keep state outside them, and make tool calls idempotent so a replay can't order twice.

Vector stores need an RTO and RPO too. Either replicate the index, or replicate the sources and re-embed, which is cheap to keep ready but slow to run. OpenSearch Service offers active-passive cross-cluster replication. DynamoDB vector search (GA August 2026) replicates the index definition with global tables, but index contents propagate asynchronously, even under MRSC. The S3 Vectors guide documents no replication or backup as of October 2026, so replicate the source documents and count re-embedding in your RTO. Only replicate into Regions your residency rules allow.

Still immature: FIS has no Bedrock fault action, so simulate 429s in your client or a gateway. For provider-neutral fallback, the open-source LiteLLM proxy handles retries, cooldowns and cross-provider failover.

AI as an engineering aid

AI tools are good at the tedious parts, like drafting a dependency map from your IaC or turning a runbook into an FIS template. AWS DevOps Agent (GA March 2026) correlates telemetry, code and deployments during incidents, though its "hours to minutes" MTTR claim comes with no published measurement.

They also make things up, like a Bedrock fault action that doesn't exist, or two copies in one AZ treated as independent. Validate generated templates against the API model (botocore's ParamValidator checked the FIS template above), refuse experiments without a stop condition, and keep a human approval on any production failover an agent proposes.

Who: owners and the shared responsibility split

Activity	Product owner	Architect	Developers	SRE / platform
SLOs and error budget policy	A	R	C	R
Dependency map and DR strategy	C	A, R	C	C
Timeouts, retries, idempotency, degradation ladder	I	C	A, R	C
Experiments, drills, burn-rate alarms	I	C	C	A, R

R = responsible, A = accountable, C = consulted, I = informed.

AWS owns "resiliency of the cloud" and you own "resiliency in the cloud". Your share depends on the compute.

EC2. You place instances across AZs, keep spare capacity and run the self-healing.
Containers. You spread tasks or pods across AZ subnets and own health checks. Fargate takes the hosts off your hands.
Lambda. AWS runs functions across AZs. For VPC-attached functions, picking multi-AZ subnets is on you.
Managed AI. The provider sets the lifecycle and AWS serves the model. Quotas, profiles, the ladder, EOL tracking and evals are yours.

Monday checklist

[ ] Write one SLO for your most important journey and get the error budget policy signed.
[ ] Multiply your hard dependencies. If the product is below your SLO, that gap is your backlog.
[ ] Replace every recovery step that calls a control plane with pre-provisioned capacity or a health-check switch.
[ ] Strip shared dependencies out of /healthz.
[ ] Write down RTO and RPO per journey, then pick the DR strategy from the table.
[ ] Run the DynamoDB endpoint-loss experiment in a sandbox account.
[ ] Cut your busiest path down to one retry layer.
[ ] List every production model ID with its EOL date.
[ ] Decide, and test, what your AI feature shows when no model answers.

Conclusion

Availability is arithmetic before it's architecture. Multiply your dependencies to find your ceiling. Ask whether your copies fail independently to learn whether your redundancy is real. Ask whether recovery needs a control plane to learn whether it'll work on the day.

AI doesn't change those rules. It adds a dependency with a quota, a lifecycle and a failure mode that returns HTTP 200. Put it in the math, give it a ladder, and make sure the last rung needs no model.

Nines are arithmetic. Recovery is rehearsal.

Series: Non-functional Application Requirements

Non-functional Application Requirements: An Introduction
Non-functional Application Requirements: Security
Non-functional Application Requirements: Compliance
Accessibility
Performance
Availability
Scalability (coming soon)
Maintainability (coming soon)
Cost (coming soon)
Sustainability (coming soon)

Cover image: AWS re:Invent 2023 – Keynote with Dr. Werner Vogels

Next up, Scalability. A cost-benefit look at how you grow, where tokens per minute become a capacity unit as real as vCPUs.

Sources

Non-functional Application Requirements: Accessibility

Oluwasegun Adedigba — Wed, 11 Nov 2026 14:55:36 +0000

On 4 June 2026, a court in Caen, France, ordered Carrefour to make its website and app fully accessible to disabled customers within six months, with a penalty for every day it runs late. Two disability associations, apiDV and Droit Pluriel, brought the case under the French law that carries the European Accessibility Act.

Carrefour's defense was a number. It said the site already met 71% of the accessibility criteria. The judge said it had to be totally accessible (Deque, June 2026).

Seventy-one percent sounds like a pass. For a blind shopper who can't get through checkout, it's a locked door.

This part takes six popular myths about accessibility, puts each one against the evidence, and then builds the CI gate that stops you shipping your own 71%.

TL;DR

Aim for WCAG 2.2 Level AA. Meeting 2.2 also meets 2.0 and 2.1, which most laws still name.

Six boring error types made up 96% of what WebAIM found in 2026. Native HTML prevents most of them for free.

Gate CI on axe-core, add role-based Playwright tests, and keep a human screen reader pass, because a clickable div sails through axe.

The law is live: the EAA since 28 June 2025, the US ADA Title II rule from 26 April 2027.

Streaming LLM answers should be announced in sentences, not tokens. The same markup serves screen readers and AI browser agents.

What: accessibility, defined precisely

Accessibility means people with disabilities can perceive, understand, operate and navigate your product, with or without assistive technology like screen readers, magnifiers or voice control. Don't confuse it with conformance, which is passing a standard's testable criteria. Conformance is the measurable proxy. Accessibility is the outcome, and you can have the first without the second.

The measurable form is the Web Content Accessibility Guidelines (WCAG). Version 2.2 became a W3C Recommendation in 2023 and the international standard ISO/IEC 40500:2025 in October 2025. Its criteria come in levels A, AA and AAA, and when a law says "accessible", it almost always means AA.

Scope: web content, and in most legal readings apps and documents too.

What 2.2 adds at A and AA:

2.4.11 Focus Not Obscured (AA): sticky headers can't completely hide the focused element.
2.5.7 Dragging Movements (AA): anything you drag also works with a single tap.
2.5.8 Target Size (AA): pointer targets of at least 24 by 24 CSS pixels, or enough spacing.
3.2.6 Consistent Help (A): help links stay in the same place.
3.3.7 Redundant Entry (A): don't make people retype what they gave you.
3.3.8 Accessible Authentication (AA): no memory test or puzzle to sign in. Paste, password managers and passkeys satisfy it.

A testable requirement reads: "Every route passes automated WCAG 2.2 AA checks on every pull request, sign-in and checkout complete with a keyboard and a screen reader, and no pointer target is under 24 by 24 CSS pixels."

Why: who you're leaving out, and who's watching

About 1.3 billion people, 16% of the world's population, live with a significant disability (WHO, 2023). Add the temporary and situational cases: a broken wrist, glare on a phone, a supervisor on a loud factory floor who can't hear an alert.

The web mostly fails them. WebAIM's 2026 scan of the top million home pages found detectable WCAG failures on 95.9% of them, up from 94.8% and reversing six years of slow improvement. The average page had 56.1 errors (WebAIM Million, 2026).

Then there's the law, which is where accessibility starts to look like Part 3 on Compliance: a regulation, a standard it points to, and evidence you produce on demand. Myth 5 has the dates.

Six myths, checked against the evidence

Myth 1: "Accessibility is for a small group of users"

One in six people isn't a niche, and it undercounts your users. Captions get used in noisy rooms. A visible focus ring helps anyone fighting a bad trackpad.

There's a newer group of users, too. Screen readers, voice control, your Playwright tests and AI browser agents all read the same thing: the accessibility tree the browser builds from your markup. OpenAI's developer FAQ says "making your website more accessible helps ChatGPT Agent in Atlas understand it better". Microsoft's Playwright MCP server drives a browser for an LLM from that tree, not from screenshots.

flowchart LR
  M["Your HTML<br/>and ARIA"] --> T["Browser accessibility tree<br/>roles, names, states"]
  T --> SR["Screen readers"]
  T --> VC["Voice control"]
  T --> PT["Playwright<br/>getByRole tests"]
  T --> AG["AI browser agents"]

Every one of these reads the same tree, so one missing button name breaks all of them at once.

Myth 2: "It's expensive and slows us down"

Here are WebAIM's most common failures in 2026, by share of home pages: low contrast text (83.9%), missing alt text (53.1%), missing form labels (51%), empty links (46.3%), empty buttons (30.6%) and no page language (13.5%). Those six made up 96% of every error detected.

Each one is a line of code. A checked color token, an alt attribute, a <label for>, text in a link, a name on an icon button, lang="en". None of that slows a sprint, and native <button> and <a href> bring keyboard support and the right role for free.

The real cost lives in two places: retrofitting, which Carrefour now does on a judge's clock, and custom widgets like comboboxes and data grids. For those I'd use a tested library such as React Aria, which ships over 40 unstyled components tested with screen readers, rather than build my own. You still test what you build with it.

Myth 3: "A scanner, or an overlay, makes us compliant"

Automated tools are necessary and nowhere near enough. In 2017 the UK government built a page with 143 deliberate barriers and ran ten free checkers on it. Together they found 71%, and 42 barriers got past every tool (GOV.UK, 2017). Deque, which makes axe, puts its own coverage at 57% of issues by volume (2021), a generous way of counting that still leaves four in ten for people.

Here's what that looks like. A sign-in page with a <div onclick> reading "Continue" passed axe-core 4.14 with zero violations. A Playwright test looking for a button named "Continue" failed immediately.

flowchart LR
  D["div onclick<br/>with text Continue"] --> AX["axe-core scan"]
  AX -->|"no rule fails"| G["Green build"]
  D --> SR["Screen reader<br/>and keyboard"]
  SR -->|"read as plain text<br/>skipped by Tab"| X["User stuck<br/>at sign-in"]
  D --> RT["getByRole button test"]
  RT -->|"button not found"| R["Red build"]

The same clickable div passes the scanner and fails both the user and a role-based test.

Overlays, the one-script-tag fixes, fail the same way at a higher price. In April 2025 the US Federal Trade Commission finalized a $1 million order against accessiBe, saying its AI-powered widget did not make all customer sites WCAG-compliant, and barred it from claiming otherwise without evidence (FTC, 2025).

Myth 4: "More ARIA makes it more accessible"

In WebAIM's 2026 data, pages using ARIA averaged 59.1 errors against 42 without. WebAIM calls that a correlation, since ARIA-heavy pages are also more complex. It still matches the W3C's first rule of ARIA: if a native element already does what you need, use it.

ARIA changes what assistive technology is told. It doesn't add behavior. role="button" on a div won't make it focusable or make Enter work.

Agents add a twist. That OpenAI FAQ talks about "ARIA tags", and accessibility consultant Adrian Roselli replied that native HTML is the right advice and that ARIA labels will get keyword-stuffed like alt text was. Keep ARIA for what HTML can't say, like aria-expanded or aria-describedby wired up with React's useId. And never write an aria-label for a bot. WCAG 2.5.3 Label in Name expects the accessible name to contain the visible label, so voice control users can say what they see.

Myth 5: "We're not in the EU or government, so the law can't reach us"

The European Accessibility Act has applied since 28 June 2025 (EUR-Lex). It covers products and services for EU consumers, including e-commerce, banking, e-books, transport and telephony. Only microenterprises providing services (under 10 staff and €2 million turnover) are exempt, and each member state sets its own penalties. The standard behind it, EN 301 549, published version 4.1.1 with WCAG 2.2 AA in September 2026. At the time of writing it isn't yet cited in the EU's Official Journal, so version 3.2.1 is still the legal reference.

In the US, the ADA Title II rule requires WCAG 2.1 AA for state and local government web content and apps, including what contractors supply. An April 2026 rule moved the dates to 26 April 2027 for entities serving 50,000 people or more and 26 April 2028 for the rest (Federal Register, 2026). The Justice Department says it may revisit the substance, so check before you plan around it.

Enforcement is real but uneven. A month before Carrefour, a Lille court dismissed a similar case against Auchan's online arm on a revenue threshold, and that's under appeal. If you sell into Europe, or a city council buys your software, the law already reaches you.

When: where accessibility enters the lifecycle

Design: check contrast and focus styles in your tokens, keep targets at 24 by 24 CSS pixels, and annotate headings and button names in mockups. Developers can't guess what an icon was meant to say.
Build: native elements first, a tested library for complex widgets, eslint-plugin-jsx-a11y in the editor.
Test: axe-core plus role and keyboard tests on every pull request, then a screen reader pass (NVDA with Chrome, VoiceOver with Safari) on key journeys before each release.
Operate: publish an accessibility statement with a working feedback channel, treat accessibility bugs as defects, and re-check production journeys on a schedule.

Where: AWS building blocks and the open-source tools around them

AWS doesn't sell an accessibility service. It gives you pieces that make particular criteria easier, and places to run checks.

AWS Amplify UI: accessible React components, including the sign-in Authenticator. Useful on Amplify, unnecessary if you have a design system. Your theme can still break contrast.
Amazon Cognito managed login: passkeys and one-time codes from the Essentials tier up, which gives you a sign-in that meets 3.3.8. Test your branding's contrast.
Amazon CloudWatch Synthetics: scheduled Playwright canaries (syn-nodejs-playwright-8.0, Playwright 1.61.1) to replay journey tests against production. Its Playwright trails the current release.
Amazon Transcribe: live captions, with partial-result stabilization so words don't flicker. Recorded media still needs human-edited captions.
Amazon Polly: read-aloud at $4 to $100 per million characters by engine. Word-highlighting speech marks don't work with generative voices.

The core tools are open source. axe-core powers most scanners, Lighthouse included, so Lighthouse isn't a second opinion. pa11y-ci checks a URL list with no test code. And a free screen reader like NVDA is the only tool that tells you what a user will actually hear.

Decide: which check for which problem

Check	Catches	Misses	Cost
eslint-plugin-jsx-a11y	Click handlers on `div`s, missing `alt`, invalid ARIA	Anything that depends on CSS or runtime state	Free, seconds
axe-core via Playwright	Contrast, names, labels, `lang`, invalid ARIA, target size with the 2.2 tag	Fake controls, focus order, alt text quality	Free, seconds per route
Role and keyboard tests	Fake controls, broken Tab order, mouse-only journeys	Visual issues, meaning	Free, you write them
CloudWatch Synthetics canary	Regressions in live journeys	Whatever the script doesn't test	Billed per run
Manual screen reader pass	Announcements, reading order, confusing flows	Coverage at scale	People's time
Testing with disabled users	Whether it actually works for them	Nothing a cheaper check finds better	Recruiting and paying people

When you're not sure which check a requirement needs, walk it down this tree.

flowchart TB
  Q["A WCAG requirement to verify"] --> Q1{"Visible in the DOM alone?<br/>contrast, names, labels, lang"}
  Q1 -->|"yes"| AX["axe-core in CI"]
  Q1 -->|"no"| Q2{"Depends on behavior?<br/>focus, keyboard, announcements"}
  Q2 -->|"yes"| PW["Playwright role<br/>and keyboard test"]
  Q2 -->|"no"| Q3{"Depends on meaning?<br/>alt text, captions, plain language"}
  Q3 -->|"yes"| HR["Human review<br/>AI may draft"]
  Q3 -->|"no"| AU
  AX --> AU["Before any conformance claim<br/>manual audit and user testing"]
  PW --> AU
  HR --> AU

Pick the cheapest check that can see the problem, and never let a passing tool stand in for the audit.

Accessibility also pulls with and against the rest of this series.

NFR	Pulls the same way	Pulls against it
Security	Passkeys and password-manager paste meet 3.3.8 and cut phishing	CAPTCHAs, and timeouts without the warning and 20-second extension WCAG 2.2.1 asks for
Compliance	The same evidence trail as Part 3	Laws name different versions: 2.1 for ADA Title II, 2.2 in EN 301 549 v4.1.1
Performance	Native HTML ships less JavaScript than custom widgets or overlays	Throttled announcements arrive after the text does
Availability	Announced error messages make degraded modes usable	An accessible fallback is one more path to test
Scalability	Text and voice share one backend	Nova 2 Sonic caps at 20 concurrent sessions per account per Region
Maintainability	Fewer custom widgets, less keyboard code	Hand-built ARIA widgets need their own tests forever
Cost	A lint error takes minutes to fix	Manual testing and user research are real budget lines
Sustainability	Lean, semantic pages	Speech output adds compute per interaction

Hands-on: a WCAG 2.2 AA gate in CI

Take Harbor Works, the fictional plant in Luso, the open-source AI watchman for production lines I'm building alongside this series. Its web app has a status page, a sign-in page and an alerts page, and a shift supervisor has to be able to use each one with a keyboard and a screen reader.

flowchart LR
  PR["Pull request"] --> L["ESLint jsx-a11y<br/>markup rules"]
  L --> B["Build the app"]
  B --> S["Playwright serves<br/>the production build"]
  S --> A["axe-core scan<br/>every route, 2.2 AA tags"]
  A --> J["Role and keyboard<br/>journey test"]
  J -->|"all pass"| M["Merge"]
  L -->|"fail"| F["Build fails with<br/>rule ids and selectors"]
  A -->|"fail"| F
  J -->|"fail"| F
  M --> H["Manual screen reader<br/>pass before release"]

Each stage catches what the one before it can't, and a person still checks key journeys before release.

For the lint step, add jsxA11y.flatConfigs.recommended from eslint-plugin-jsx-a11y 6.10.2 to your eslint.config.js. The Playwright config below assumes a Vite app with npm run preview. Set BASE_URL to test a preview deployment instead.

Checked against Playwright accessibility testing and Playwright reporters on Oct 29, 2026. Ran locally with Node 22, @playwright/test 1.64.1, @axe-core/playwright 4.14.0 and Chromium against static test pages. Not deployed — test in a sandbox account first.

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  forbidOnly: !!process.env.CI,
  reporter: [
    ['list'],
    ['html', { open: 'never' }],                              // full report as a CI artifact
    ['junit', { outputFile: 'reports/a11y-junit.xml' }],      // CodeBuild test reports read this
  ],
  use: { baseURL: process.env.BASE_URL ?? 'http://localhost:4173' },
  projects: [{ name: 'chromium', use: { ...devices['Desktop Chrome'] } }],
  // Serve the production build unless BASE_URL points at a preview deployment
  webServer: process.env.BASE_URL ? undefined : {
    command: process.env.A11Y_SERVE ?? 'npm run preview -- --port 4173 --strictPort',
    url: 'http://localhost:4173',
    reuseExistingServer: !process.env.CI,
  },
});

The test scans every route with axe, then walks sign-in the way a screen reader user would: by label, role and keyboard.

Checked against Playwright accessibility testing and axe-core API tags on Oct 29, 2026. Ran locally as above. Not deployed — test in a sandbox account first.

// tests/a11y.spec.ts
import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';

// axe-core tags for WCAG 2.0, 2.1 and 2.2 at levels A and AA.
// Drop 'wcag22aa' and the WCAG 2.2 target-size rule never runs.
const WCAG_22_AA = ['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa', 'wcag22aa'];
const ROUTES = (process.env.A11Y_ROUTES ?? '/,/sign-in/,/alerts/').split(',');

for (const route of ROUTES) {
  test(`${route} has no detectable WCAG 2.2 AA violations`, async ({ page }, testInfo) => {
    await page.goto(route);
    await page.getByRole('main').waitFor();            // also fails pages with no main landmark

    const results = await new AxeBuilder({ page })
      .withTags(WCAG_22_AA)
      .exclude('[data-a11y-ignore]')                   // third-party embeds you track as known issues
      .analyze();

    // Full JSON in the report: selectors, failure summaries and help URLs for whoever fixes it
    await testInfo.attach('axe-violations.json', {
      body: JSON.stringify(results.violations, null, 2),
      contentType: 'application/json',
    });

    const summary = results.violations.map(
      (v) => `${v.id} [${v.impact}] x${v.nodes.length}: ${v.help}`,
    );
    expect(summary).toEqual([]);
  });
}

// axe can't tell a clickable <div> from plain text. Role and label queries can:
// if this fails, screen readers, voice control and browser agents can't find the control either.
test('sign-in works by label, role and keyboard alone', async ({ page }) => {
  await page.goto(process.env.A11Y_SIGN_IN ?? '/sign-in/');
  await page.getByLabel('Email').fill('supervisor@harbor.example');
  await page.keyboard.press('Tab');
  const submit = page.getByRole('button', { name: 'Continue' });
  await expect(submit).toBeFocused();                 // reachable, in order, by keyboard
  await page.keyboard.press('Enter');
  await expect(page.getByRole('heading', { level: 1, name: 'Alerts' })).toBeVisible();
});

Running it showed three things. A broken test page failed with four named rules (html-has-lang, image-alt, link-name, color-contrast). The clickable-div page passed the scan and failed the journey test. And axe-core 4.14 only checks target size when you ask for the wcag22aa tag. A default run let 10-pixel buttons through. Keep exclude rare, since it switches off every rule inside that element.

On GitHub Actions:

Checked against Playwright CI docs on Oct 29, 2026. YAML validated, not run on GitHub. Not deployed — test in a sandbox account first.

# .github/workflows/a11y.yml
name: a11y
on:
  pull_request:
  push:
    branches: [main]

jobs:
  a11y:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-node@v6
        with:
          node-version: 22
          cache: npm
      - run: npm ci
      - run: npx eslint .                       # jsx-a11y: div buttons, missing alt, before a browser starts
      - run: npm run build
      - run: npx playwright install --with-deps chromium
      - run: npx playwright test                # GitHub sets CI=true, so test.only fails the build
      - uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}                 # upload the report when tests fail, too
        with:
          name: a11y-report
          path: playwright-report/
          retention-days: 30

If your pipeline runs in AWS, here's the same gate for CodeBuild. Playwright supports only Debian and Ubuntu, so use the Ubuntu standard image. CodeBuild doesn't document setting CI, so the buildspec sets it. The JUnit file feeds CodeBuild test reports, so each failing page shows up as a failed test case.

Checked against the CodeBuild buildspec reference and CodeBuild available runtimes on Oct 29, 2026. YAML validated, not run in CodeBuild. Not deployed — test in a sandbox account first.

# buildspec.yml
# CodeBuild image: aws/codebuild/standard:7.0 (Ubuntu 22.04). Playwright supports Debian and Ubuntu only.
# Pinned alternative image: mcr.microsoft.com/playwright:v1.64.1-noble
version: 0.2

env:
  variables:
    CI: "true"                 # the Playwright config reads it

phases:
  install:
    runtime-versions:
      nodejs: 22
    commands:
      - npm ci
      - npx playwright install --with-deps chromium
  pre_build:
    commands:
      - npx eslint .
      - npm run build
  build:
    commands:
      - npx playwright test

reports:
  a11y-tests:                  # becomes report group <project-name>-a11y-tests
    files:
      - a11y-junit.xml
    base-directory: reports
    file-format: JUNITXML

artifacts:
  name: a11y-report
  base-directory: playwright-report
  files:
    - '**/*'

That's the automated slice, about half the problem by the most generous count. The manual pass in the diagram isn't optional.

Accessibility in the age of AI

Myth 6: "AI makes accessibility free"

The Justice Department answered this one. Delaying the ADA dates, it wrote that "advanced technology, such as generative AI, does not yet reliably automate the remediation of inaccessible content at scale". That's a regulator that wanted the shortcut to exist.

Research agrees. A 2025 study comparing web code from GPT-4o and Qwen2.5-Coder with human code found the models often better on basics like contrast and alt text, and weak on ARIA. Better prompts helped a little. Feeding test results back to the model helped a lot. WebAIM even suggests AI-assisted "vibe coding" may be part of why errors rose in 2026, though it didn't measure that.

Alt text shows why "free" is the wrong word. A model sees pixels, but alt text depends on purpose: the same photo needs empty alt as decoration and a description when it informs. Screen reader users can already get generic descriptions themselves (60.1% in WebAIM's 2026 screen reader survey use AI to describe images). What only you can add is why the image is there. Let AI draft alt text and captions, and have a person approve them.

AI as a workload: streaming answers and live regions

Streaming makes chat feel fast and can make it unbearable with a screen reader. If the answer container is an aria-live region, every token is a change, and users hear fragments, restarts, or a queue that runs long after the text has finished.

The fix is to separate what's shown from what's announced. A Harbor Works supervisor asks why line 3 stopped. The visible answer updates per token. A separate role="status" region, polite by default, only gets whole sentences, at most one every 1.5 seconds, then "Answer complete."

sequenceDiagram
  participant U as Supervisor
  participant UI as Chat UI
  participant B as Bedrock
  participant L as Live region
  participant SR as Screen reader
  U->>UI: Why did line 3 stop
  UI->>B: ConverseStream request
  loop every token
    B-->>UI: contentBlockDelta text
    UI->>UI: Append to visible answer and sentence buffer
    opt sentence ended and 1.5 s since last announcement
      UI->>L: Replace text with the whole sentence
      L->>SR: Polite announcement
    end
  end
  B-->>UI: messageStop
  UI->>L: Remaining text plus Answer complete
  L->>SR: Final polite announcement

Tokens reach the screen as fast as they arrive, but the screen reader only hears whole sentences.

Your backend calls Bedrock's ConverseStream and forwards each contentBlockDelta text, and getTokens reads that stream. Add a standard visually-hidden CSS class.

Checked against MDN ARIA live regions and the Bedrock ConverseStream reference on Oct 29, 2026. Type-checked with React 19 and run in Chromium against a simulated stream, not with a real screen reader or Bedrock. Not deployed — test in a sandbox account first.

import { useEffect, useRef, useState } from 'react';

type Props = {
  // A factory, not a stream: StrictMode runs effects twice, so each run needs its own request
  getTokens: (signal: AbortSignal) => AsyncIterable<string>;
  announce?: 'sentences' | 'complete'; // let users pick, and save it in their settings
};

const MIN_GAP_MS = 1500;               // never speak more often than this
const SENTENCE_END = /[.!?]\s*$|\n$/;

export function StreamingAnswer({ getTokens, announce = 'sentences' }: Props) {
  const [shown, setShown] = useState('');    // what sighted users see: every token
  const [spoken, setSpoken] = useState('');  // what the live region says: whole sentences
  const pending = useRef('');
  const lastSpoke = useRef(0);

  useEffect(() => {
    const ctrl = new AbortController();
    setShown(''); setSpoken(''); pending.current = '';
    (async () => {
      try {
        for await (const t of getTokens(ctrl.signal)) {
          if (ctrl.signal.aborted) return;
          setShown((s) => s + t);
          pending.current += t;
          const now = Date.now();
          if (announce === 'sentences' && SENTENCE_END.test(pending.current)
              && now - lastSpoke.current >= MIN_GAP_MS) {
            setSpoken(pending.current.trim());
            pending.current = '';
            lastSpoke.current = now;
          }
        }
        if (ctrl.signal.aborted) return;
        // Flush the tail, then say it's finished so nobody sits waiting for more
        setSpoken(announce === 'complete'
          ? 'Answer ready.'
          : `${pending.current.trim()} Answer complete.`);
        pending.current = '';
      } catch {
        if (!ctrl.signal.aborted) setSpoken('The answer failed to load. Try again.');
      }
    })();
    return () => ctrl.abort();
  }, [getTokens, announce]);

  return (
    <section aria-labelledby="answer-heading">
      <h2 id="answer-heading">Answer</h2>
      <div>{shown}</div>{/* not live: reading it is the user's choice */}
      {/* In the DOM from first render so screen readers register it before it changes */}
      <div role="status" className="visually-hidden">{spoken}</div>
    </section>
  );
}

In a simulated run, a 48-token answer produced two announcements instead of 48 and passed the same axe scan. Some users would rather read the finished answer at their own pace, so make announce="complete" a user setting.

Voice interfaces

Voice helps users with motor or vision impairments, and anyone whose hands are busy. On AWS you can chain Amazon Transcribe, a text model and Amazon Polly, keeping a transcript at every step. Or use Amazon Nova 2 Sonic (amazon.nova-2-sonic-v1:0, generally available since December 2025), a speech-to-speech model on Bedrock's InvokeModelWithBidirectionalStream API that accepts speech and text in one session, in seven languages.

Know its limits. It runs In-Region only (us-east-1, us-west-2, eu-north-1, ap-northeast-1), connections last eight minutes, and the quota of 20 concurrent sessions per account per Region isn't adjustable. Its model card lists Bedrock Guardrails as unsupported, so content controls live in your prompts and tools. And voice must never be the only way in. A Deaf user needs the same task in text.

AI agents as users

Browser agents read the accessibility tree, so Myths 1 and 4 apply to them as well. A form a screen reader can finish is a form an agent can finish, which makes your getByRole tests a fair proxy for agent-readiness. If an agent can't find a button, the fix is the same native <button> a person needs.

AI as an engineering aid

Assistants happily generate a div with an onClick, and that 2025 study suggests prompting alone won't fix it. Feedback does. So:

Put "native elements, a label on every input, no click handlers on non-interactive elements" in your assistant's project instructions.
Feed it the axe JSON and the failing role test, not "make it accessible".
Have a person review every ARIA attribute it adds, since that's where models were weakest.

What's still immature is AI doing the manual pass. No tool today replaces a person with a screen reader on your checkout.

Who: owners and the shared responsibility split

Activity	Product owner	Designer	Developers	QA / platform
Set the target and legal scope, with legal advice	A	C	C	C
Contrast, focus and target-size tokens	I	R/A	C	I
Semantic components, labels, live regions	I	C	R/A	C
CI gate and production canaries	I	I	R	A
Screen reader pass before release	C	C	R	A
Approve AI-drafted alt text and captions	A	R	R	I

R = responsible, A = accountable, C = consulted, I = informed.

Shared responsibility barely changes with compute type. On EC2, containers or Lambda, AWS owns its console's accessibility and nothing in your app. With Amplify UI, AWS owns component semantics, and you own labels, composition and theme. With Bedrock, the provider owns what the model generates, and you own how it's presented, announced, captioned and reviewed.

Monday checklist

[ ] Write down your target (WCAG 2.2 AA) and which laws reach you: the EAA, ADA Title II through a public-sector customer, or both.
[ ] Add eslint-plugin-jsx-a11y with the recommended config and fix what it finds in shared components.
[ ] Add the axe-core Playwright test with the wcag22aa tag to CI for your three busiest routes.
[ ] Write one role-and-keyboard journey test for sign-in and one for your main task.
[ ] Check design tokens for 4.5:1 text contrast and a visible focus style.
[ ] Spend 15 minutes on checkout or sign-in with NVDA or VoiceOver, and file what you hear.
[ ] If you stream LLM output, announce sentences from a separate role="status" region.
[ ] Put a human approval step on AI-generated alt text and captions.

Conclusion

Every myth here grows from one belief: that accessibility is a feature you add, buy or generate. The evidence says it's mostly a property of ordinary code. Six boring error types cover nearly every failure. A real <button> beats a clever widget. Scanners find part of the problem, overlays and AI drafts don't close the rest, and courts and regulators have stopped accepting "mostly".

Carrefour's 71% measured the wrong thing. Your users don't experience a percentage. They finish the task or they don't.

So build the gate, write the role tests, and put a person with a screen reader on the journeys that matter.

If a screen reader can't find it, neither can your users, your tests or your agents.

Series: Non-functional Application Requirements

Non-functional Application Requirements: An Introduction
Non-functional Application Requirements: Security
Non-functional Application Requirements: Compliance
Accessibility
Performance (coming soon)
Availability (coming soon)
Scalability (coming soon)
Maintainability (coming soon)
Cost (coming soon)
Sustainability (coming soon)

Cover image: AWS re:Invent 2023 – Keynote with Dr. Werner Vogels

Next up, Performance: p95 budgets, caching layers and time-to-first-token, and why a page that's merely less slow still sells more.

Sources

Non-functional Application Requirements: Performance

Oluwasegun Adedigba — Wed, 04 Nov 2026 09:42:35 +0000

In 2021, Vodafone published a test every team arguing about performance should know. Two landing pages looked and worked the same, with paid traffic split 50/50, about 34,000 visits a day each. One version rendered its critical HTML on the server, moved a widget off the client and resized its hero image. Its Largest Contentful Paint was 31% better in field data (5.7 s against 8.3 s).

That page sold 8% more, with 15% more leads and 11% more carts per visit (web.dev).

Here's the part I like: 5.7 seconds is still "poor" by Google's thresholds (over 4). The winning page wasn't fast, just less slow, and people still bought more.

So when someone says performance can wait until after launch, show them this.

TL;DR

Budget on percentiles (p95/p99), and say whether you mean latency or throughput. Averages hide the users you lose.

Cache in layers (edge, application, database), picked by how fresh the data must be.

Lambda cold starts hit under 1% of invocations (AWS), which is exactly your tail.

Load-test with an open model and fail the build when p95 goes over budget.

For LLM features, budget time-to-first-token, output tokens/sec and end-to-end p95 separately. Streaming and prompt caching are your biggest levers.

What performance actually means

Performance is how fast, and how much, work your system does for a user under a stated load. Two numbers make it testable.

Latency is the time one request takes between a defined start and end, so latency at the load balancer and in the browser are different numbers. Throughput is work completed per unit of time (requests, bytes, tokens) at an acceptable latency. Without a latency target it's meaningless, since any queue accepts work if nobody cares when it finishes.

Percentiles describe latency's shape: p50 is the median, p95 the value 95% of requests beat, p99 the slowest 1%. Averages describe no real user: if 98 requests take 50 ms and 2 take 3 seconds, the average is 109 ms and the p99 is 3 seconds.

That high end is tail latency, and it compounds. If one server is slow on 1 request in 100, a request fanning out to 100 such servers is slow 63% of the time (The Tail at Scale). Microservices, RAG pipelines and agent tool calls all fan out.

For frontends, Google's Core Web Vitals are judged at p75 of page loads. LCP (loading) is good at 2.5 s or less and poor over 4.0 s. INP (responsiveness, which replaced FID in 2024) is good at 200 ms or less and poor over 500 ms. CLS (visual stability) is good at 0.1 or less and poor over 0.25.

A testable requirement reads: "Product page LCP ≤ 2.5 s at p75 on mobile. GET /products/{id} p95 ≤ 150 ms and p99 ≤ 400 ms at 200 requests/second."

Why it's worth the effort

It isn't a one-off. A Google-commissioned Deloitte and 55 study of 37 European and US brands and more than 30 million sessions found a 0.1-second improvement in mobile site speed lifted retail conversions by 8.4% and travel conversions by 10.1% (Milliseconds Make Millions, 2020).

Distance matters too. AWS's first CloudFront edge location in Lagos (June 2023) brought up to a 20% average latency improvement for content it served. If your users are in West Africa and your origin is in eu-west-1, physics is part of your budget.

When to think about it

Design: write the budget per user journey and split it across hops. Of the page's 2.5 seconds, how much goes to the API, and how much to the database?
Build: emit latency with a cache hit or miss dimension, and profile before you optimize.
Test: load-test at the target arrival rate before every release, on production-shaped data, with thresholds that fail the pipeline.
Operate: alarm on p95 and p99, not the average, and watch CloudWatch RUM beside synthetic tests.

Where the time goes, and what to use

Most performance mistakes are the right tool on the wrong job. First, the layers.

flowchart LR
    U["User"] -->|"TLS ends at a nearby edge"| CF["CloudFront<br/>static assets and cacheable GETs"]
    CF -->|"miss or dynamic request"| AG["API Gateway"]
    AG --> L["Lambda<br/>connections reused when warm"]
    L -->|"check cache first"| V["ElastiCache for Valkey"]
    V -->|"hit: return"| L
    L -->|"on miss: GetItem"| D["DynamoDB"]
    D --> L
    L -->|"SET with jittered TTL"| V

Each hop to the right adds latency and load, and each cache exists to stop the request going further.

Edge caching with CloudFront

CloudFront's speed comes from its hit ratio, which comes from the cache key. Every header, cookie or query string in it splits the cache. Two cookies with three values each can mean nine origin requests per object.

flowchart LR
    R["Requests for one object"] --> T["Tight cache key<br/>only what the origin uses"]
    R --> W["Cache key with two cookies<br/>three values each"]
    T --> T1["1 cached copy<br/>1 origin request"]
    W --> W1["3 x 3 = 9 cached copies<br/>9 origin requests"]
    T1 --> H["Higher hit ratio"]
    W1 --> M["Lower hit ratio"]

Every value you add to the cache key multiplies the copies CloudFront has to fetch from your origin.

Pick the policy by content:

CachingOptimized (24-hour default TTL) suits versioned static assets. Keep it off changing HTML and API responses, because its 1-second minimum TTL overrides an origin's no-cache.
UseOriginCacheControlHeaders works when your origin sends correct Cache-Control.
CachingDisabled is for personalized or write paths, not public content.
Origin Shield helps with global viewers and expensive origins. It adds a per-request charge, so skip it for dynamic or rarely requested content.

For edge logic, CloudFront Functions do sub-millisecond cache-key and header work, with no network access and 10 KB of code. Lambda@Edge gets network access and up to 30 seconds, but caps at 10,000 requests per second per Region.

Add stale-while-revalidate and stale-if-error to Cache-Control so users get a cached answer while CloudFront refreshes. The same goes for Cloudflare or Fastly. The key matters more than the CDN.

Compute: Lambda cold starts

AWS says cold starts "typically occur in under 1% of invocations" in production (AWS Compute Blog, 2025). That 1% is your p99. Since 1 August 2025 INIT is billed in all configurations, so slow init costs money.

Your four options solve different problems:

Right-size memory first. CPU scales with memory, and 1,769 MB is one vCPU. AWS Lambda Power Tuning (open source) finds the sweet spot, and shorter duration often offsets the higher GB-second price.
SnapStart restores a snapshot of an initialized environment, for Java 11+, Python 3.12+ and .NET 8+ with heavy init. It can't be used with provisioned concurrency, EFS and ephemeral storage over 512 MB. It's free for Java, while Python and .NET pay cache and restore charges.
Provisioned concurrency keeps N environments initialized, for strict latency on spiky but predictable traffic. It bills continuously, used or not.
Lambda Managed Instances run Lambda on EC2 in your account, many invocations per environment, no cold starts. They fit steady, high-volume traffic at the EC2 price plus a 15% management fee, and code must be thread-safe.

flowchart TB
    A["Is p99 dominated by Init Duration?"] -->|"no"| Z["Fix the hot path first:<br/>memory size, client reuse, slow downstreams"]
    A -->|"yes"| B["Java 11+, Python 3.12+ or .NET 8+?"]
    B -->|"yes"| C["Try SnapStart<br/>check EFS and storage limits"]
    B -->|"no"| D["What does traffic look like?"]
    C -->|"still over budget"| D
    D -->|"steady and high volume"| E["Lambda Managed Instances<br/>or Fargate"]
    D -->|"spiky with known peaks"| F["Provisioned concurrency<br/>with scheduled scaling"]
    D -->|"spiky and rare"| G["Accept cold starts<br/>trim package and init code"]

The first question is a measurement, because many "cold start problems" are slow code running during INIT.

On an Application Load Balancer with uneven request cost, set the target group to least_outstanding_requests, since round robin queues behind busy targets. You lose slow start.

Data: DynamoDB vs DAX vs ElastiCache

Part 1 promised this comparison. ElastiCache runs Valkey (the open-source Linux Foundation fork of Redis), Redis OSS or Memcached, serverless or node-based.

	DynamoDB alone	DynamoDB + DAX	ElastiCache for Valkey
Typical read latency	Single-digit milliseconds	Microseconds for cached eventually consistent reads	In-memory. Measure your own p99
Code change	None	Swap in the DAX client	You write the cache logic
Consistency	Strong or eventual, per read	Eventual only. Strongly consistent reads pass through uncached	Whatever you build: TTLs, invalidation, write-through
Staleness trap	None	Query/Scan cache is not invalidated by item writes, and writes made directly to DynamoDB bypass DAX	Forgotten invalidation. Stampedes when hot keys expire together
Choose when	Latency already meets budget	Read-heavy DynamoDB with hit rate above 90%	Multiple data sources, computed results, or semantic caching for AI
Watch out	Hot partitions	VPC-only. Unbounded attribute names exhaust memory	Serverless caches need TLS. Vector search needs node-based Valkey 8.2

The staleness row catches teams out, so read the DAX consistency docs first.

Start with DynamoDB alone and measure. Before you add a cache, I'd check the access pattern (a scan, a hot partition key), because fixing key design is cheaper than running a cluster.

Load testing tools

The load model matters more than the tool. k6 (open source) gives you an open model via constant-arrival-rate, with thresholds that turn p95 budgets into CI pass or fail, though big tests need distributed runners. Locust (open source) suits Python teams and complex flows, with scripts in Python only. Apache JMeter (GUI and XML plans) covers the most protocols but is heavy to maintain as code. Distributed Load Testing on AWS (v4.3.2) runs JMeter, k6 or Locust on Fargate across Regions, though you pay for the stack and still design the test.

Use an open model. In a closed model each virtual user waits for its last response, so a slowing system receives less load and the test hides the problem. That's coordinated omission (k6 docs).

Decide: the right fix, and what it costs elsewhere

Match the symptom to the fix before buying capacity.

If your problem is…	Reach for	Not
Static assets slow far from the origin	CloudFront with a tight cache key	A bigger origin
Repeated identical reads on DynamoDB	DAX	A hand-written cache
Expensive computed results or mixed sources	ElastiCache for Valkey, cache-aside	DAX (it caches items, not your computations)
Init Duration in your p99	SnapStart, then provisioned concurrency	More memory alone
Steady high-volume compute	Lambda Managed Instances or Fargate	Provisioned concurrency sized for peak all day
LLM responses feel slow	Streaming, prompt caching, smaller model	A bigger instance of anything

Each fix touches the other NFRs. Some effects help. Each cache hit is load the database never sees (scalability), stale-if-error serves cached content while the origin is down (availability), fast INP helps low-end devices (accessibility), and cache hits and smaller models cost less (cost) and use less compute per request (sustainability).

Others cost you:

Security: a cache key missing the user ID serves one user's page to another.
Compliance: latency-optimized and cross-Region inference process prompts outside your home Region.
Accessibility: streaming text floods screen readers.
Availability: a cold cache after failover stampedes the database.
Scalability: a cache sized for today's keys evicts hot ones at tomorrow's volume.
Maintainability: invalidation logic is a classic source of bugs.
Cost: provisioned concurrency, the Priority tier and idle nodes bill regardless.
Sustainability: always-warm capacity burns energy at idle.

None of that means skip the work. Just write down what you traded.

Hands-on: cache-aside in front of DynamoDB, with a p95 gate

Let's build the middle of the first diagram: a Lambda read path for GET /products/{id} with ElastiCache Serverless for Valkey in front of DynamoDB, plus a k6 gate. The cache can speed requests up but never break them, TTLs carry jitter so hot keys don't expire together, and latency is emitted per path.

sequenceDiagram
    participant G as API Gateway
    participant L as Lambda
    participant V as Valkey
    participant D as DynamoDB
    participant CW as CloudWatch
    G->>L: GET /products/id
    L->>V: GET product key, 50 ms timeout
    alt cache hit
        V-->>L: cached product
    else miss, timeout or cache error
        L->>D: GetItem, eventually consistent
        D-->>L: item
        L->>V: SET with TTL of 300 s, plus or minus 10%
        Note over L,V: a failed SET is ignored
    end
    L->>CW: EMF log line, ProductReadLatency by CacheResult
    L-->>G: 200 with the product, or 404

Every cache failure falls through to DynamoDB, and the jittered TTL spreads out when hot keys expire.

Checked against the valkey-py docs, boto3 DynamoDB get_item, ElastiCache in-transit encryption and CloudWatch Embedded Metric Format on Oct 28, 2026. Not deployed, so try it in a sandbox account first.

# handler.py — cache-aside read path. Python 3.12 Lambda in the cache's VPC,
# with a DynamoDB gateway VPC endpoint so it can reach the table without NAT.
# Package dependency: valkey (pip install valkey). boto3 is in the Lambda runtime.
import json, os, random, time
import boto3
from boto3.dynamodb.types import TypeDeserializer
from valkey.cluster import ValkeyCluster

TABLE = os.environ["TABLE_NAME"]
TTL_SECONDS = int(os.environ.get("CACHE_TTL_SECONDS", "300"))

# Created once per execution environment, outside the handler, so warm invocations reuse them.
ddb = boto3.client("dynamodb")
deser = TypeDeserializer()
_cache = None

def cache():
    # Lazy: if the cache is unreachable at cold start, the function still serves from DynamoDB.
    global _cache
    if _cache is None:
        _cache = ValkeyCluster(
            host=os.environ["CACHE_ENDPOINT"],  # serverless cache endpoint
            port=6379,
            ssl=True,             # serverless caches have in-transit encryption enabled
            socket_timeout=0.05,  # 50 ms: a slow cache must not be slower than the database
            decode_responses=True,
        )
    return _cache

def emit_latency(ms, cache_result):
    # Embedded Metric Format: this log line becomes a CloudWatch metric, no PutMetricData call.
    print(json.dumps({
        "_aws": {
            "Timestamp": int(time.time() * 1000),
            "CloudWatchMetrics": [{
                "Namespace": "Nfr/Performance",
                "Dimensions": [["CacheResult"]],
                "Metrics": [{"Name": "ProductReadLatency", "Unit": "Milliseconds"}],
            }],
        },
        "CacheResult": cache_result,
        "ProductReadLatency": ms,
    }))

def get_product(product_id):
    key = f"product:{product_id}"
    try:
        cached = cache().get(key)
        if cached is not None:
            return json.loads(cached), "hit"
    except Exception:
        pass  # cache trouble degrades to a database read; it never fails the request
    # Eventually consistent read (the default): half the cost of a strongly consistent one.
    item = ddb.get_item(TableName=TABLE, Key={"pk": {"S": product_id}}).get("Item")
    if item is None:
        return None, "miss"  # not cached: consider caching misses briefly if bots probe random IDs
    product = {k: deser.deserialize(v) for k, v in item.items()}
    try:
        ttl = int(TTL_SECONDS * random.uniform(0.9, 1.1))  # +/-10% jitter spreads expiry
        cache().set(key, json.dumps(product, default=str), ex=ttl)
    except Exception:
        pass
    return product, "miss"

def handler(event, context):
    start = time.perf_counter()
    product, result = get_product(event["pathParameters"]["id"])
    emit_latency((time.perf_counter() - start) * 1000, result)
    if product is None:
        return {"statusCode": 404, "body": "{}"}
    return {"statusCode": 200, "headers": {"Content-Type": "application/json"},
            "body": json.dumps(product, default=str)}

Graph ProductReadLatency p99 by CacheResult. If hit and miss sit close, the cache isn't helping. Delete it.

Now the gate. This k6 test holds 200 requests per second for ten minutes and exits non-zero if p95 or p99 breaks budget.

Checked against the k6 thresholds and constant-arrival-rate docs on Oct 28, 2026. Not deployed, so try it in a sandbox account first.

// load-test.js — run: k6 run -e BASE_URL=https://your-api.example.com load-test.js
import http from 'k6/http';
import { check } from 'k6';

export const options = {
  scenarios: {
    steady_reads: {
      executor: 'constant-arrival-rate', // open model: load does not drop when the API slows
      rate: 200,                         // 200 iterations...
      timeUnit: '1s',                    // ...per second
      duration: '10m',
      preAllocatedVUs: 50,
      maxVUs: 400,                       // headroom to hold the rate as latency rises
    },
  },
  thresholds: {
    'http_req_duration{endpoint:product}': ['p(95)<150', 'p(99)<400'],
    http_req_failed: [{ threshold: 'rate<0.01', abortOnFail: true, delayAbortEval: '30s' }],
    checks: ['rate>0.99'],
  },
};

const ids = ['p-001', 'p-002', 'p-003', 'p-004', 'p-005'];

export default function () {
  // Skewed traffic: 80% of reads hit one hot item, as in most real catalogs.
  const id = Math.random() < 0.8 ? ids[0] : ids[Math.floor(Math.random() * ids.length)];
  const res = http.get(`${__ENV.BASE_URL}/products/${id}`, { tags: { endpoint: 'product' } });
  check(res, { 'status is 200': (r) => r.status === 200 });
}

For a real run, load thousands of IDs with SharedArray so the hit ratio looks like production.

Performance in the age of AI

LLM features need three budgets

An LLM call spends most of its time generating output, but users judge it by when the first words appear. Give it three budgets.

Time to first token (TTFT) covers queueing and prompt processing. Prompt length and caching, model and tier move it, and Bedrock added a TimeToFirstToken CloudWatch metric for streaming APIs in March 2026. Output tokens per second (OTPS) is generation speed, set by the model, the tier and latency-optimized inference. You derive it as OutputTokenCount / (InvocationLatency − TimeToFirstToken). End-to-end p95 is everything, retrieval and tools included. maxTokens, agent steps and retrieval move it, and your trace spans show it.

sequenceDiagram
    participant U as Browser
    participant G as API Gateway
    participant L as Lambda
    participant R as Retriever
    participant B as Bedrock
    U->>G: POST /chat
    G->>L: invoke in STREAM mode
    Note over L: cold start only if no warm environment
    L->>R: embed question and vector search
    R-->>L: top-k chunks
    L->>B: ConverseStream with cached prefix
    Note over B: queue and prefill, a cache hit shortens this
    B-->>L: first token
    L-->>U: first chunk, TTFT ends here
    loop decode at OTPS
        B-->>L: tokens
        L-->>U: chunks
    end
    B-->>L: metadata with usage and latencyMs
    L-->>U: stream closes, end-to-end ends here

Everything before the first chunk (cold start, retrieval, prefill) is dead air for the user, so optimize there first.

Streaming, and the plumbing that limits it

Streaming is the biggest perceived-performance lever, but the plumbing has limits. API Gateway REST integrations time out at 29 seconds. Regional and private APIs can raise it, possibly lowering your throttle quota. HTTP APIs stop at a fixed 30 seconds. Since November 2025, REST APIs can stream proxy integrations for up to 15 minutes (5-minute idle timeout on Regional endpoints), without endpoint caching or VTL.

Lambda streams natively on Node.js, while Python needs a custom runtime or the Lambda Web Adapter. Streams reach 200 MB, capped at 2 MBps after the first 6 MB.

The Bedrock levers

Once you stream, these are Bedrock's knobs.

Lever	Effect	Use when	Caveat (as of Oct 2026)
Prompt caching	Up to 85% lower latency, 90% lower cost per AWS	Long, stable system prompts, documents, tool schemas	512–4,096-token minimum by model. 5-minute default TTL. Not for batch
Priority tier	Up to 25% better OTPS than Standard on most supported models, at a premium	Customer-facing paths at peak	Limited model list. Shares your on-demand quota
Flex tier	Discount for slower processing	Evals, summarization, background agents	Never on an interactive path
Latency-optimized inference	`performanceConfig: {latency: "optimized"}`	Nova Pro, Claude 3.5 Haiku, Llama 3.1 70B/405B	Preview. US Regions via cross-Region inference only. Falls back to standard past its quota
Distilled model	Up to 500% faster, 75% cheaper, under 2% accuracy loss for RAG, per AWS	High-volume, narrow tasks	You need evals to prove quality held
Intelligent Prompt Routing	Routes each prompt to a cheaper or stronger model in one family	Mixed easy and hard traffic	English-optimized. Same-family models only

Start with prompt caching. The launch figures are AWS maximums, so measure your own.

Exact vs semantic caching

Why not cache answers too? An exact cache, keyed on a hash of the normalized prompt, is safe but rarely hits on free text. A semantic cache answers when a past question's embedding is close. ElastiCache for Valkey 8.2 on node-based clusters has offered vector search since October 2025 at no extra charge, with up to 99% recall per AWS.

The risk is a confident wrong hit, since "cancel my order" and "cancel my subscription" sit close in embedding space. Scope keys by tenant and intent, use a strict threshold and short TTLs, and never semantically cache user-specific answers.

flowchart LR
    Q["New question<br/>cancel my subscription"] --> E["Embed it and search<br/>past questions"]
    E --> N["Closest past question<br/>cancel my order"]
    N -->|"loose threshold, unscoped keys"| W["Confident wrong hit<br/>returns the order answer"]
    N -->|"strict threshold, keys scoped by tenant and intent"| M["Treated as a miss<br/>the model answers"]

Two questions can sit close in embedding space and still need different answers, so the threshold and key scope decide what counts as a hit.

Retrieval latency

Retrieval lands in TTFT. In AWS's September 2026 benchmarks, OpenSearch measured p50 31 ms and p99 52 ms (1.2M vectors), Aurora PostgreSQL with pgvector about 32 ms and 89–94 ms, and S3 Vectors 294 ms and 506 ms (1M vectors). The workloads differed, so compare shape, not ranking (AWS ML Blog, 2026). S3 Vectors holds 2 billion vectors per index at up to 90% lower cost (GA December 2025). It's a good full-corpus store behind a faster hot tier, and a poor fit for a 1-second TTFT budget.

Agent loops multiply everything

Plan, three sequential tool calls, answer: at least five round-trips, each paying TTFT again. Parallelize independent tools, cap iterations and trace each step. CloudWatch generative AI observability (GA October 2025) traces model calls through Amazon Bedrock AgentCore operations.

Measure before you tune

This prints all three budgets and the prompt cache result.

Checked against boto3 converse_stream and Bedrock prompt caching on Oct 28, 2026. Not deployed, so try it in a sandbox account first.

# ttft.py — TTFT, output tokens/sec and end-to-end latency for streamed Bedrock calls
import os, time, boto3

client = boto3.client("bedrock-runtime")
MODEL_ID = os.environ["MODEL_ID"]                  # a model or inference profile you can invoke
SYSTEM_PROMPT = open("system_prompt.txt").read()   # long, stable prefix: eligible for caching

def measure(question):
    start, first = time.perf_counter(), None
    usage = {}
    resp = client.converse_stream(
        modelId=MODEL_ID,
        system=[{"text": SYSTEM_PROMPT}, {"cachePoint": {"type": "default"}}],
        messages=[{"role": "user", "content": [{"text": question}]}],
        inferenceConfig={"maxTokens": 400},        # an output cap is also a latency cap
        serviceTier={"type": os.environ.get("SERVICE_TIER", "default")},  # or "priority"
    )
    for event in resp["stream"]:
        if "contentBlockDelta" in event and first is None:
            first = time.perf_counter()
        if "metadata" in event:
            usage = event["metadata"]["usage"]
    end = time.perf_counter()
    first = first or end
    print({
        "ttft_ms": round((first - start) * 1000),
        "e2e_ms": round((end - start) * 1000),
        "otps": round(usage.get("outputTokens", 0) / max(end - first, 1e-6), 1),
        "cache_read": usage.get("cacheReadInputTokens", 0),
        "cache_write": usage.get("cacheWriteInputTokens", 0),
    })

for q in ["What is the refund window?", "Do you ship to Ghana?"]:
    measure(q)  # the second call should report cache_read > 0 if the prefix met the minimum

If Bedrock's TimeToFirstToken is low but your client-side TTFT is high, look at your own path.

AI as an engineering aid

AI tools help with the tedious parts: turning a trace into a hypothesis, drafting a k6 scenario, explaining a flame graph. CloudWatch investigations (GA June 2025, no additional cost) proposes root-cause hypotheses from your metrics, logs and traces.

They also mislead predictably: caches with no invalidation story, closed-model load tests, numbers from nowhere. Three rules help. Every proposed optimization ships with a before-and-after measurement from your environment. Generated load tests use arrival-rate executors and thresholds. Any number an assistant offers needs a link before it reaches a design doc.

Who owns what

Performance slips when nobody owns it. The product owner is accountable for latency budgets per journey, which the architect writes. Developers own p95/p99 and TTFT instrumentation. SRE or platform is accountable for CI load tests (developers write them) and owns alarms and regression response.

The AWS line moves with compute. On EC2 it handles hardware and hypervisor, and you own instance type, OS tuning, scaling and connection pools. ECS on Fargate takes hosts and OS off you, leaving task sizing, scaling and app code. Lambda runs environments and scaling mechanics, and you own memory, init code and concurrency settings. Bedrock handles model serving and tier capacity, and you own model choice, prompt and output length, streaming, caching and quotas.

Put simply, the provider owns how fast a model generates. You own how many tokens you send and request, and how often you call.

Your Monday checklist

[ ] Write one budget sentence (percentile, endpoint, load) for your most important user journey.
[ ] Replace every average-latency alarm with p95 and p99 alarms.
[ ] Check LCP, INP and CLS at p75 on mobile from field data.
[ ] Remove every header, cookie and query string from CloudFront cache keys that the origin doesn't use.
[ ] Compare Init Duration in Lambda logs with your p99, and run Lambda Power Tuning on your busiest function.
[ ] Add a k6 constant-arrival-rate test with p(95) thresholds to your pipeline.
[ ] Chart TimeToFirstToken and OTPS for every LLM feature, and confirm responses stream.
[ ] Move stable instructions to the front of prompts, add a cachePoint, and check cacheReadInputTokens.

Conclusion

Comparisons need a ruler. A cache, tier or runtime option is only better against a budget, at a percentile, under a load. With one, the options here become decisions. DAX when DynamoDB reads repeat. Valkey when results are computed. SnapStart when INIT owns your tail. Streaming and prompt caching when an LLM is the slowest hop. Without it, you're tuning by feel.

Vodafone's faster page was still slow, and it sold 8% more. You don't need to be fast in the abstract, just measurably faster where your user is waiting.

Pick the percentile first. The architecture follows from it.

Series: Non-functional Application Requirements

Non-functional Application Requirements: An Introduction
Non-functional Application Requirements: Security
Non-functional Application Requirements: Compliance
Accessibility
Performance
Availability (coming soon)
Scalability (coming soon)
Maintainability (coming soon)
Cost (coming soon)
Sustainability (coming soon)

Cover image: AWS re:Invent 2023 – Keynote with Dr. Werner Vogels

Next up, Availability: a fast system that's down is infinitely slow, so we'll look at composite SLOs, static stability, and what happens when your model provider throttles.

Sources

Non-functional Application Requirements: Compliance

Oluwasegun Adedigba — Tue, 25 Nov 2025 12:58:58 +0000

Compliance isn't just a checkbox. It's the difference between a thriving business and a company facing €1.2 billion in fines. In 2023, Meta paid that staggering amount for GDPR violations related to improperly transferring user data from the EU to the U.S. Meanwhile, healthcare organizations faced $4.75 million settlements for HIPAA failures, and retailers continued paying for breaches years after they occurred.

Understanding compliance isn't optional anymore. It's the foundation of sustainable software development. Whether you're building a healthcare app, processing payments, or launching a SaaS platform, compliance requirements shape your architecture, influence your cloud provider choices, and directly impact your bottom line.

This article breaks down what compliance really means, why it matters more than ever, and exactly how to implement it using AWS services before you become the next cautionary tale.

What: Understanding Compliance Frameworks and Requirements

Compliance represents your legal and contractual obligation to handle data according to specific regulatory standards. Unlike optional best practices, compliance failures result in mandatory fines, lawsuits, and business shutdowns. Modern applications typically need to comply with multiple frameworks simultaneously. A healthcare SaaS might need HIPAA, GDPR, and SOC 2 compliance all at once.

The Major Compliance Frameworks

1. GDPR (General Data Protection Regulation)

Scope: Any organization processing personal data of EU residents, regardless of company location.

Key Requirements:

Lawful Basis for Processing: You must have a valid legal reason to process personal data (consent, contract, legitimate interest, etc.)
Data Subject Rights: Users can request access, deletion, portability, and rectification of their data
Privacy by Design: Data protection must be built into systems from the start
Breach Notification: Must notify authorities within 72 hours of discovering a breach
Data Protection Officer (DPO): Required for organizations processing data at scale

Real-World Violation: In May 2023, Ireland's Data Protection Commission fined Meta €1.2 billion for transferring EU user data to the U.S. without adequate safeguards. This remains the largest GDPR fine ever imposed. More recently, in 2024, the Dutch DPA fined Uber €290 million for transferring sensitive driver data from the EU to the US without proper transfer mechanisms after they stopped using Standard Contractual Clauses in 2021.

2. HIPAA (Health Insurance Portability and Accountability Act)

Scope: U.S. healthcare providers, health plans, healthcare clearinghouses, and their business associates.

Key Requirements:

Privacy Rule: Establishes standards for protecting individuals' medical records and personal health information (PHI)
Security Rule: Requires administrative, physical, and technical safeguards to protect electronic PHI (ePHI)
Breach Notification Rule: Mandates notification to affected individuals, HHS, and sometimes the media
Risk Analysis: Organizations must conduct thorough security risk assessments
Business Associate Agreements (BAAs): Required contracts with any vendor handling PHI

Real-World Violation: In February 2024, Montefiore Medical Center settled with HHS for $4.75 million following a malicious insider cybersecurity investigation. The breach occurred when an employee stole and sold the protected health information of 12,517 patients. This case highlighted that threats don't just come from external hackers. Insider threats require equally rigorous controls.

3. PCI DSS (Payment Card Industry Data Security Standard)

Scope: Any organization that stores, processes, or transmits cardholder data.

Key Requirements:

Build and Maintain a Secure Network: Install firewalls and don't use vendor-supplied defaults
Protect Cardholder Data: Encrypt transmission and storage of cardholder data
Maintain a Vulnerability Management Program: Use and regularly update antivirus, develop secure systems
Implement Strong Access Control Measures: Restrict access on a need-to-know basis
Regularly Monitor and Test Networks: Track and monitor all access, regularly test security systems
Maintain an Information Security Policy: Create comprehensive security policies for personnel

Real-World Violation: In 2013, Target suffered one of the most infamous PCI compliance breaches when hackers stole 40 million credit card numbers. Despite having a $1.6 million malware detection tool in place, Target missed critical warnings for three weeks. The result? An $18.5 million multi-state settlement, over $202 million in total costs, and lasting reputational damage.

4. SOC 2 (System and Organization Controls 2)

Scope: Service organizations that store customer data in the cloud.

Key Requirements (Trust Service Criteria):

Security: Protection against unauthorized access
Availability: System accessibility as contractually agreed
Processing Integrity: System processing is complete, valid, accurate, timely, and authorized
Confidentiality: Information designated as confidential is protected
Privacy: Personal information is collected, used, retained, disclosed, and disposed of properly

Business Impact: Many enterprise organizations won't even consider vendors without SOC 2 Type II certification. A failed audit can mean losing major deals and damaging your market position.

5. ISO 27001

Scope: Any organization seeking to establish, implement, maintain, and continually improve an information security management system (ISMS).

Key Requirements:

Risk Assessment: Identify and assess information security risks
Risk Treatment: Implement controls to address identified risks
Documentation: Maintain comprehensive documentation of the ISMS
Internal Audits: Regular internal audits of the ISMS
Management Review: Senior management oversight and commitment
Continuous Improvement: Ongoing enhancement of the ISMS

Business Impact: ISO 27001 certification is often required for government contracts and enterprise partnerships, particularly in Europe and internationally.

6. CCPA/CPRA (California Consumer Privacy Act/California Privacy Rights Act)

Scope: Businesses collecting personal information of California residents meeting certain thresholds.

Key Requirements:

Right to Know: Consumers can request what personal information is collected
Right to Delete: Consumers can request deletion of their personal information
Right to Opt-Out: Consumers can opt out of the sale of their personal information
Right to Correct: Consumers can request correction of inaccurate information (CPRA)
Right to Limit Use: Consumers can limit use of sensitive personal information (CPRA)

Why: The Critical Importance of Compliance

Security isn't just a "nice-to-have" feature. It's essential. Let's consider why a robust compliance strategy is non-negotiable:

Financial Impact

The numbers tell a compelling story:

€5.88 billion: Total GDPR fines issued since 2018 through 2025
€1.2 billion: Single largest GDPR fine (Meta, 2023)
$4.88 million: Average cost of a data breach globally in 2024 (IBM)
$174,000: Additional average cost for non-compliant organizations per breach
10%: Year-over-year increase in breach costs, the highest total ever recorded

For payment processing, the stakes are equally high:

$5,000 - $100,000/month: Typical PCI non-compliance penalties
$50 - $90 per customer: Additional fines per affected individual after a breach
Revoked merchant license: The ultimate penalty, losing the ability to accept credit cards entirely

Trust and Reputation

A single data breach can erode user trust for years. The reputational damage leads to:

Customer churn and loss of market share
Media fallout and public relations crises
Difficulty attracting enterprise customers
Challenges in recruiting top talent
Investor concern and reduced valuations

When TikTok faced its €530 million GDPR fine in 2025 for transferring EU user data to China without adequate safeguards, the damage extended far beyond the financial penalty. The investigation revealed that engineers in China routinely accessed sensitive information of European users. That revelation intensified regulatory scrutiny and user concern globally.

Operational Stability

Insecure, non-compliant applications are more vulnerable to:

Cyber attacks that cause downtime and data loss
Ransomware incidents that halt operations
Regulatory investigations that consume resources
Mandatory forensic examinations after breaches
Court-ordered operational changes

The Compound Effect

When compliance is compromised, it's not just a technical issue. It's a business crisis. Consider LinkedIn's 2024 experience: the Irish DPC fined them €310 million for processing user data for behavioral analysis and targeted advertising without a valid legal basis. The company not only paid the fine but was ordered to bring its data processing into compliance, requiring significant technical and operational changes.

When: Embedding Compliance from Day One

In today's rapid development environments, compliance needs to be part of the design and development process from the outset. The earlier compliance is embedded, the less costly it is to manage and mitigate issues later. This "shift-left" approach focuses on integrating compliance practices throughout the entire development lifecycle.

Key Stages to Integrate Compliance

Design Phase

Data Flow Mapping: Document what data you collect, where it's stored, how it flows through your system, and who has access
Compliance Requirements Analysis: Identify which frameworks apply to your business based on geography, industry, and data types
Privacy Impact Assessments: Evaluate how new features or systems might affect user privacy
Architecture Decisions: Choose infrastructure and services that support compliance requirements (e.g., data residency)

Development Phase

Secure Coding Practices: Implement encryption, input validation, and secure authentication from the start
Data Minimization: Collect only the data you actually need
Access Controls: Build role-based access control (RBAC) into your application
Audit Logging: Implement comprehensive logging of data access and modifications
Configuration as Code: Use infrastructure as code to ensure consistent, compliant deployments

Testing and QA Phase

Compliance Testing: Automated checks for compliance with relevant standards
Penetration Testing: Regular security assessments to identify vulnerabilities
Data Protection Testing: Verify encryption, access controls, and data handling procedures
Audit Trail Verification: Ensure all required events are being logged correctly

Deployment and Operations Phase

Continuous Monitoring: Real-time compliance posture assessment
Automated Remediation: Immediate response to compliance drift
Incident Response Plans: Documented procedures for breach notification and handling
Regular Audits: Scheduled compliance assessments and certifications

The Cost of Delay

Addressing compliance after the fact is exponentially more expensive:

Design phase fix: ~1x cost
Development phase fix: ~6x cost
Testing phase fix: ~15x cost
Production fix: ~100x cost

By shifting compliance left, you address potential issues early, reducing the risk of costly fixes, regulatory penalties, and reputational damage down the road.

Where: Leveraging AWS Compliance Services

AWS offers a comprehensive suite of services that cover different aspects of application compliance. Let's explore the key tools and how they work together to create a robust compliance infrastructure.

AWS CloudTrail: Your Audit Log Foundation

CloudTrail is the foundational service for compliance on AWS. It records all API activity across your AWS infrastructure, providing an immutable audit log that's essential for compliance.

Key Features:

Comprehensive Logging: Captures API calls from the console, CLI, SDKs, and other AWS services
Event History: 90 days of management events free, with trails for longer retention
CloudTrail Lake: Enables SQL-based querying of event data for investigations
Integrity Validation: Log file integrity validation ensures logs haven't been tampered with

Compliance Value:

Satisfies audit logging requirements for HIPAA, PCI DSS, SOC 2, and GDPR
Provides evidence for security investigations and incident response
Enables detection of unauthorized access or suspicious activity

# Example: Creating a CloudTrail trail with log file validation
import boto3

cloudtrail = boto3.client('cloudtrail')

response = cloudtrail.create_trail(
    Name='compliance-trail',
    S3BucketName='your-audit-logs-bucket',
    IncludeGlobalServiceEvents=True,
    IsMultiRegionTrail=True,
    EnableLogFileValidation=True,
    KMSKeyId='your-kms-key-arn'  # Encrypt logs at rest
)

AWS Config: Continuous Configuration Assessment

AWS Config continuously monitors and records your AWS resource configurations and allows you to automate the evaluation of configurations against desired compliance rules.

Key Features:

Configuration Recording: Tracks resource configurations over time
Config Rules: Define rules that evaluate configurations against compliance requirements
Conformance Packs: Collections of rules mapped to compliance standards (PCI DSS, HIPAA, etc.)
Automatic Remediation: Trigger automated fixes when resources drift from compliance

Pre-Built Conformance Packs:

AWS Operational Best Practices for PCI DSS 3.2.1
AWS Operational Best Practices for HIPAA
AWS Operational Best Practices for GDPR
AWS Operational Best Practices for CIS AWS Foundations Benchmark

# Example: AWS Config Rule for S3 bucket encryption
Resources:
  S3BucketEncryptionRule:
    Type: AWS::Config::ConfigRule
    Properties:
      ConfigRuleName: s3-bucket-server-side-encryption-enabled
      Description: Checks whether S3 buckets have server-side encryption enabled
      Source:
        Owner: AWS
        SourceIdentifier: S3_BUCKET_SERVER_SIDE_ENCRYPTION_ENABLED
      Scope:
        ComplianceResourceTypes:
          - AWS::S3::Bucket

AWS Security Hub: Centralized Security Posture Management

Security Hub aggregates findings from multiple AWS security services and third-party tools, providing a comprehensive view of your security and compliance posture.

Key Features:

Automated Security Checks: Continuous compliance evaluation against industry standards
Finding Aggregation: Centralized view from GuardDuty, Inspector, Config, and third-party tools
Security Score: At-a-glance view of overall compliance status
Cross-Account Visibility: Aggregate findings across your entire AWS organization

Supported Compliance Standards:

CIS AWS Foundations Benchmark (v1.2.0, v1.4.0, v3.0.0)
AWS Foundational Security Best Practices
PCI DSS v3.2.1
NIST Special Publication 800-53 Revision 5

# Example: Enabling Security Hub with standards
import boto3

securityhub = boto3.client('securityhub')

# Enable Security Hub
securityhub.enable_security_hub(
    EnableDefaultStandards=False
)

# Enable specific standards
securityhub.batch_enable_standards(
    StandardsSubscriptionRequests=[
        {
            'StandardsArn': 'arn:aws:securityhub:::ruleset/cis-aws-foundations-benchmark/v/1.4.0'
        },
        {
            'StandardsArn': 'arn:aws:securityhub:::ruleset/pci-dss/v/3.2.1'
        }
    ]
)

AWS Audit Manager: Automated Audit Evidence Collection

Audit Manager helps you continuously audit your AWS usage and simplify how you assess risk and compliance with regulations and industry standards.

Key Features:

Prebuilt Frameworks: Ready-to-use frameworks for GDPR, HIPAA, PCI DSS, SOC 2, ISO 27001, and more
Automated Evidence Collection: Automatically gathers evidence from CloudTrail, Config, Security Hub
Custom Frameworks: Create frameworks tailored to your specific compliance needs
Assessment Reports: Generate audit-ready reports for stakeholders and auditors

Evidence Types Collected:

Configuration data from API calls
User activity from CloudTrail logs
Compliance check results from Security Hub and AWS Config
Manual evidence uploads for documentation and procedures

# Example: Creating an assessment in Audit Manager
import boto3

audit_manager = boto3.client('auditmanager')

# Create an assessment using the SOC 2 framework
response = audit_manager.create_assessment(
    name='SOC2-Annual-Audit-2025',
    description='Annual SOC 2 Type II audit assessment',
    assessmentReportsDestination={
        'destinationType': 'S3',
        's3BucketName': 'your-audit-reports-bucket'
    },
    scope={
        'awsAccounts': [
            {'id': '123456789012'}
        ],
        'awsServices': [
            {'serviceName': 'S3'},
            {'serviceName': 'EC2'},
            {'serviceName': 'RDS'},
            {'serviceName': 'Lambda'}
        ]
    },
    roles=[
        {
            'roleType': 'PROCESS_OWNER',
            'roleArn': 'arn:aws:iam::123456789012:role/AuditProcessOwner'
        }
    ],
    frameworkId='SOC2-framework-id'  # Use prebuilt SOC 2 framework
)

AWS Artifact: Compliance Documentation Hub

AWS Artifact provides on-demand access to AWS compliance documentation and enables you to manage agreements with AWS.

Key Features:

Compliance Reports: Access AWS SOC reports, PCI DSS attestation, ISO certifications
Agreements: Accept and manage AWS Business Associate Agreement (BAA) for HIPAA
Third-Party Audit Reports: Review independent auditor assessments of AWS
Self-Service Access: Download reports anytime for audit preparation

Available Reports Include:

SOC 1, SOC 2, and SOC 3 Reports
PCI DSS Attestation of Compliance
ISO 27001, 27017, 27018, and 9001 Certifications
FedRAMP Documentation
HIPAA Eligible Services and BAA

AWS Macie: Automated Data Discovery and Protection

Amazon Macie uses machine learning to automatically discover, classify, and protect sensitive data in AWS.

Key Features:

Automated Discovery: Continuously scans S3 buckets for sensitive data
Classification: Identifies PII, PHI, financial data, and custom data types
Security Alerts: Generates findings when sensitive data is at risk
Data Inventory: Provides visibility into what sensitive data exists and where

Compliance Value:

Supports GDPR data mapping requirements
Helps maintain HIPAA PHI inventories
Identifies PCI DSS cardholder data exposure

Layer 1 - CloudTrail (Foundation): Records all API activity
Layer 2 - AWS Config (Configuration): Monitors resource configurations against rules
Layer 3 - Security Hub (Dashboard): Aggregates findings and provides compliance scores
Layer 4 - Audit Manager (Evidence): Collects evidence and generates audit reports

Who: The Roles and Responsibilities in Compliance

Compliance is a shared responsibility. It's not just within your organization but also with your cloud provider. Understanding AWS's Shared Responsibility Model is crucial for compliance success.

The AWS Shared Responsibility Model

AWS is responsible for:

Physical security of data centers
Infrastructure hardware and software
Network infrastructure
Virtualization layer

You are responsible for:

Data encryption and protection
Identity and access management
Network traffic protection
Application security
Operating system and platform configuration
Compliance with specific regulations for your industry

Roles Within Your Organization

Compliance Officers

Define compliance requirements and policies
Coordinate with legal and regulatory bodies
Oversee audit processes and certifications
Report to executive leadership on compliance posture

Cloud Architects and DevOps Engineers

Design compliant architectures
Implement security controls and automation
Configure AWS compliance services
Maintain infrastructure as code for consistent deployments

Developers

Write secure, compliant code
Implement data protection at the application level
Follow secure SDLC practices
Conduct code reviews with security in mind

Security Analysts

Monitor security alerts and compliance dashboards
Investigate potential violations
Respond to incidents
Conduct risk assessments

Data Protection Officers (where required)

Ensure GDPR compliance
Handle data subject requests
Conduct privacy impact assessments
Advise on data protection matters

Building a Compliance Culture

Successful compliance requires more than tools and processes. It requires a culture where everyone understands their role:

Executive Sponsorship: Compliance initiatives must have visible support from leadership
Training Programs: Regular training on compliance requirements and procedures
Clear Ownership: Defined responsibilities for each compliance domain
Incentive Alignment: Ensure security and compliance goals are reflected in team objectives
Continuous Communication: Regular updates on compliance status and changes

Compliance Best Practices: Building a Resilient Application

1. Implement Defense in Depth

Don't rely on a single control. Layer your compliance measures:

Network segmentation and encryption
Application-level access controls
Data encryption at rest and in transit
Comprehensive logging and monitoring

2. Automate Everything Possible

Manual compliance processes don't scale:

Use AWS Config conformance packs for automated evaluation
Implement automated remediation for common issues
Deploy infrastructure as code for consistent, compliant deployments
Schedule automated compliance reports

3. Maintain Comprehensive Documentation

Auditors will ask for evidence:

Document all security policies and procedures
Maintain records of training and awareness programs
Keep evidence of regular reviews and updates
Archive audit logs with integrity verification

4. Plan for Breach Notification

Hope for the best, prepare for the worst:

Create incident response playbooks
Define notification procedures for each regulation
Maintain contact information for regulatory authorities
Practice incident response with tabletop exercises

5. Implement Data Minimization

You can't lose what you don't have:

Collect only the data you need
Implement data retention policies
Securely delete data when no longer needed
Anonymize or pseudonymize where possible

6. Regular Compliance Audits

Don't wait for the external audit to find issues:

Conduct internal audits quarterly
Use AWS Audit Manager for continuous assessment
Address findings promptly
Track remediation progress

7. Monitor Third-Party Risk

Your compliance extends to your vendors:

Require compliance certifications from vendors
Include compliance requirements in contracts
Regular assessment of third-party security
Maintain business associate agreements (BAAs) where required

Conclusion: Compliance as a Competitive Advantage

In an era where data breaches make headlines daily and regulatory scrutiny intensifies, compliance isn't just about avoiding fines. It's a competitive differentiator.

Companies that prioritize compliance:

Win enterprise deals that require SOC 2 and ISO 27001 certifications
Expand globally with confidence in meeting GDPR and regional requirements
Build customer trust through demonstrated commitment to data protection
Reduce risk of costly breaches and regulatory actions
Operate efficiently with automated compliance processes

AWS provides the tools to make compliance manageable and affordable. By combining CloudTrail for audit logging, AWS Config for continuous monitoring, Security Hub for centralized visibility, and Audit Manager for evidence collection, you can build a compliance program that scales with your business.

Remember: The best time to implement compliance was when you started building. The second best time is now.

Cover Image: AWS re:Invent 2023 - Keynote with Dr. Werner Vogels

This article is part of a series on Non-functional Application Requirements. Read the previous articles:

Next up: Accessibility - Designing for Everyone

Provisioning AWS Infrastructure using Terraform and GitHub Actions

Oluwasegun Adedigba — Thu, 02 Oct 2025 12:32:15 +0000

Introduction

Availability, repeatability, and security are table stakes for production workloads. This guide provisions a baseline AWS stack with Terraform that's resilient to AZ failures, enforces least-privilege boundaries, integrates with CI/CD, and manages Terraform state with an S3 and DynamoDB backend. We'll deploy four Apache web servers across two availability zones with a Multi-AZ RDS database, all automated through GitHub Actions.

Why This Matters

Incidents rarely happen at convenient times. You want deterministic deployments, blast-radius isolation, and multi-AZ redundancy so failures degrade gracefully. When one availability zone experiences issues, your application continues running on servers in the healthy zone without any manual intervention.

Equally important is proper Terraform state management. State must be remote rather than stored on a laptop, it must be locked to prevent concurrent modifications, it must be encrypted and versioned for security and recovery, and it should be accessible via IAM rather than passed around in Slack or email. This becomes critical when working in teams or using automated CI/CD pipelines because everyone needs access to the same source of truth about what infrastructure exists.

This article covers both infrastructure resources and state management so you can run with confidence. We'll also set up a complete GitHub Actions pipeline that automatically detects changes to your Terraform code and deploys them, replacing the manual apply process with automated continuous deployment.

What We're Building

The architecture we're building includes these components working together to create a highly available system. We'll provision a VPC with public and private subnets spread across two availability zones in the London region. In the public subnets, we'll deploy four EC2 instances running Apache web servers, with two instances in each availability zone. An Application Load Balancer will distribute incoming traffic across these four servers, automatically routing requests away from any unhealthy instances.

For the database tier, we'll create a Multi-AZ RDS MySQL instance that automatically maintains a standby replica in a different availability zone. If the primary database fails, RDS automatically promotes the standby to become the new primary without requiring any code changes. The database will live in private subnets with no internet access, protected by security groups that only allow connections from the web servers.

For state management, we'll configure an S3 bucket with versioning and encryption to store the Terraform state file, along with a DynamoDB table that provides locking to prevent multiple people or automation pipelines from modifying the infrastructure simultaneously. Finally, we'll set up GitHub Actions workflows that automatically run terraform plan on pull requests so you can review changes, and terraform apply when changes merge to the main branch, giving you the same automation benefits that Jenkins provides but using GitHub's native platform.

Here's what we're deploying:

Networking: VPC, 2 public subnets, 2 private subnets, Internet Gateway, route tables
Compute: 4 Apache web servers across 2 availability zones in an Auto Scaling Group
Ingress: Application Load Balancer with health checks and automatic failover
Data: RDS MySQL Multi-AZ in isolated private subnets
Security: Security groups scoped per role, encrypted storage, IMDSv2 enforcement
State: S3 remote state with DynamoDB locking, versioned and encrypted
CI/CD: GitHub Actions pipeline for automated terraform plan and apply
Observability: CloudWatch metrics and alarms

Prerequisites

Before starting, you'll need Terraform version 1.6 or higher installed on your local machine. You'll also need the AWS CLI configured with an IAM user or role that has permissions to create VPC, EC2, RDS, and S3 resources. While we'll create the S3 bucket and DynamoDB table for state management as our first step, you'll need initial AWS credentials to bootstrap that infrastructure.

You should also have a GitHub account and a repository where you'll store your Terraform code. The GitHub Actions workflows will run directly in your repository, so you'll need to configure AWS credentials as GitHub Secrets to allow the automation to deploy infrastructure on your behalf.

Step 1. Remote State Backend (S3 + DynamoDB)

The first thing we need to do is create the infrastructure that will manage our Terraform state. This is a one-time bootstrap process. We're creating an S3 bucket to store the state file and a DynamoDB table to provide state locking. The bucket will have versioning enabled so you can recover from accidental deletions or corrupted state, and we'll enforce encryption at rest using AES256. We're also blocking all public access to ensure the state file, which may contain sensitive information like database passwords, remains private.

The DynamoDB table uses on-demand billing so you only pay for the lock operations that actually occur, which is minimal. Terraform will write a lock entry to this table whenever someone runs an apply or plan operation, preventing others from making concurrent changes that could corrupt your infrastructure.

Create a file called backend-bootstrap.tf:

# This is a one-time setup file to create the S3 bucket and DynamoDB table
# After running this once, you can delete this file or move it to a separate directory

provider "aws" {
  region = "eu-west-2"
}

# S3 bucket to store Terraform state files
resource "aws_s3_bucket" "state" {
  bucket = "tf-state-prod-stack-eu-west-2"

  # Prevent accidental deletion of the state bucket
  lifecycle {
    prevent_destroy = true
  }
}

# Enable versioning so we can recover from bad state changes
resource "aws_s3_bucket_versioning" "state" {
  bucket = aws_s3_bucket.state.id

  versioning_configuration {
    status = "Enabled"
  }
}

# Encrypt state files at rest for security
resource "aws_s3_bucket_server_side_encryption_configuration" "state" {
  bucket = aws_s3_bucket.state.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

# Block all public access to the state bucket
resource "aws_s3_bucket_public_access_block" "state" {
  bucket                  = aws_s3_bucket.state.id
  block_public_acls       = true
  block_public_policy     = true
  restrict_public_buckets = true
  ignore_public_acls      = true
}

# DynamoDB table for state locking to prevent concurrent modifications
resource "aws_dynamodb_table" "lock" {
  name         = "tf-state-locks-prod-stack"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }
}

Run this bootstrap process once:

terraform init
terraform apply

After the bucket and table are created, you can delete this bootstrap file or move it to a separate directory. The state infrastructure is now ready to use.

Step 2. Backend Configuration

Now that we have our state storage infrastructure, we need to configure Terraform to use it. We'll create a backend configuration file for each environment. This separation allows you to have different state files for development, staging, and production environments, preventing changes in one environment from affecting others.

Create a directory structure for your environments and add a backend configuration file. For production, create envs/prod/backend.hcl:

bucket         = "tf-state-prod-stack-eu-west-2"
key            = "envs/prod/global.tfstate"
region         = "eu-west-2"
dynamodb_table = "tf-state-locks-prod-stack"
encrypt        = true

In your main Terraform directory, create a main.tf file and add the backend configuration block. Notice that we don't specify the actual bucket name here because we'll pass that in via the backend config file. This allows us to use the same Terraform code across multiple environments:

terraform {
  # Require Terraform version 1.6 or higher
  required_version = ">= 1.6"

  # Backend configuration for remote state storage
  backend "s3" {}

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = "eu-west-2"
}

Initialize Terraform with the backend configuration:

terraform init -backend-config=envs/prod/backend.hcl

Terraform will now store its state remotely in S3 and use DynamoDB for locking. You can verify this worked by checking that Terraform created a .terraform directory with backend configuration.

Step 3. Variables

We need to define variables for values that change between environments or that should not be hardcoded. Database credentials are particularly important to handle as variables because you never want to commit passwords to version control. The sensitive flag ensures these values won't appear in Terraform's output logs.

Create a variables.tf file:

variable "db_username" {
  description = "Database administrator username"
  type        = string
  sensitive   = true
}

variable "db_password" {
  description = "Database administrator password"
  type        = string
  sensitive   = true
}

variable "environment" {
  description = "Environment name used for resource tagging and naming"
  type        = string
  default     = "prod"
}

variable "aws_region" {
  description = "AWS region for resource deployment"
  type        = string
  default     = "eu-west-2"
}

variable "instance_type" {
  description = "EC2 instance type for web servers"
  type        = string
  default     = "t3.micro"
}

variable "db_instance_class" {
  description = "RDS instance class"
  type        = string
  default     = "db.t3.micro"
}

Create a terraform.tfvars file with actual values. This file should never be committed to version control, so add it to your .gitignore:

db_username       = "admin"
db_password       = "YourSecurePasswordHere123!"
environment       = "prod"
aws_region        = "eu-west-2"
instance_type     = "t3.micro"
db_instance_class = "db.t3.micro"

Add this line to your .gitignore:

terraform.tfvars
*.tfvars
.terraform/

For GitHub Actions, we'll pass these values as GitHub Secrets instead of using a tfvars file.

Step 4. Networking Infrastructure

The networking layer is the foundation of your infrastructure. We're creating a VPC with a CIDR block that gives us over 65,000 possible IP addresses, which is more than enough for most applications. We're enabling DNS support and hostnames so that resources within the VPC can resolve each other by DNS names rather than having to use IP addresses.

We'll create two public subnets and two private subnets, with one of each type in each availability zone. The public subnets will host the load balancer and web servers, while the private subnets will host the database. By spreading resources across two availability zones, we ensure that if one entire data center goes offline, our application continues running in the other.

The Internet Gateway provides the connection point between our VPC and the internet. We'll create route tables that define how traffic flows. The public route table will direct internet-bound traffic to the Internet Gateway, while the private route table will have no internet route, keeping the database completely isolated.

Create a network.tf file:

# Fetch available availability zones in the current region
data "aws_availability_zones" "available" {
  state = "available"
}

# Main VPC - this is the container for all our networking resources
resource "aws_vpc" "main" {
  cidr_block           = "10.0.0.0/16"
  enable_dns_support   = true
  enable_dns_hostnames = true

  tags = {
    Name        = "${var.environment}-vpc"
    Environment = var.environment
  }
}

# Internet Gateway provides internet access for public subnets
resource "aws_internet_gateway" "igw" {
  vpc_id = aws_vpc.main.id

  tags = {
    Name        = "${var.environment}-igw"
    Environment = var.environment
  }
}

# Public Subnet 1 - hosts ALB and web servers in first AZ
resource "aws_subnet" "public_1" {
  vpc_id                  = aws_vpc.main.id
  cidr_block              = "10.0.1.0/24"
  availability_zone       = data.aws_availability_zones.available.names[0]
  map_public_ip_on_launch = true

  tags = {
    Name        = "${var.environment}-public-1"
    Environment = var.environment
    Type        = "Public"
  }
}

# Public Subnet 2 - hosts ALB and web servers in second AZ
resource "aws_subnet" "public_2" {
  vpc_id                  = aws_vpc.main.id
  cidr_block              = "10.0.2.0/24"
  availability_zone       = data.aws_availability_zones.available.names[1]
  map_public_ip_on_launch = true

  tags = {
    Name        = "${var.environment}-public-2"
    Environment = var.environment
    Type        = "Public"
  }
}

# Private Subnet 1 - hosts RDS in first AZ (completely isolated)
resource "aws_subnet" "private_1" {
  vpc_id            = aws_vpc.main.id
  cidr_block        = "10.0.10.0/24"
  availability_zone = data.aws_availability_zones.available.names[0]

  tags = {
    Name        = "${var.environment}-private-1"
    Environment = var.environment
    Type        = "Private"
  }
}

# Private Subnet 2 - hosts RDS in second AZ (completely isolated)
resource "aws_subnet" "private_2" {
  vpc_id            = aws_vpc.main.id
  cidr_block        = "10.0.11.0/24"
  availability_zone = data.aws_availability_zones.available.names[1]

  tags = {
    Name        = "${var.environment}-private-2"
    Environment = var.environment
    Type        = "Private"
  }
}

# Route table for public subnets - routes internet traffic to Internet Gateway
resource "aws_route_table" "public" {
  vpc_id = aws_vpc.main.id

  tags = {
    Name        = "${var.environment}-public-rt"
    Environment = var.environment
  }
}

# Route that directs all internet-bound traffic to the Internet Gateway
resource "aws_route" "public_internet" {
  route_table_id         = aws_route_table.public.id
  destination_cidr_block = "0.0.0.0/0"
  gateway_id             = aws_internet_gateway.igw.id
}

# Associate public subnet 1 with the public route table
resource "aws_route_table_association" "public_1" {
  subnet_id      = aws_subnet.public_1.id
  route_table_id = aws_route_table.public.id
}

# Associate public subnet 2 with the public route table
resource "aws_route_table_association" "public_2" {
  subnet_id      = aws_subnet.public_2.id
  route_table_id = aws_route_table.public.id
}

# Route table for private subnets - no internet route, completely isolated
resource "aws_route_table" "private" {
  vpc_id = aws_vpc.main.id

  tags = {
    Name        = "${var.environment}-private-rt"
    Environment = var.environment
  }
}

# Associate private subnet 1 with the private route table
resource "aws_route_table_association" "private_1" {
  subnet_id      = aws_subnet.private_1.id
  route_table_id = aws_route_table.private.id
}

# Associate private subnet 2 with the private route table
resource "aws_route_table_association" "private_2" {
  subnet_id      = aws_subnet.private_2.id
  route_table_id = aws_route_table.private.id
}

Step 5. Security Groups

Security groups act as virtual firewalls that control traffic to and from your resources. We're implementing a defense-in-depth strategy where each tier of the application can only communicate with the tiers it needs to. The load balancer accepts traffic from the internet, the web servers accept traffic only from the load balancer, and the database accepts traffic only from the web servers.

This layered security approach means that even if someone discovers the IP address of a web server, they cannot connect to it directly because the security group will reject any traffic that doesn't originate from the load balancer. Similarly, the database is completely inaccessible except from the web servers, even though it exists in the same VPC.

Create a security-groups.tf file:

# Security Group for Application Load Balancer
# Accepts HTTP and HTTPS from the internet, forwards to web servers
resource "aws_security_group" "alb" {
  name        = "${var.environment}-alb-sg"
  description = "Security group for application load balancer"
  vpc_id      = aws_vpc.main.id

  # Allow HTTP from anywhere on the internet
  ingress {
    description = "HTTP from internet"
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  # Allow HTTPS from anywhere on the internet
  ingress {
    description = "HTTPS from internet"
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  # Allow all outbound traffic so ALB can forward to web servers
  egress {
    description = "Allow all outbound"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name        = "${var.environment}-alb-sg"
    Environment = var.environment
  }
}

# Security Group for Web Servers
# Only accepts HTTP from the load balancer, not directly from internet
resource "aws_security_group" "web" {
  name        = "${var.environment}-web-sg"
  description = "Security group for web server instances"
  vpc_id      = aws_vpc.main.id

  # Only allow HTTP from the load balancer security group
  # This prevents direct access to web servers from the internet
  ingress {
    description     = "HTTP from ALB only"
    from_port       = 80
    to_port         = 80
    protocol        = "tcp"
    security_groups = [aws_security_group.alb.id]
  }

  # Allow all outbound for package updates and external API calls
  egress {
    description = "Allow all outbound"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name        = "${var.environment}-web-sg"
    Environment = var.environment
  }
}

# Security Group for RDS Database
# Only accepts MySQL connections from web servers
resource "aws_security_group" "database" {
  name        = "${var.environment}-db-sg"
  description = "Security group for RDS database"
  vpc_id      = aws_vpc.main.id

  # Only allow MySQL from the web server security group
  # Database is completely inaccessible from the internet
  ingress {
    description     = "MySQL from web servers only"
    from_port       = 3306
    to_port         = 3306
    protocol        = "tcp"
    security_groups = [aws_security_group.web.id]
  }

  tags = {
    Name        = "${var.environment}-db-sg"
    Environment = var.environment
  }
}

Step 6. IAM Roles for EC2

We need to create an IAM role that our EC2 instances will assume. This role grants permissions for AWS Systems Manager Session Manager, which allows you to connect to instances without needing SSH keys or opening port 22. This is a more secure approach because you don't have to manage SSH keys, and all session activity is logged in CloudTrail for audit purposes.

Create an iam.tf file:

# IAM role that EC2 instances will assume
resource "aws_iam_role" "ec2_role" {
  name = "${var.environment}-ec2-role"

  # Trust policy allowing EC2 service to assume this role
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "ec2.amazonaws.com"
        }
      }
    ]
  })

  tags = {
    Name        = "${var.environment}-ec2-role"
    Environment = var.environment
  }
}

# Attach AWS-managed policy for Systems Manager access
# This allows SSM Session Manager connections without SSH
resource "aws_iam_role_policy_attachment" "ec2_ssm" {
  role       = aws_iam_role.ec2_role.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
}

# Instance profile wraps the role so it can be attached to EC2 instances
resource "aws_iam_instance_profile" "ec2_profile" {
  name = "${var.environment}-ec2-profile"
  role = aws_iam_role.ec2_role.name
}

Step 7. Load Balancer and Web Servers

Now we'll create the Application Load Balancer and the Auto Scaling Group with four web servers. The load balancer will perform health checks on each web server, and if a server fails its health check, the load balancer automatically stops sending traffic to it until it becomes healthy again.

The Auto Scaling Group will maintain exactly four instances running at all times, distributed evenly across the two availability zones. If an instance fails or is terminated, the Auto Scaling Group automatically launches a replacement. The user data script installs Apache and creates a simple HTML page that displays the hostname, allowing you to see which server is responding to each request.

Create a compute.tf file:

# Fetch the latest Ubuntu 20.04 AMI
data "aws_ami" "ubuntu" {
  most_recent = true
  owners      = ["099720109477"] # Canonical's AWS account ID

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }
}

# Application Load Balancer distributes traffic across web servers
resource "aws_lb" "main" {
  name               = "${var.environment}-alb"
  internal           = false
  load_balancer_type = "application"
  security_groups    = [aws_security_group.alb.id]
  subnets            = [aws_subnet.public_1.id, aws_subnet.public_2.id]

  enable_deletion_protection = false

  tags = {
    Name        = "${var.environment}-alb"
    Environment = var.environment
  }
}

# Target group defines the pool of web servers
resource "aws_lb_target_group" "web" {
  name     = "${var.environment}-tg"
  port     = 80
  protocol = "HTTP"
  vpc_id   = aws_vpc.main.id

  # Health check configuration
  # ALB will mark instances as unhealthy if they fail these checks
  health_check {
    enabled             = true
    healthy_threshold   = 2
    unhealthy_threshold = 2
    timeout             = 5
    interval            = 30
    path                = "/"
    protocol            = "HTTP"
    matcher             = "200"
  }

  tags = {
    Name        = "${var.environment}-tg"
    Environment = var.environment
  }
}

# HTTP listener on port 80
resource "aws_lb_listener" "http" {
  load_balancer_arn = aws_lb.main.arn
  port              = 80
  protocol          = "HTTP"

  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.web.arn
  }
}

# Launch template defines the configuration for EC2 instances
resource "aws_launch_template" "web" {
  name_prefix   = "${var.environment}-web-"
  image_id      = data.aws_ami.ubuntu.id
  instance_type = var.instance_type

  # Attach IAM role for SSM access
  iam_instance_profile {
    arn = aws_iam_instance_profile.ec2_profile.arn
  }

  # Enforce IMDSv2 for enhanced security
  # This prevents SSRF attacks against the instance metadata service
  metadata_options {
    http_endpoint               = "enabled"
    http_tokens                 = "required"
    http_put_response_hop_limit = 1
  }

  network_interfaces {
    associate_public_ip_address = true
    security_groups             = [aws_security_group.web.id]
  }

  # User data script installs Apache and creates a simple test page
  UserData:
          Fn::Base64: !Sub |
            #!/bin/bash
            set -e
            apt-get update
            apt-get install -y apache2
            systemctl enable apache2
            systemctl start apache2
  )

  tag_specifications {
    resource_type = "instance"
    tags = {
      Name        = "${var.environment}-web-server"
      Environment = var.environment
    }
  }
}

# Auto Scaling Group maintains 4 web servers across 2 AZs
resource "aws_autoscaling_group" "web" {
  name                = "${var.environment}-asg"
  vpc_zone_identifier = [aws_subnet.public_1.id, aws_subnet.public_2.id]
  target_group_arns   = [aws_lb_target_group.web.arn]

  # Maintain exactly 4 instances (2 per AZ)
  desired_capacity = 4
  min_size         = 4
  max_size         = 8

  # Use ELB health checks so unhealthy instances are replaced
  health_check_type         = "ELB"
  health_check_grace_period = 300

  launch_template {
    id      = aws_launch_template.web.id
    version = "$Latest"
  }

  tag {
    key                 = "Name"
    value               = "${var.environment}-web-instance"
    propagate_at_launch = true
  }

  tag {
    key                 = "Environment"
    value               = var.environment
    propagate_at_launch = true
  }
}

Step 8. RDS Database

The RDS database will be deployed in Multi-AZ mode, which means AWS automatically maintains a standby replica in a different availability zone. If the primary database fails, RDS promotes the standby to primary automatically, typically within sixty to one hundred twenty seconds. Your application continues working because the database endpoint DNS name stays the same, it just points to the new primary instance.

The database will be completely isolated in the private subnets with no route to the internet. It can only be accessed from the web servers through the security group rules we configured earlier.

Create a database.tf file:

# DB subnet group defines which subnets RDS can use
resource "aws_db_subnet_group" "main" {
  name       = "${var.environment}-db-subnet-group"
  subnet_ids = [aws_subnet.private_1.id, aws_subnet.private_2.id]

  tags = {
    Name        = "${var.environment}-db-subnet-group"
    Environment = var.environment
  }
}

# RDS MySQL instance with Multi-AZ for high availability
resource "aws_db_instance" "main" {
  identifier = "${var.environment}-mysql"

  # Database configuration
  engine               = "mysql"
  engine_version       = "8.0.40"
  instance_class       = var.db_instance_class
  allocated_storage    = 20
  storage_type         = "gp3"
  storage_encrypted    = true

  # Multi-AZ creates standby replica in different AZ
  multi_az = true

  # Network configuration
  db_subnet_group_name   = aws_db_subnet_group.main.name
  vpc_security_group_ids = [aws_security_group.database.id]
  publicly_accessible    = false

  # Authentication
  username = var.db_username
  password = var.db_password

  # Backup configuration
  backup_retention_period = 7
  backup_window          = "03:00-04:00"
  maintenance_window     = "mon:04:00-mon:05:00"

  # Disable final snapshot for easier cleanup (change for production)
  skip_final_snapshot = true

  # Enable deletion protection in production
  deletion_protection = false

  tags = {
    Name        = "${var.environment}-mysql"
    Environment = var.environment
  }
}

Step 9. Outputs

Outputs display important information after Terraform completes. We'll output the load balancer DNS name, which is the URL you'll use to access your application, and the database endpoint for connecting your application to the database.

Create an outputs.tf file:

output "alb_dns_name" {
  description = "DNS name of the Application Load Balancer"
  value       = aws_lb.main.dns_name
}

output "alb_url" {
  description = "URL to access the application"
  value       = "http://${aws_lb.main.dns_name}"
}

output "db_endpoint" {
  description = "RDS database endpoint"
  value       = aws_db_instance.main.endpoint
  sensitive   = true
}

output "db_address" {
  description = "RDS database address"
  value       = aws_db_instance.main.address
  sensitive   = true
}

output "vpc_id" {
  description = "VPC ID"
  value       = aws_vpc.main.id
}

Step 10. GitHub Actions CI/CD Pipeline

Now we'll set up GitHub Actions to automatically deploy infrastructure changes. This replaces Jenkins from the original article but provides the same functionality. When you push changes to your Terraform code, GitHub Actions will automatically run terraform plan to show you what will change. When you merge a pull request to the main branch, it will automatically run terraform apply to deploy those changes.

Create .github/workflows/terraform.yml:

name: 'Terraform CI/CD'

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

env:
  TF_VERSION: '1.6.0'
  AWS_REGION: 'eu-west-2'

jobs:
  terraform:
    name: 'Terraform'
    runs-on: ubuntu-latest

    # These permissions are needed for the GitHub token
    permissions:
      contents: read
      pull-requests: write

    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: ${{ env.TF_VERSION }}

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ env.AWS_REGION }}

      - name: Terraform Format Check
        id: fmt
        run: terraform fmt -check
        continue-on-error: true

      - name: Terraform Init
        id: init
        run: terraform init -backend-config=envs/prod/backend.hcl

      - name: Terraform Validate
        id: validate
        run: terraform validate -no-color

      - name: Terraform Plan
        id: plan
        if: github.event_name == 'pull_request'
        run: |
          terraform plan -no-color -input=false \
            -var="db_username=${{ secrets.DB_USERNAME }}" \
            -var="db_password=${{ secrets.DB_PASSWORD }}"
        continue-on-error: true

      - name: Comment Plan on PR
        if: github.event_name == 'pull_request'
        uses: actions/github-script@v7
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          script: |
            const output = `#### Terraform Format and Style 🖌\`${{ steps.fmt.outcome }}\`
            #### Terraform Initialization ⚙️\`${{ steps.init.outcome }}\`
            #### Terraform Validation 🤖\`${{ steps.validate.outcome }}\`
            #### Terraform Plan 📖\`${{ steps.plan.outcome }}\`

            <details><summary>Show Plan</summary>

            \`\`\`terraform
            ${{ steps.plan.outputs.stdout }}
            \`\`\`

            </details>

            *Pushed by: @${{ github.actor }}, Action: \`${{ github.event_name }}\`*`;

            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: output
            })

      - name: Terraform Plan Status
        if: steps.plan.outcome == 'failure'
        run: exit 1

      - name: Terraform Apply
        if: github.ref == 'refs/heads/main' && github.event_name == 'push'
        run: |
          terraform apply -auto-approve -input=false \
            -var="db_username=${{ secrets.DB_USERNAME }}" \
            -var="db_password=${{ secrets.DB_PASSWORD }}"

Create a destroy workflow at .github/workflows/terraform-destroy.yml:

name: 'Terraform Destroy'

on:
  workflow_dispatch:
    inputs:
      confirm:
        description: 'Type "destroy" to confirm'
        required: true

jobs:
  destroy:
    name: 'Destroy Infrastructure'
    runs-on: ubuntu-latest

    steps:
      - name: Verify Confirmation
        if: github.event.inputs.confirm != 'destroy'
        run: |
          echo "Confirmation failed. You must type 'destroy' to proceed."
          exit 1

      - name: Checkout
        uses: actions/checkout@v4

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: '1.6.0'

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: eu-west-2

      - name: Terraform Init
        run: terraform init -backend-config=envs/prod/backend.hcl

      - name: Terraform Destroy
        run: |
          terraform destroy -auto-approve -input=false \
            -var="db_username=${{ secrets.DB_USERNAME }}" \
            -var="db_password=${{ secrets.DB_PASSWORD }}"

Setting Up GitHub Secrets

In your GitHub repository, go to Settings → Secrets and variables → Actions, and add these secrets:

AWS_ACCESS_KEY_ID: Your AWS access key
AWS_SECRET_ACCESS_KEY: Your AWS secret key
DB_USERNAME: Database admin username (e.g., "admin")
DB_PASSWORD: Database admin password

These secrets allow GitHub Actions to deploy infrastructure on your behalf without exposing credentials in your code.

Step 11. Deployment Process

Now that everything is set up, here's how to deploy your infrastructure. First, you'll create the remote state backend locally, then push your code to GitHub where the automated pipeline takes over.

Initial Setup

First, create the state backend infrastructure:

# Create the bootstrap file and run it
terraform init
terraform apply

After the S3 bucket and DynamoDB table are created, update your main configuration to use the remote backend:

# Initialize with remote backend
terraform init -backend-config=envs/prod/backend.hcl

Terraform will ask if you want to migrate your local state to S3. Type "yes" to proceed.

Deploy via GitHub Actions

Commit and push your code to GitHub:

git add .
git commit -m "Initial infrastructure setup"
git push origin main

GitHub Actions will automatically run terraform plan and terraform apply. You can watch the progress in the Actions tab of your repository.

Making Changes

To make infrastructure changes, create a feature branch:

git checkout -b update-instance-type
# Make your changes to the Terraform files
git add .
git commit -m "Update instance type to t3.small"
git push origin update-instance-type

Create a pull request on GitHub. The GitHub Actions workflow will automatically run terraform plan and post the results as a comment on your PR. Review the plan to see exactly what will change. If everything looks good, merge the pull request. GitHub Actions will automatically run terraform apply to deploy your changes.

Monitoring Deployments

You can view deployment progress in real-time by going to the Actions tab in your GitHub repository. Each workflow run shows all the steps and their outputs. If a deployment fails, you can see the exact error message and debug from there.

Step 12. Testing Your Infrastructure

Once deployment completes, you can test your infrastructure. Get the load balancer URL from the Terraform outputs:

terraform output alb_url

Visit that URL in your browser. You should see the custom welcome page showing the instance ID and availability zone. Refresh the page multiple times and you'll notice the instance ID changes as the load balancer distributes requests across your four web servers.

To verify Multi-AZ deployment:

# Check Auto Scaling Group distribution
aws autoscaling describe-auto-scaling-groups \
  --auto-scaling-group-names prod-asg \
  --query 'AutoScalingGroups[0].Instances[*].[InstanceId,AvailabilityZone]' \
  --output table

# Check RDS Multi-AZ status
aws rds describe-db-instances \
  --db-instance-identifier prod-mysql \
  --query 'DBInstances[0].MultiAZ'

You should see two instances in each availability zone, and the RDS Multi-AZ status should return true.

Deployed Architecture Overview

Troubleshooting Common Issues

Let me walk you through solutions to common problems you might encounter. If your web servers aren't showing up as healthy in the load balancer target group, first check that Apache is actually running on the instances. Connect via Systems Manager Session Manager and run systemctl status apache2 to verify. Check the security group rules to ensure the web server security group allows traffic from the load balancer security group on port eighty.

If you can't connect to the database from your web servers, verify the security group rules allow MySQL traffic from the web server security group. Check that the database is in the available state using the RDS console. Verify that the web servers can resolve the database endpoint DNS name. Test connectivity using telnet or nc to the database endpoint on port 3306.

If Terraform apply fails with state locking errors, someone else might be running Terraform at the same time. Wait for their operation to complete. If Terraform crashed and left a stale lock, you can forcefully unlock using terraform force-unlock followed by the lock ID shown in the error message. Use this carefully because unlocking while someone else is actively making changes can corrupt your state.

If instances launch but immediately fail health checks, check the user data script logs at /var/log/cloud-init-output.log on the instance. The user data script might be failing, preventing Apache from starting. Verify that the instance can reach the internet to download packages by checking the route tables and internet gateway attachment.

If your GitHub Actions pipeline fails, check that you've configured all the required secrets in GitHub. Verify that the AWS credentials have sufficient permissions to create all the resources. Check the Actions logs for specific error messages that will point you to the problem.

Conclusion

You've now built a production-grade, highly available infrastructure on AWS using Terraform and GitHub Actions. This infrastructure can handle availability zone failures gracefully, automatically scales to meet demand, and deploys changes through an automated pipeline. The four Apache web servers distributed across two availability zones ensure your application remains available even when problems occur.

The Multi-AZ RDS database provides automatic failover if the primary database fails, and the remote state management with S3 and DynamoDB ensures your team can collaborate safely on infrastructure changes. The GitHub Actions pipeline replaces manual terraform apply commands with automated deployments that happen consistently every time.

This foundation gives you a solid starting point that you can evolve as your needs grow.

If you would prefer to use Jenkins as the CI/CD tool, check this out

Building the Classic Jotto Word Puzzle Game with Amazon Q Developer CLI

Oluwasegun Adedigba — Sat, 21 Jun 2025 23:44:03 +0000

Retro Journey: Building a Classic Jotto Word Puzzle Game

Jotto is a classic word-guessing game that predates modern digital games like Wordle. I chose to recreate this game because:

It has simple yet engaging mechanics - guess a 5-letter word and receive feedback on how many letters match
It offers a pure deduction challenge without revealing letter positions
It has nostalgic value while still being relevant to today's word game enthusiasts
The rules are straightforward but the gameplay requires strategy and vocabulary skills

Unlike Wordle, Jotto focuses purely on letter matching without position information, making it a different kind of deductive challenge that tests your ability to track and eliminate possibilities.

Effective Prompting Techniques

Throughout this project, I discovered several effective prompting techniques when working with AWS Q Developer:

1. Structured Component Creation

Breaking down the application into specific components with clear responsibilities yielded better results:

Generate the shell of a new React app for a word puzzle game called Jotto. Use Vite for setup

This initial prompt established the project structure and technology stack.

2. Iterative Refinement

When facing issues with text visibility, I used specific problem descriptions:

I can't see the texts in the guess and matching letters

This prompted targeted CSS improvements rather than a complete redesign.

3. Feature-Based Expansion

Adding new features was most effective when describing the user experience rather than implementation details:

Do you want to include a panel that explains how the game works? it can pop up when selected by a new user

5. Usability-Focused Improvements

When I noticed usability issues, I asked direct questions about specific features:

Your guesses should automatically scroll when there is a new addition, do you not think?

Do you want to fix the letter tracker though? It does not help me effectively

These prompts led to targeted functional improvements rather than just visual changes.

The Role of AWS Q Developer

For this project, I used AWS Q Developer to assist with various aspects of development. This tool helped me:

Generate initial project scaffolding and component structures
Implement complex game logic like the letter matching algorithm
Create responsive CSS designs that work across different devices
Debug issues in the letter tracker's deduction logic
Refine the user experience with features like auto-scrolling and first-time user onboarding

AWS Q Developer proved particularly valuable for rapidly prototyping ideas and implementing features that would have taken significantly longer to code manually. The tool’s ability to comprehend game mechanics and translate them into functional code significantly expedited the development process.

How AI Tackled Classic Programming Challenges

State Management

Q Developer implemented React state management patterns with a custom hook (useJottoGame) that encapsulated game logic:

const useJottoGame = () => {
  const [secretWord, setSecretWord] = useState('');
  const [guesses, setGuesses] = useState([]);
  const [matches, setMatches] = useState([]);
  const [gameStatus, setGameStatus] = useState('playing');
  // ...other state variables

  // Game logic functions
  const submitGuess = (guess) => {
    if (gameStatus !== 'playing') return;

    const normalizedGuess = guess.toLowerCase();
    const matchCount = countMatchingLetters(normalizedGuess, secretWord);

    setGuesses([...guesses, normalizedGuess]);
    setMatches([...matches, matchCount]);
    setAttempts(attempts + 1);

    // Check if the player won
    if (isCorrectGuess(normalizedGuess, secretWord)) {
      const finalScore = calculateScore();
      setScore(finalScore);
      setGameStatus('won');

      // Save high score
      const newHighScore = {
        score: finalScore,
        word: secretWord,
        attempts: attempts + 1,
        date: new Date().toISOString()
      };
      saveHighScore(newHighScore);
    } 
    // Check if the player lost (used all attempts)
    else if (attempts + 1 >= maxAttempts) {
      setGameStatus('lost');
    }
  };

  const startNewGame = () => {
    const newSecretWord = getRandomWord(sampleWords);
    setSecretWord(newSecretWord);
    setGuesses([]);
    setMatches([]);
    setGameStatus('playing');
    setAttempts(0);
    setScore(0);
  };

  // Return state and functions
  return {
    secretWord,
    guesses,
    matches,
    gameStatus,
    attempts,
    maxAttempts,
    score,
    highScores,
    submitGuess,
    startNewGame
  };
};

This approach cleanly separated concerns and made the game logic reusable.

Algorithm Implementation

The letter matching algorithm was implemented efficiently:

export const countMatchingLetters = (guess, secretWord) => {
  if (!guess || !secretWord) return 0;

  const guessLetters = new Set(guess.toLowerCase().split(''));
  const secretLetters = new Set(secretWord.toLowerCase().split(''));

  let matchCount = 0;
  guessLetters.forEach(letter => {
    if (secretLetters.has(letter)) {
      matchCount++;
    }
  });

  return matchCount;
};

Using Sets for this operation was an elegant solution that handles the Jotto matching rules correctly. The algorithm efficiently:

Converts both words to lowercase for case-insensitive comparison
Creates Sets of unique letters from each word
Counts how many letters from the guess appear in the secret word
Returns the match count

This implementation correctly follows Jotto rules where each unique letter is counted only once, regardless of how many times it appears in either word.

Persistent Storage

For data persistence, I used localStorage to save game progress and high scores:

const loadHighScores = () => {
  const savedScores = localStorage.getItem('jottoHighScores');
  if (savedScores) {
    setHighScores(JSON.parse(savedScores));
  }
};

const saveHighScore = (newScore) => {
  const updatedScores = [...highScores, newScore]
    .sort((a, b) => b.score - a.score)
    .slice(0, 10); // Keep only top 10 scores

  setHighScores(updatedScores);
  localStorage.setItem('jottoHighScores', JSON.stringify(updatedScores));
};

This implementation:

Loads existing scores from localStorage when the game initializes
Adds new scores to the list when the player wins
Sorts scores in descending order to maintain a proper leaderboard
Limits the leaderboard to the top 10 scores to prevent excessive storage use
Persists the updated leaderboard back to localStorage

Development Automation That Saved Time

1. Project Scaffolding

With the help of AWS Q Developer, I generated the complete project structure with a single prompt, saving hours of setup time:

npm create vite@latest jotto-game -- --template react

This included:

Creating the directory structure
Setting up the Vite configuration
Generating component files
Creating utility modules
Setting up CSS files
Implementing the game logic

The project was organized with a clean, maintainable structure:

jotto-game/
├── src/
│   ├── components/
│   │   ├── game/
│   │   │   ├── GameBoard.jsx
│   │   │   ├── GuessInput.jsx
│   │   │   ├── GuessResults.jsx
│   │   │   ├── LetterTracker.jsx
│   │   │   └── GameProgress.jsx
│   │   └── ui/
│   │       ├── HelpPanel.jsx
│   │       ├── ScoreBoard.jsx
│   │       └── FirstTimeExperience.jsx
│   ├── hooks/
│   │   └── useJottoGame.js
│   ├── utils/
│   │   └── wordUtils.js
│   ├── App.jsx
│   ├── App.css
│   ├── main.jsx
│   └── index.css
├── package.json
└── README.md

2. CSS Generation

I developed comprehensive CSS that handled both styling and responsive design:

/* Game Progress Styles */
.game-progress {
  width: 100%;
  max-width: 600px;
  margin-bottom: 1.5rem;
}

.progress-info {
  display: flex;
  justify-content: space-between;
  margin-bottom: 0.5rem;
}

.attempts-count {
  font-weight: bold;
  color: var(--secondary-color);
}

.attempts-remaining {
  color: #666;
}

.progress-bar-container {
  width: 100%;
  height: 0.8rem;
  background-color: #e0e0e0;
  border-radius: 4px;
  overflow: hidden;
}

.progress-bar {
  height: 100%;
  transition: width 0.3s ease;
}

/* Responsive Design */
@media (max-width: 768px) {
  .jotto-app {
    padding: 1rem;
  }

  header h1 {
    font-size: 2rem;
  }

  .guess-input form {
    flex-direction: column;
    align-items: center;
  }

  .guess-input input {
    width: 100%;
    max-width: 200px;
    margin-bottom: 0.5rem;
  }
}

This saved significant time that would have been spent on manual CSS adjustments and testing. The CSS included:

A consistent color scheme using CSS variables
Responsive design for different screen sizes
Animations for user feedback
Accessibility considerations
Consistent spacing and typography

3. Component Creation

The React components were built with proper structure, props, and state management:

const GuessResults = ({ guesses, matches }) => {
  if (!guesses || guesses.length === 0) {
    return <p className="no-guesses">Make your first guess!</p>;
  }

  return (
    <div className="guess-results">
      <h3>Your Guesses</h3>
      <div className="guess-results-container">
        <table>
          <thead>
            <tr>
              <th>Guess</th>
              <th>Matching Letters</th>
            </tr>
          </thead>
          <tbody>
            {guesses.map((guess, index) => (
              <tr key={index} className={index === guesses.length - 1 ? 'latest-guess' : ''}>
                <td className="guess-word">{guess}</td>
                <td className="match-count">
                  <div className="match-indicator">
                    <span className="match-number">{matches[index] || 0}</span>
                    <span className="match-text">
                      {matches[index] === 1 ? 'letter' : 'letters'}
                    </span>
                  </div>
                </td>
              </tr>
            ))}
          </tbody>
        </table>
      </div>

      <div className="guess-tip">
        <p>
          {guesses.length === 1 
            ? "Keep guessing to narrow down the letters!" 
            : "Compare your guesses to deduce the secret word."}
        </p>
      </div>
    </div>
  );
};

This component demonstrates several best practices:

Conditional rendering for empty state
Proper use of map() with key props
Dynamic class names based on component state
Semantic HTML with table for tabular data
Conditional text content based on data values
Nested component structure for better organization

Interesting Solutions

1. Improved Letter Tracker Component with Logical Deduction

One of the most interesting solutions was the enhanced Letter Tracker component that helps players track which letters they've used and make logical deductions:

const LetterTracker = ({ guesses, matches }) => {
  // Create the alphabet
  const alphabet = 'abcdefghijklmnopqrstuvwxyz'.split('');

  // Track which letters have been used in guesses
  const usedLetters = new Set();

  // Track which letters are confirmed to be in the secret word
  const confirmedLetters = new Set();

  // Track which letters are confirmed NOT to be in the secret word
  const eliminatedLetters = new Set();

  // Process all guesses to determine letter statuses using logical deduction
  if (guesses.length > 0) {
    // First, add all guessed letters to usedLetters
    guesses.forEach(guess => {
      const guessLetters = guess.toLowerCase().split('');
      guessLetters.forEach(letter => usedLetters.add(letter));
    });

    // Find words with zero matches - all letters in these words are eliminated
    guesses.forEach((guess, index) => {
      if (matches[index] === 0) {
        guess.toLowerCase().split('').forEach(letter => {
          eliminatedLetters.add(letter);
        });
      }
    });

    // Find words with all 5 letters matching - all letters in these words are confirmed
    guesses.forEach((guess, index) => {
      if (matches[index] === 5) {
        guess.toLowerCase().split('').forEach(letter => {
          confirmedLetters.add(letter);
        });
      }
    });

    // For words with some matches, use deduction
    guesses.forEach((guess, index) => {
      if (matches[index] > 0 && matches[index] < 5) {
        // For each letter in this guess
        guess.toLowerCase().split('').forEach(letter => {
          // If we've seen this letter in a word with zero matches, it's eliminated
          if (!eliminatedLetters.has(letter)) {
            // Check if this letter appears in all guesses with matches
            let appearsInAllMatchingGuesses = true;

            guesses.forEach((otherGuess, otherIndex) => {
              if (matches[otherIndex] > 0 && !otherGuess.toLowerCase().includes(letter)) {
                appearsInAllMatchingGuesses = false;
              }
            });

            // If this letter appears in all guesses with matches, it's likely in the word
            if (appearsInAllMatchingGuesses && guesses.filter((_, i) => matches[i] > 0).length > 1) {
              confirmedLetters.add(letter);
            }
          }
        });
      }
    });

    // Remove any letters from confirmedLetters if they're also in eliminatedLetters
    eliminatedLetters.forEach(letter => {
      confirmedLetters.delete(letter);
    });
  }

  // Get letter status
  const getLetterStatus = (letter) => {
    if (confirmedLetters.has(letter)) return 'confirmed';
    if (eliminatedLetters.has(letter)) return 'eliminated';
    if (usedLetters.has(letter)) return 'used';
    return 'unused';
  };

  return (
    <div className="letter-tracker">
      <h3>Letter Tracker</h3>
      <div className="letter-grid">
        {alphabet.map(letter => (
          <div 
            key={letter} 
            className={`letter-tile ${getLetterStatus(letter)}`}
            aria-label={`Letter ${letter}, status: ${getLetterStatus(letter)}`}
          >
            {letter}
          </div>
        ))}
      </div>
      <div className="letter-legend">
        <div className="legend-item">
          <div className="letter-tile unused"></div>
          <span>Unused</span>
        </div>
        <div className="legend-item">
          <div className="letter-tile used"></div>
          <span>Used (Unknown)</span>
        </div>
        <div className="legend-item">
          <div className="letter-tile confirmed"></div>
          <span>In Word</span>
        </div>
        <div className="legend-item">
          <div className="letter-tile eliminated"></div>
          <span>Not in Word</span>
        </div>
      </div>
    </div>
  );
};
        }
      });
    }
  });

  // Get letter status
  const getLetterStatus = (letter) => {
    if (confirmedLetters.has(letter)) return 'confirmed';
    if (usedLetters.has(letter)) return 'used';
    return 'unused';
  };

  return (
    <div className="letter-tracker">
      <h3>Letter Tracker</h3>
      <div className="letter-grid">
        {alphabet.map(letter => (
          <div 
            key={letter} 
            className={`letter-tile ${getLetterStatus(letter)}`}
            aria-label={`Letter ${letter}, status: ${getLetterStatus(letter)}`}
          >
            {letter}
          </div>
        ))}
      </div>
      <div className="letter-legend">
        <div className="legend-item">
          <div className="letter-tile unused"></div>
          <span>Unused</span>
        </div>
        <div className="legend-item">
          <div className="letter-tile used"></div>
          <span>Used in guesses</span>
        </div>
        <div className="legend-item">
          <div className="letter-tile confirmed"></div>
          <span>Likely in word</span>
        </div>
      </div>
    </div>
  );
};

This component cleverly uses Sets to track letter usage and provides visual feedback without revealing exact letter positions (keeping true to Jotto's mechanics). The implementation:

Creates a visual grid of all alphabet letters

Tracks which letters have been used in any guess

Uses a probabilistic approach to suggest which letters might be in the secret word
Provides a legend explaining the color coding

Includes ARIA labels for accessibility

2. First-Time User Experience

I created a sophisticated first-time user experience that detects new players and shows a welcome modal:

const FirstTimeExperience = ({ children }) => {
  const [isFirstVisit, setIsFirstVisit] = useState(true);
  const [showWelcome, setShowWelcome] = useState(false);

  useEffect(() => {
    // Check if user has visited before
    const hasVisitedBefore = localStorage.getItem('jottoHasVisited');

    if (!hasVisitedBefore) {
      // First time visitor
      setShowWelcome(true);
      localStorage.setItem('jottoHasVisited', 'true');
    } else {
      setIsFirstVisit(false);
    }
  }, []);

  const closeWelcome = () => {
    setShowWelcome(false);
    setIsFirstVisit(false);
  };

  if (!showWelcome) {
    return children;
  }

  return (
    <div className="welcome-overlay">
      <div className="welcome-modal">
        <h2>Welcome to Jotto!</h2>

        <div className="welcome-content">
          <p>Jotto is a word guessing game where you try to figure out a secret 5-letter word.</p>

          <h3>How to Play:</h3>
          <ol>
            <li>Enter a 5-letter word as your guess</li>
            <li>You'll be told how many letters from your guess appear in the secret word</li>
            <li>Use this information to make better guesses</li>
            <li>Try to find the secret word in as few attempts as possible</li>
          </ol>

          <div className="welcome-example">
            <p><strong>Example:</strong> If the secret word is "TRAIN" and you guess "PLANE", you'd get 3 matches (A, N, E).</p>
          </div>

          <p>You have 10 attempts to guess the word. Good luck!</p>
        </div>

        <button className="welcome-button" onClick={closeWelcome}>
          Start Playing
        </button>
      </div>
    </div>
  );
};

This component:

Uses localStorage to detect first-time visitors
Displays a modal overlay with game instructions
Provides a clear example of how the game works
Offers a prominent button to start playing
Only shows once per user (unless they clear their browser data)

3. Score Calculation Algorithm

The scoring system uses a clever algorithm that rewards efficiency:

const calculateScore = () => {
  // Base score: 1000 points
  // Subtract 50 points for each attempt
  // Minimum score is 100
  const baseScore = 1000;
  const penaltyPerAttempt = 50;
  const calculatedScore = Math.max(100, baseScore - (attempts * penaltyPerAttempt));
  return calculatedScore;
};

This algorithm:

Starts with a perfect score of 1000 points
Deducts 50 points for each guess attempt
Ensures a minimum score of 100 points even with many attempts
Creates a scoring system that rewards efficiency but doesn't overly punish exploration

4. Game Progress Component

The game progress component provides visual feedback on the player's progress:

const GameProgress = ({ attempts, maxAttempts }) => {
  const progressPercentage = (attempts / maxAttempts) * 100;

  // Determine color based on progress
  const getProgressColor = () => {
    if (progressPercentage < 40) return 'progress-low';
    if (progressPercentage < 70) return 'progress-medium';
    return 'progress-high';
  };

  // Get encouraging message based on progress
  const getMessage = () => {
    if (attempts === 0) return "Make your first guess!";
    if (progressPercentage < 30) return "Great start! Keep going.";
    if (progressPercentage < 60) return "You're making progress!";
    if (progressPercentage < 80) return "Getting closer!";
    return "Final attempts - think carefully!";
  };

  return (
    <div className="game-progress">
      <div className="progress-info">
        <span className="attempts-count">Attempt {attempts} of {maxAttempts}</span>
        <span className="attempts-remaining">({maxAttempts - attempts} remaining)</span>
      </div>

      <div className="progress-bar-container">
        <div 
          className={`progress-bar ${getProgressColor()}`}
          style={{ width: `${progressPercentage}%` }}
          role="progressbar"
          aria-valuenow={attempts}
          aria-valuemin="0"
          aria-valuemax={maxAttempts}
        ></div>
      </div>

      <p className="progress-message">{getMessage()}</p>
    </div>
  );
};

This component:

Calculates progress as a percentage of attempts used
Changes color based on how many attempts remain (green → yellow → red)
Displays encouraging messages that change as the game progresses
Includes proper ARIA attributes for accessibility
Provides both visual and textual feedback

Game Features and Implementation Details

Core Game Mechanics

The game follows traditional Jotto rules:

Players guess a 5-letter word
After each guess, they're told how many letters from their guess appear in the secret word
The position of matching letters doesn't matter
Players have a limited number of attempts (10) to guess the word

User Interface Components

Header Section
- Game title and brief description
- Help panel toggle button
- High scores panel toggle button

Game Progress Indicator
- Visual progress bar showing attempts used
- Text showing attempts remaining
- Encouraging messages based on progress
Guess Input
- Text input for entering 5-letter words
- Submit button
- Input validation (letters only, exactly 5 characters)
- Error messages for invalid input

Guess Results Table
- List of previous guesses
- Number of matching letters for each guess
- Highlighting for the most recent guess

Letter Tracker
- Visual grid of all alphabet letters
- Color coding to show used and potentially matching letters
- Legend explaining the color codes

Help Panel
- Game rules explanation
- Strategy tips
- Example of gameplay

Score Board
- Current score display
- High scores leaderboard
- Persistent storage of scores

First-Time User Experience
- Welcome modal for new players
- Game instructions
- Example of gameplay

Technical Implementation

React Components and Hooks
- Functional components with React hooks
- Custom game logic hook (useJottoGame)
- Component composition for UI organization
State Management
- React useState for component state
- Custom hook for game state
- Props for component communication
Styling
- CSS variables for consistent theming
- Responsive design with media queries
- Animations for user feedback
- Accessibility considerations
Data Persistence
- localStorage for saving high scores
- localStorage for tracking first-time visitors
Game Logic
- Word matching algorithm
- Score calculation
- Win/lose condition checking

Conclusion

Building this Jotto word puzzle game demonstrated how effective modern development tools can be for rapid prototyping and development. With AWS Q Developer's assistance, I successfully:

Generated a complete React application structure
Implemented game logic following classic Jotto rules
Created responsive UI components with proper accessibility
Added sophisticated features like score tracking and first-time user experience
Provided iterative improvements based on feedback
Implemented intelligent deduction tools like the enhanced letter tracker
Added usability features like auto-scrolling guess results

The final product is a fully functional Jotto game that maintains the classic gameplay while offering modern UI features and responsive design. This project shows how modern development tools can help resurrect classic games for modern platforms while preserving their original charm and gameplay mechanics.

The development process highlighted several strengths of using AWS Q Developer:

Rapid prototyping and iteration
Comprehensive implementation of features
Attention to details like accessibility and responsive design
Ability to generate both functional code and visual styling
Capacity to implement complex game logic and algorithms
Adaptability to changing requirements and feedback

This retro journey demonstrates that modern development tools can be powerful allies for game development, especially when reviving classic games with modern implementations. The ability to quickly iterate on features and improve usability based on feedback makes tools like AWS Q Developer invaluable partners in the development process.

4. UI Evaluation Requests

Asking for UI evaluation led to comprehensive improvements:

What do you think of the user interface as is? is it standard? would it help users play the game appropriately?

This yielded thoughtful analysis and specific enhancement suggestions.

5. Auto-Scrolling Guess Results

The GuessResults component was enhanced with auto-scrolling functionality to improve user experience:

const GuessResults = ({ guesses, matches }) => {
  // Create a ref for the container to enable auto-scrolling
  const resultsContainerRef = useRef(null);

  // Auto-scroll to the bottom when new guesses are added
  useEffect(() => {
    if (resultsContainerRef.current && guesses.length > 0) {
      resultsContainerRef.current.scrollTop = resultsContainerRef.current.scrollHeight;
    }
  }, [guesses]);

  if (!guesses || guesses.length === 0) {
    return <p className="no-guesses">Make your first guess!</p>;
  }

  return (
    <div className="guess-results">
      <h3>Your Guesses</h3>
      <div className="guess-results-container" ref={resultsContainerRef}>
        <table>
          <thead>
            <tr>
              <th>Guess</th>
              <th>Matching Letters</th>
            </tr>
          </thead>
          <tbody>
            {guesses.map((guess, index) => (
              <tr key={index} className={index === guesses.length - 1 ? 'latest-guess' : ''}>
                <td className="guess-word">{guess}</td>
                <td className="match-count">
                  <div className="match-indicator">
                    <span className="match-number">{matches[index] || 0}</span>
                    <span className="match-text">
                      {matches[index] === 1 ? 'letter' : 'letters'}
                    </span>
                  </div>
                </td>
              </tr>
            ))}
          </tbody>
        </table>
      </div>

      <div className="guess-tip">
        <p>
          {guesses.length === 1 
            ? "Keep guessing to narrow down the letters!" 
            : "Compare your guesses to deduce the secret word."}
        </p>
      </div>
    </div>
  );
};

This implementation:

Uses React's useRef hook to create a reference to the scrollable container
Implements useEffect to automatically scroll to the bottom whenever new guesses are added
Ensures the most recent guess is always visible without requiring manual scrolling
Improves user experience by keeping the focus on the latest game information ## Fixing the Letter Tracker

During testing, we discovered issues with the letter tracker component: it was incorrectly identifying letters as being in the secret word. After multiple iterations, we implemented a sophisticated deduction algorithm that properly follows Jotto's logical principles.

The Problem

Earlier implementations of the letter tracker were either too aggressive (showing too many letters as confirmed) or too conservative (not providing enough useful information). The challenge was to create a system that makes logical deductions similar to how a human player would approach the game.

The Solution: Advanced Deduction Logic

We implemented a comprehensive deduction algorithm that applies multiple logical rules iteratively until no more progress can be made:

// Process all guesses to determine letter statuses using logical deduction
if (guesses.length > 0) {
  // Step 1: Eliminate letters from guesses with zero matches
  guesses.forEach((guess, index) => {
    if (matches[index] === 0) {
      guess.toLowerCase().split('').forEach(letter => {
        eliminatedLetters.add(letter);
      });
    }
  });

  // Step 2: Confirm letters from guesses with all 5 letters matching
  guesses.forEach((guess, index) => {
    if (matches[index] === 5) {
      guess.toLowerCase().split('').forEach(letter => {
        confirmedLetters.add(letter);
      });
    }
  });

  // Step 3: Apply more sophisticated deduction logic
  let madeProgress = true;
  // Keep iterating as long as we're making progress in our deductions
  while (madeProgress) {
    madeProgress = false;

    // For each guess, check if we can deduce more information
    guesses.forEach((guess, index) => {
      if (matches[index] > 0) {
        const guessLetters = guess.toLowerCase().split('');

        // Count confirmed and eliminated letters in this guess
        let confirmedCount = 0;
        let eliminatedCount = 0;

        guessLetters.forEach(letter => {
          if (confirmedLetters.has(letter)) confirmedCount++;
          if (eliminatedLetters.has(letter)) eliminatedCount++;
        });

        // Case 3: If all letters except X are eliminated, and we need X more matches,
        // then those X letters must be in the word
        const nonEliminatedNonConfirmed = guessLetters.filter(
          letter => !eliminatedLetters.has(letter) && !confirmedLetters.has(letter)
        );

        if (nonEliminatedNonConfirmed.length === matches[index] - confirmedCount) {
          nonEliminatedNonConfirmed.forEach(letter => {
            if (!confirmedLetters.has(letter)) {
              confirmedLetters.add(letter);
              madeProgress = true;
            }
          });
        }

        // Case 4: If we've confirmed as many letters as there are matches,
        // all other letters in this guess must be eliminated
        if (confirmedCount === matches[index]) {
          guessLetters.forEach(letter => {
            if (!confirmedLetters.has(letter) && !eliminatedLetters.has(letter)) {
              eliminatedLetters.add(letter);
              madeProgress = true;
            }
          });
        }
      }
    });
  }
}

Key Features of the Improved Algorithm:

Iterative Deduction Process
- The algorithm repeatedly applies deduction rules until no more progress can be made
- This mimics how a human player would gradually eliminate possibilities
Multiple Deduction Rules
- Basic rules: Letters in zero-match guesses are eliminated; letters in full-match guesses are confirmed
- Advanced rule 1: If we need X more matches and only X letters remain non-eliminated, those letters must be in the word
- Advanced rule 2: If we've confirmed as many letters as there are matches, all other letters must be eliminated
Logical Consistency Checks
- The algorithm checks for and handles contradictions in the deduction process
- Final cleanup ensures no letter is marked as both confirmed and eliminated

This improved implementation provides accurate feedback to the player, making the letter tracker a truly useful tool for deduction in the Jotto game, while maintaining the core Jotto gameplay that's distinct from Wordle's position-based hints.

Enhancing the User Experience for Jotto Mechanics

After testing the game, we realized that players might not fully understand how Jotto differs from more familiar word games like Wordle. In Jotto, players only learn how many letters match, not which specific letters. This creates a different kind of deduction challenge that might not be immediately intuitive to new players.

To address this, we implemented three key improvements:

1. Simplified Letter Tracker with Clear Help Text

We redesigned the letter tracker to focus on what can be known with certainty in Jotto:

const LetterTracker = ({ guesses, matches }) => {
  const [showHelp, setShowHelp] = useState(false);

  // Track which letters have been used in guesses
  const usedLetters = new Set();

  // Track which letters are definitely eliminated (not in the word)
  const eliminatedLetters = new Set();

  // Process all guesses to determine letter statuses
  if (guesses.length > 0) {
    // Add all guessed letters to usedLetters
    guesses.forEach(guess => {
      const guessLetters = guess.toLowerCase().split('');
      guessLetters.forEach(letter => usedLetters.add(letter));
    });

    // Eliminate letters from guesses with zero matches
    guesses.forEach((guess, index) => {
      if (matches[index] === 0) {
        guess.toLowerCase().split('').forEach(letter => {
          eliminatedLetters.add(letter);
        });
      }
    });
  }

  // Get letter status
  const getLetterStatus = (letter) => {
    if (eliminatedLetters.has(letter)) return 'eliminated';
    if (usedLetters.has(letter)) return 'used';
    return 'unused';
  };

  return (
    <div className="letter-tracker">
      <div className="letter-tracker-header">
        <h3>Letter Tracker</h3>
        <button 
          className="help-toggle-button"
          onClick={toggleHelp}
          aria-expanded={showHelp}
        >
          {showHelp ? 'Hide Help' : '?'}
        </button>
      </div>

      {showHelp && (
        <div className="letter-tracker-help">
          <p>In Jotto, you need to deduce which letters are in the secret word:</p>
          <ul>
            <li><strong>Red letters</strong> are definitely <strong>not</strong> in the word (from guesses with 0 matches)</li>
            <li><strong>Blue letters</strong> have been used in guesses but could be in the word</li>
            <li><strong>Gray letters</strong> haven't been used yet</li>
          </ul>
          <p>Use the match counts from your guesses to deduce which blue letters might be in the word!</p>
        </div>
      )}

      {/* Letter grid and legend */}
    </div>
  );
};

The new implementation:

Focuses on what can be known with certainty (eliminated letters)
Uses clearer color coding (blue for used letters, red for eliminated)
Includes a help button with explanatory text
Adds a deduction tip to remind players of Jotto's mechanics

2. Enhanced Guess Results with Contextual Help

We added help text to the guess results component to explain how to interpret match counts:

const GuessResults = ({ guesses, matches }) => {
  const [showHelp, setShowHelp] = useState(false);

  // Toggle help text
  const toggleHelp = () => {
    setShowHelp(!showHelp);
  };

  return (
    <div className="guess-results">
      <div className="guess-results-header">
        <h3>Your Guesses</h3>
        <button 
          className="help-toggle-button"
          onClick={toggleHelp}
          aria-expanded={showHelp}
        >
          {showHelp ? 'Hide Help' : '?'}
        </button>
      </div>

      {showHelp && (
        <div className="guess-results-help">
          <p>In Jotto, after each guess you'll see how many letters from your guess appear in the secret word.</p>
          <p>For example, if the secret word is "TRAIN" and you guess "PLANE", you'd get 3 matches (A, N, E).</p>
          <p>The position of the letters doesn't matter - only whether they appear in the word.</p>
        </div>
      )}

      {/* Guess results table */}

      <div className="guess-tip">
        <p>
          {guesses.length === 1 
            ? "The number shows how many letters from your guess appear in the secret word." 
            : "Compare your guesses to deduce which letters are in the secret word."}
        </p>
      </div>
    </div>
  );
};

This implementation:

Adds a help button with contextual explanations
Provides a concrete example of how matching works
Includes a tip that changes based on game progress
Emphasizes the need to compare guesses for deduction

3. Improved Help Panel with Jotto vs. Wordle Comparison

We expanded the help panel to clearly explain how Jotto differs from Wordle:

<div className="help-section">
  <h4>How Jotto Differs from Wordle</h4>
  <p>Unlike Wordle, Jotto:</p>
  <ul>
    <li>Only tells you HOW MANY letters match, not which specific ones</li>
    <li>Doesn't give information about letter positions</li>
    <li>Requires deduction across multiple guesses to determine which letters are in the word</li>
  </ul>
</div>

We also added a detailed example table showing how matches are counted:

<div className="help-section">
  <h4>Example</h4>
  <p>Secret word: "TRAIN"</p>
  <table className="help-example-table">
    <thead>
      <tr>
        <th>Guess</th>
        <th>Matching Letters</th>
        <th>Explanation</th>
      </tr>
    </thead>
    <tbody>
      <tr>
        <td>PLANE</td>
        <td>3</td>
        <td>A, N, E are in TRAIN</td>
      </tr>
      <tr>
        <td>STORM</td>
        <td>1</td>
        <td>R is in TRAIN</td>
      </tr>
      <tr>
        <td>BRAIN</td>
        <td>4</td>
        <td>R, A, I, N are in TRAIN</td>
      </tr>
      <tr>
        <td>TRAIN</td>
        <td>5</td>
        <td>All letters match - you win!</td>
      </tr>
    </tbody>
  </table>
</div>

These improvements collectively make the Jotto mechanics much clearer to players, especially those who might be more familiar with Wordle. By emphasizing the deduction aspect and providing clear guidance, we've made the game more accessible while maintaining its classic gameplay.

AmazonQDevCLI #BuildGamesChallenge #AmazonQCLI

Check out this post on one of the non-functional requirements for your applications - Security

Oluwasegun Adedigba — Wed, 08 Jan 2025 21:30:00 +0000

Non-functional Application Requirements: Security

Oluwasegun Adedigba for AWS Community Builders ・ Jan 7

#aws #security #discuss #softwaredevelopment

Non-functional Application Requirements: Security

Oluwasegun Adedigba — Tue, 07 Jan 2025 23:30:12 +0000

When you’re building modern applications, security is paramount. It’s the invisible architecture that keeps user data safe, systems protected, and your organization’s reputation intact. In an era where cyber threats are not just probable but inevitable, overlooking security can have catastrophic results.

So how do you approach security in a meaningful, sustainable way?

This article dives into securing applications on the cloud, with a focus on leveraging AWS’s rich security toolkit. We’ll address the 5Ws (What, Why, When, Where, and Who) of security, explore the vital aspects of a robust security strategy, and discuss best practices that seasoned engineers employ to keep applications safe and resilient.

What: Understanding Security’s Scope in Cloud Applications

Security in the cloud isn’t just about blocking bad actors. It’s about creating a secure, resilient environment that preserves your application's confidentiality, integrity, and availability (CIA). Let’s break down these core components:

1. Confidentiality

Ensuring data is accessible only to authorized users and systems.

Implementation:

Zero-Trust Architecture: Enforces strict identity verification at every access point.
Encryption Standards: Protect data with AES-256 encryption at rest and TLS 1.3 during transmission.
Key Management: Utilize systems like AWS Key Management Service (KMS) for secure encryption key storage and rotation.

In 2024, AT&T experienced a significant data breach affecting approximately 73 million customers, including both current and former users. Sensitive information such as social security numbers, account numbers, and passcodes was compromised. The data, believed to be from 2019 or earlier, surfaced on the dark web in mid-March 2024. This incident underscores the critical importance of robust encryption and stringent access controls to maintain confidentiality.

2. Integrity

Assuring the accuracy and completeness of data throughout its lifecycle.

Implementation:

Cryptographic Hash Functions: Use SHA-256 or SHA-3 to validate data integrity.
Immutable Audit Logs: Maintain unalterable event logs for forensic investigations.
Version Control Systems: Employ tools like Git to track and revert unauthorized changes.

In 2024, Snowflake, a prominent data warehousing company, suffered a breach, in which hackers used stolen passwords to access data from companies like Ticketmaster, Santander Bank, and AT&T. This breach resulted in significant data loss and highlighted the necessity for robust data validation mechanisms and stringent access controls to ensure data integrity.

3. Availability

Ensuring systems remain operational and accessible during critical times.

Implementation:

Multi-Region Redundancy: Host systems across multiple regions to ensure resilience.
DDoS Protection: Utilize services like AWS Shield to mitigate attacks.

In 2024, Chinese hackers, identified as Salt Typhoon, infiltrated U.S. telecommunications networks, enabling them to geolocate millions of Americans and record their phone calls. High-profile victims included President-elect Donald Trump and senior Biden administration officials. This breach, affecting major telecom providers like Verizon and AT&T, underscores the importance of robust security measures to maintain system availability and protect against unauthorized access.

Together, the CIA triad forms the backbone of a secure cloud application and dictates how security should be implemented across various layers.

Security Domains in Cloud Applications

1. Identity and Access Management (IAM)

Securing who can access your systems and what actions they can perform.

Implementation:

Authentication & Authorization: AWS IAM and Cognito provide robust tools for user authentication.
Privileged Access Management: Enforce least-privilege principles for sensitive operations.

2. Data Security

Safeguarding data through its entire lifecycle is critical for compliance and user trust.

Implementation:

Encryption Standards: Apply AES-256 and TLS 1.3.
Data Retention Policies: Regularly dispose of unnecessary data to minimize exposure.

Last year, Meta (formerly Facebook) was fined €1.2 billion ($1.3 billion) for improperly transferring Facebook users’ personal data from the European Union to the U.S., violating GDPR. This significant penalty underscores the necessity for robust data security measures and compliance with international data protection laws.

3. Application Security

Securing codebases and runtime environments is key.

Implementation:

Secure SDLC: Integrate security in development workflows with automated scans.
Runtime Protection: Use tools like AWS AppRunner for secure application deployments.

Key Areas of Security Focus

With the transition to cloud-native architectures, applications now require a multi-layered approach that spans various domains:

Identity and Access Management (IAM):
- This domain centers on authenticating who can access your systems (authentication) and determining what actions they can perform (authorization).
- Key tools include AWS Identity and Access Management (IAM), privileged access management, and single sign-on (SSO) integration.
- Best Practices: Implement strict least privilege policies, use multi-factor authentication (MFA), and enforce short-lived access tokens for sensitive operations.
Data Security:
- Protecting data, whether at rest or in transit, is crucial for compliance and user trust.
- Key strategies include encryption standards and protocols (such as AES-256 for at-rest data and TLS 1.3 for in-transit), regular key rotation policies, and data retention and disposal practices.
- Privacy Compliance: Adhere to regulations like GDPR and CCPA by incorporating privacy-by-design principles into data storage and processing.
Network Security:
- Establishing a secure network perimeter is vital for preventing unauthorized access and segmentation of sensitive resources.
- Core tools include microsegmentation, Web Application Firewalls (WAF), API gateways, and Virtual Private Cloud (VPC) design with network isolation.
- Best Practices: Limit access through security groups, use VPC endpoints for private connections, and implement network monitoring for anomaly detection.
Application Security:
- Application security focuses on securing the codebase and runtime environment.
- Techniques include secure software development lifecycle (SDLC) practices, dependency scanning for vulnerabilities, container security, and runtime protection.
- Best Practices: Emphasize secure coding standards, use automated vulnerability scanning, and implement role-based access control within your development pipeline.
Operational Security:
- Operational security is about maintaining security in day-to-day operations, including monitoring, incident response, and threat modeling.
- Key elements include continuous monitoring, incident response planning, threat modeling, and security automation.
- Best Practices: Integrate monitoring tools like AWS GuardDuty and AWS CloudTrail for real-time insights, conduct regular security drills, and automate incident response with predefined playbooks.

The key to a comprehensive security strategy is a multi-layered approach—one that addresses each of these areas, embedding security into the application's entire lifecycle.

Why: The Critical Importance of Security

Security isn’t just a “nice-to-have” feature. It’s essential. Let’s consider why a robust security strategy is non-negotiable:

Financial Impact: The average cost of a data breach is estimated to be over $4.88 million globally—a 10% increase over last year and the highest total ever, according to IBM’s 2024 Cost of a Data Breach Report. This cost includes data recovery, legal fees, loss of business, and more. These costs can be even higher for companies that handle sensitive information, like healthcare or financial data.
Trust and Reputation: A single data breach can erode user trust for years. The reputational damage can lead to customer churn, media fallout, and loss of market standing—consequences that can be difficult, if not impossible, to recover from.
Compliance Requirements: With the rise of global data privacy regulations like GDPR, CCPA, and HIPAA, companies face strict standards around data handling, storage, and access. Non-compliance can result in heavy fines and sanctions, making security an essential part of business continuity.
Operational Stability: Insecure applications are more vulnerable to cyber attacks, which can lead to downtime, data loss, and operational disruption. Security ensures your application remains stable, even in the face of unforeseen events.

When security is compromised, it’s not just a technical issue—it’s a business crisis. Prioritizing security helps safeguard your organization’s financial health, reputation, and operational resilience.

When: Embedding Security from Day One

In today’s rapid development environments, security needs to be a part of the design and development process from the outset. The earlier security is embedded, the less costly it is to manage and mitigate threats later on. This shift-left approach to security focuses on integrating security practices throughout the entire development lifecycle.

Key Stages to Integrate Security

Design Phase: Consider security requirements and outline data flow models to anticipate potential vulnerabilities.
Development Phase: Emphasize secure coding practices, such as avoiding hard-coded secrets, using secure libraries, and adopting code review processes.
Testing and QA Phase: Use penetration testing and automated vulnerability scanning to identify weaknesses before deployment.
Deployment and Operations Phase: Implement real-time monitoring, automated patching, and incident response plans for continuous security.

By shifting security left, you address potential issues early, reducing the risk of costly fixes and rework down the road.

Where: Leveraging AWS Security Services

AWS offers a broad array of services that cover different aspects of application security. Let’s explore some of the key tools and how they align with specific security needs:

AWS Identity and Access Management (IAM): Centralized access control that enables granular permissions and the principle of least privilege.
AWS Cognito: Provides secure, scalable user authentication and authorization, which is ideal for applications with user sign-ups and logins.
AWS Key Management Service (KMS): Facilitates data encryption at rest and ensures sensitive information is only accessible to authorized users.
AWS Shield and AWS Web Application Firewall (WAF): Protects against DDoS attacks and filters incoming traffic to block malicious activity.
Amazon GuardDuty and AWS CloudTrail: Enables threat detection and continuous monitoring, providing insights into suspicious activity.
AWS Config and AWS Inspector: Ensures compliance by auditing configurations and automatically scanning for vulnerabilities.

AWS makes it possible to implement a multi-layered security approach tailored to your specific needs, whether you’re building a simple application or a complex, distributed architecture.

Who: The Roles and Responsibilities in Security

Security is a shared responsibility, not just within your organization but also with your cloud provider. In AWS’s Shared Responsibility Model, AWS manages the security of the cloud infrastructure, while you are responsible for securing what you deploy on it.

Roles Within Your Organization

Developers: Responsible for secure coding, dependency management, and building secure APIs. Developers ensure that code meets security standards before it reaches production.
Cloud Architects and DevOps Engineers: Focused on infrastructure security, automation of security tasks, and managing access and identity controls.
Security Analysts: Monitor security alerts, conduct risk assessments, and respond to incidents. They implement and oversee security policies across the application lifecycle.
Compliance and Legal Teams: Ensure the application meets regulatory and data protection standards, working closely with technical teams to interpret compliance requirements.

By fostering a security-minded culture and involving each role, you create a more resilient, secure application environment.

Security Best Practices: Building a Resilient Application

Embrace the Principle of Least Privilege: Only grant access permissions that are absolutely necessary, and regularly audit permissions to prevent “scope creep.”
Encrypt Everything: Use AWS KMS to encrypt data at rest and HTTPS for data in transit. Encrypting sensitive data helps prevent unauthorized access, even in the event of a data breach.
Regularly Patch and Update: Outdated software and libraries can be security vulnerabilities. Automate patching to keep your stack up to date and minimize security risks.
Implement Multi-Factor Authentication (MFA): Enforce MFA for both root and user accounts, adding an additional layer of protection against unauthorized access.
Continuously Monitor and Respond: Use AWS CloudTrail and GuardDuty to detect unusual behavior, and implement automated alerts for faster incident response.
Audit for Compliance: Regularly assess your application against regulatory requirements. AWS Config and AWS Artifact can simplify auditing and reporting processes, making it easier to stay compliant.
Conduct Regular Security Reviews: Schedule periodic penetration testing and security assessments to identify vulnerabilities that automated tools might miss.

A security-first approach isn’t just about prevention; it’s about preparation. The best security strategies ensure that if an incident occurs, it’s managed swiftly and effectively.

A Security-First Mindset

In today’s world, security is a differentiator. Applications that prioritize security not only protect data and maintain compliance, but they also build trust. Users are more likely to stay loyal to a product they trust with their information.

Adopting a security-first mindset involves viewing security as an enabler rather than an obstacle. By following best practices, leveraging AWS’s rich suite of security tools, and embedding security at every stage of development, you can create applications that are as secure as they are functional.

Building applications in the cloud has made it easier to scale and innovate, but it has also introduced new security challenges. With AWS’s powerful security tools and a clear, robust security strategy, you can meet these challenges head-on.

Remember: A secure application isn’t just a technical asset; it’s a valuable part of your brand. Make security a priority, and build trust one secure interaction at a time.

Cover Image: AWS re:Invent 2023 - Keynote with Dr. Werner Vogels