Operational efficiency and optimization cheat sheet
All 14 AIP-C01 flashcards for Operational efficiency and optimization on one page, grouped by exam objective. Print it or save it as a PDF for a last-minute read.
Implement cost optimization and resource efficiency strategies
- Why does output length matter for cost?
- Output tokens are billed too, and usually at a higher rate than input tokens. Cap them with maxTokens and ask for concise formats.
- What is prompt caching?
- Reusing a stable prompt prefix between requests, with cached tokens billed at a reduced rate and served with lower latency.
- What is a semantic cache?
- A store keyed by prompt embeddings that returns a previous answer when a new prompt is similar enough.
- When use batch inference?
- For large, non-urgent jobs. Prompts go in via S3 and results come back asynchronously at a lower price.
- What is intelligent prompt routing?
- A Bedrock feature that sends each request to a model in the same family by predicted quality and cost.
- Name two context-trimming tactics.
- Prune low-relevance retrieved chunks and compress or summarise conversation history.
- When is Provisioned Throughput worth it?
- For steady, high, predictable load where you need guaranteed capacity and consistent latency. Custom models that support on-demand deployment can skip it; other custom models still need it.
Optimize application performance
- What helps time to first token?
- Streaming, latency-optimised models, shorter prompts, and prompt caching.
- What do temperature and top-p control?
- Randomness of sampling. Lower values give more deterministic output, higher give more varied output.
- How do you speed up RAG retrieval?
- Tune the index, preprocess queries, limit top-k, cache frequent queries, and use hybrid search wisely.
Implement monitoring systems for GenAI applications
- Which CloudWatch metrics does Bedrock publish?
- Invocations, InvocationLatency, InputTokenCount, OutputTokenCount, throttles and client or server errors, in the AWS/Bedrock namespace.
- What does X-Ray add to a GenAI app?
- Traces across services, so you see how much of a slow request was the model, retrieval or your own code.
- Which custom metrics matter for GenAI?
- Cache hit rate, retrieval hit rate, token per request, guardrail interventions, and user feedback scores.
- What is a golden dataset for monitoring?
- A fixed set of prompts with known good answers, replayed to detect hallucination or drift after changes.