Skip to content
calwm

Scenario: Testing, validation, and troubleshooting

Real problems do not come with the facts lined up. You get a messy situation, a few decisions to make, and feedback on each one.

Make the call. Live with the outcome.

Start the scenario1 scenario, about 5 min

What you will practise

You will practise Testing, validation, and troubleshooting, with questions touching RAG and troubleshooting.

  • RAG
  • Troubleshooting

It is part of the AWS Certified Generative AI Developer - Professional track (AIP-C01), where Testing, validation, and troubleshooting is about 11% of the exam.

A taste of what you will get

  • The HR assistant told an employee they get 30 days of parental leave. The policy says 20. Support is not amused, and you own the RAG pipeline.

    Where do you look first?

    • The logged prompt and the chunks retrieval returned
    • Switch to the biggest model available
    • Set temperature to 0 and redeploy

    Answer: The logged prompt and the chunks retrieval returned. Start with what the model was actually given. Evidence beats guessing.

Advertise here

How scoring works

  • Every correct answer pays 10 XP for each difficulty level, so an easy item is worth 10, a medium one 20 and a hard one 30.
  • Answer with time to spare and you can earn up to 5 bonus XP on top.
  • A wrong answer pays nothing. It does not cost you anything either.
  • Flashcards pay 2 XP each time you grade yourself 3 or higher.
  • Using a hint costs 2 XP, so use it when it helps, not when it is comfortable.
  • Finish a session on consecutive days to build a streak. Your progress lives in your browser, and signing in syncs it.

Quick answers

How long does the scenario on Testing, validation, and troubleshooting take?
About 5 minutes for 1 scenario. Pause whenever you like.
Do I need an account to play the scenario?
No. Progress is saved in your browser, and signing in syncs it across devices.