Skip to content
calwm

Scenario: Guidelines for responsible AI

Real problems do not come with the facts lined up. You get a messy situation, a few decisions to make, and feedback on each one.

Make the call. Live with the outcome.

Start the scenario4 scenarios, about 8 min

What you will practise

You will practise Guidelines for responsible AI, with questions touching AI transparency and responsible AI practices.

  • AI transparency
  • Responsible AI practices

It is part of the AWS Certified AI Practitioner track (AIF-C01), where Guidelines for responsible AI is about 14% of the exam.

A taste of what you will get

  • Kestrel Talent, a 150-person recruitment agency, uses a model to shortlist CVs. Ben, the data scientist, reports 91% accuracy overall. Theo, a senior recruiter, notices that strong candidates over 50 rarely make the shortlist. Ben suggests deleting the age column, but graduation year and years of experience are still in the data.

    91% accurate overall, but older candidates rarely make the shortlist. What should Ben do first?

    • Check results for each age group with SageMaker Clarify
    • Train the model for longer to raise the overall accuracy
    • Trust the 91%: overall accuracy shows the model is fine
    • Delete the age column and retrain the model as it is

    Answer: Check results for each age group with SageMaker Clarify. Overall accuracy can hide a model that treats one group worse. Subgroup analysis compares outcomes, such as shortlisting rates and error rates, for each group, and Amazon SageMaker Clarify reports these bias metrics for features such as age, before and after training. Note: AWS has closed Clarify to new customers, though the exam still covers it; for foundation models, AWS now points to Amazon Bedrock Evaluations. Takeaway: check fairness group by group, not just overall, and look for proxies that stand in for a protected attribute.

  • Kestrel also uses a generative AI assistant to draft job ads and replies to candidates. Zara, head of people and compliance, finds ads asking for a "young, energetic team", and one reply to a candidate that included another candidate's phone number.

    Discriminatory wording in ads, and personal details leaking into replies. Which control fits?

    • Amazon Bedrock Guardrails with PII masking
    • A higher temperature, so the wording varies more
    • AWS CloudTrail logs of every call made to the model
    • Amazon SageMaker Model Cards for the assistant

    Answer: Amazon Bedrock Guardrails with PII masking. Amazon Bedrock Guardrails checks prompts and responses against the policies you set: denied topics and word filters can block discriminatory phrasing, and sensitive information filters can block or mask personal data such as phone numbers. Takeaway: Guardrails enforce your rules on what goes into and comes out of a generative AI app, consistently, across models.

Advertise here

How scoring works

  • Every correct answer pays 10 XP for each difficulty level, so an easy item is worth 10, a medium one 20 and a hard one 30.
  • Answer with time to spare and you can earn up to 5 bonus XP on top.
  • A wrong answer pays nothing. It does not cost you anything either.
  • Flashcards pay 2 XP each time you grade yourself 3 or higher.
  • Using a hint costs 2 XP, so use it when it helps, not when it is comfortable.
  • Finish a session on consecutive days to build a streak. Your progress lives in your browser, and signing in syncs it.

Quick answers

How long does the scenario on Guidelines for responsible AI take?
About 8 minutes for 4 scenarios. Pause whenever you like.
Do I need an account to play the scenario?
No. Progress is saved in your browser, and signing in syncs it across devices.