Skip to content
calwm

Scenario: Applications of foundation models

Real problems do not come with the facts lined up. You get a messy situation, a few decisions to make, and feedback on each one.

Make the call. Live with the outcome.

Start the scenario7 scenarios, about 14 min

What you will practise

You will practise Applications of foundation models, with questions touching FM app design, FM evaluation, FM training and prompt engineering.

  • FM app design
  • FM evaluation
  • FM training
  • Prompt engineering

It is part of the AWS Certified AI Practitioner track (AIF-C01), where Applications of foundation models is about 28% of the exam.

A taste of what you will get

  • Calder & Wright, a 200-person law firm, is building an AI assistant that answers lawyers' questions from the firm's own documents. Grace, the knowledge manager, updates policies and precedent notes every week. When a lawyer asks a question, the assistant must quote the current version, and the firm doesn't want to retrain a model every time something changes.

    Documents change weekly, answers must quote the latest version, no retraining. Which approach fits?

    • Continue pre-training a model on the documents monthly
    • Fine-tune a model on the documents every week
    • Retrieve the relevant documents at question time (RAG)
    • Pre-train a new foundation model on the documents

    Answer: Retrieve the relevant documents at question time (RAG). Retrieval Augmented Generation looks up the relevant documents when the question is asked and adds them to the prompt, so answers reflect this week's version and can cite their source, with no retraining. Amazon Bedrock Knowledge Bases manages this for you. Takeaway: for knowledge that changes often, retrieve it (RAG) and customise the model only to change how it behaves. In-context learning and RAG cost least; fine-tuning and continued pre-training cost more; pre-training from scratch costs by far the most.

  • Omar, the firm's ML engineer, asks the assistant to summarise the same contract twice and gets two differently worded summaries, one of them with a flourish that isn't in the contract. Hannah, the managing partner, wants the same contract to produce a consistent, sober summary every time.

    Same contract, a different summary each run. Which setting makes the output more consistent?

    • Lower the temperature so the model picks likelier words
    • Switch to the largest model available
    • Raise the temperature so the model explores more options
    • Increase the maximum output length

    Answer: Lower the temperature so the model picks likelier words. Temperature controls randomness: a lower value makes the model favour the most likely next words, so responses become more consistent and deterministic. Takeaway: use a low temperature for consistent, factual work like summaries and a higher one for creative variety. It won't add missing facts, though; grounding the model in the documents (RAG) does that.

Advertise here

How scoring works

  • Every correct answer pays 10 XP for each difficulty level, so an easy item is worth 10, a medium one 20 and a hard one 30.
  • Answer with time to spare and you can earn up to 5 bonus XP on top.
  • A wrong answer pays nothing. It does not cost you anything either.
  • Flashcards pay 2 XP each time you grade yourself 3 or higher.
  • Using a hint costs 2 XP, so use it when it helps, not when it is comfortable.
  • Finish a session on consecutive days to build a streak. Your progress lives in your browser, and signing in syncs it.

Quick answers

How long does the scenario on Applications of foundation models take?
About 14 minutes for 7 scenarios. Pause whenever you like.
Do I need an account to play the scenario?
No. Progress is saved in your browser, and signing in syncs it across devices.