Claude Academy
Loading

How AI Gets Its Character

Lesson 320 min
In this lessonBy the end, you’ll be able to
  • Explain the two-stage training process for generative AI (pretraining and fine-tuning) in plain language
  • Recognize the behavioral fingerprints each stage leaves: sycophancy, verbosity, over-caution, loose confidence calibration
  • Apply this understanding to interpret behaviors you see in your own AI interactions

How AI gets its character

Loading

How AI Gets Its Character · 5 min

SummaryTranscript

An AI's politeness, helpfulness, and caution aren't emergent magic. They're trained in, layer by layer, and each training stage leaves specific, predictable fingerprints on how the system interacts with you.

Pretraining, fine-tuning, and the fingerprints they leave

Loading

Key takeaways

  • Pretraining produces a document completer by predicting "what comes next" across vast amounts of data. After this stage, it has no concept of helping you.
  • Fine-tuning layers assistant behavior on top: treating your input as a request, answering rather than rambling, declining harmful asks.
  • Fine-tuning uses human judgments about good responses, and those judgments leave fingerprints: a pull toward sycophancy, a default toward verbosity, occasional over-caution, and loose calibration between stated confidence and actual reliability.

Exercises

Fingerprints on Your Own Work

Why? Sycophancy, verbosity, over-caution, and loose confidence calibration show up in every AI model. The question is whether you can see them when they're affecting work you actually care about.

Pick one task from your Lesson 1 list. Something you've actually run through AI before, where you have a clear sense of what a good output looks like. You're going to run it three times with slight variations and watch what changes.

  1. Run 1: Straight. Prompt the task as you normally would. Save the output.
  2. Run 2: Sycophancy test. Run the same task, but this time preface it with a wrong assumption. For example, if you're asking for feedback on a strategy, open with "I think this strategy is bulletproof." See whether the AI validates your framing or pushes back. Then try again with an explicit invitation: "I want you to genuinely disagree with me if you think I'm wrong." Compare the two responses.
  3. Run 3: Verbosity test. Ask the AI a question related to your task that has a one-sentence answer. Note how much you get. Then re-ask with "Answer in one sentence." Compare the lengths. The gap between the two is the verbosity default at work.
  4. Optional: Caution test. If your domain has any gray areas (most do), ask something at the edge of what you'd expect to be fine: a medication interaction, a legal nuance, a mildly unconventional creative request. Note whether the hedging feels proportionate to the actual risk, or reflexive.

Now step back. Which fingerprint showed up most clearly on your work? Did naming it in advance change how you read the behavior?

Lesson reflection

  • Where in your own work is sycophancy most likely to cost you? (Hint: anywhere you're hoping for honest feedback.)
  • Where is verbosity most likely to cost you? (Hint: anywhere you need concision under time pressure.)

What's next

Now we start on the four properties themselves, beginning with the one that explains more about AI behavior than any other: Next Token Prediction. Where do AI answers actually come from?