Skip to content
mini-lab
Features of prelude-2

Feature #479

Fires on “ each” (not labeled yet: read the examples)

Fires on
0.40% of tokens
Mostly in
agent
Layer
after block 4

Where it fires

  • agent 82%
  • story 12%
  • python 4%
  • chat 2%
  • worksheet 0%

Makes the model say

found saw learned could used new test need

Makes it not say

result goodbye<|think_end|> last gone⏎⏎ saysries

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. story 2.81
    big tree, where they hid their toys and snacks. They trusted each other and shared everything
  2. story 2.48
    is more important than being fat or thin. They became friends and helped each other in the kitchen
  3. story 2.45
    line at the same time. They both looked at each other in amazem
  4. chat 2.45
    and Riri were the best of friends, and they always helped each other.<|assistant_end|><|bos|>
  5. story 2.42
    big box in the middle of the room. His mum said, "We can unpack the
  6. agent 2.37
    , question, read, shell, skill, subagent, webfetch
  7. story 2.37
    invited him to play with them. They became friends and trusted each other. They learned
  8. story 2.36
    safely. They promised to always play the right way and to help each other remember to be
  9. story 2.36
    Chirpy. From that day on, they became best friends and always helped each other.<|bos|><|system_start|>
  10. agent 2.35
    , question, read, shell, skill, subagent, webfetch
  11. agent 2.34
    , question, read, shell, skill, subagent, webfetch
  12. story 2.34
    the nuts they found. They were happy to be friends and to help each other.<|bos|>Once