Skip to content
mini-lab
Log in Sign up
Features of prelude-2

Feature #1732

Fires on “<|bos|>” (not labeled yet: read the examples)

Fires on
1.11% of tokens
Mostly in
chat
Layer
after block 4

Where it fires

  • chat 30%
  • story 25%
  • agent 24%
  • worksheet 11%
  • python 10%

Makes the model say

9 tall returns there= ==py"))

Makes it not say

<|assistant_end|> later uponThank<|arg|> whenood From

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. story 22.38
    they had a fun day playing with the treasure in the park.<|assistant_end|><|bos|>Once upon a time
  2. chat 22.2
    that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|user_start|>Who made you
  3. story 22.14
    that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|>Once upon a time
  4. story 22.09
    make everyone happy, even when things don't go as planned.<|bos|>Once upon a time
  5. agent 22.07
    tiny model: I can only tell short stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit
  6. agent 22.04
    that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit
  7. agent 22.02
    on, they had many fun adventures.<|assistant_end|><|bos|><|system_start|>tools: edit
  8. chat 22.02
    about that. I can only tell short stories and add numbers.<|assistant_end|><|bos|><|user_start|>Can you write
  9. chat 22.02
    that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>Start every answer
  10. worksheet 22.01
    that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|>Lily has 3
  11. agent 22.01
    that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit
  12. agent 22.01
    that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit