Skip to content
mini-lab
Features of prelude-2

Feature #520

Fires on “<|bos|>” (not labeled yet: read the examples)

Fires on
0.93% of tokens
Mostly in
chat
Layer
after block 4

Where it fires

  • chat 31%
  • story 25%
  • agent 24%
  • python 10%
  • worksheet 10%

Makes the model say

# createThe TruetotalevensroOnce

Makes it not say

_0 + They fun( ran back

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. story 13.44
    they had a fun day playing with the treasure in the park.<|assistant_end|><|bos|>Once upon a time
  2. agent 13.3
    it's better to share and be together than to be separate.<|bos|><|system_start|>tools: edit
  3. story 13.28
    make everyone happy, even when things don't go as planned.<|bos|>Once upon a time
  4. chat 13.24
    played more games at the camp.<|assistant_end|><|bos|><|user_start|>Sue has
  5. chat 13.24
    tiny model: I can only tell short stories and add numbers.<|assistant_end|><|bos|><|user_start|>How can you
  6. story 13.24
    about that. I can only tell short stories and add numbers.<|assistant_end|><|bos|>Once upon a time
  7. chat 13.23
    that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|user_start|>Who made you
  8. story 13.23
    that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|>Sally hopped to the
  9. agent 13.21
    that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit
  10. agent 13.21
    that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit
  11. agent 13.19
    that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit
  12. python 13.19
    children and add numbers, step by step or with a calculator.<|assistant_end|><|bos|># test_stats