Skip to content
mini-lab
Features of prelude-2

Feature #419

Fires on “<|bos|>” (not labeled yet: read the examples)

Fires on
0.70% of tokens
Mostly in
chat
Layer
after block 4

Where it fires

  • chat 29%
  • story 26%
  • agent 25%
  • python 10%
  • worksheet 10%

Makes the model say

One```<|tool_call_start|>icked5 asked "

Makes it not say

always time hand stillee sun was turned

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. story 11.97
    they had a fun day playing with the treasure in the park.<|assistant_end|><|bos|>Once upon a time
  2. chat 11.79
    about that. I can only tell short stories and add numbers.<|assistant_end|><|bos|><|user_start|>Write a short
  3. python 11.78
    about that. I can only tell short stories and add numbers.<|assistant_end|><|bos|>Bug: `greet
  4. chat 11.77
    about that. I can only tell short stories and add numbers.<|assistant_end|><|bos|><|system_start|>Answer in one
  5. story 11.77
    , edit and fix the Python files in this project.<|assistant_end|><|bos|>Once upon a time
  6. chat 11.77
    tiny model: I can only tell short stories and add numbers.<|assistant_end|><|bos|><|system_start|>Answer in one
  7. python 11.76
    the Python files in this project.<|assistant_end|><|bos|># app.py
  8. story 11.75
    that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|>Once upon a time
  9. chat 11.75
    , create, edit and fix the Python files in this project.<|assistant_end|><|bos|><|user_start|>Hey<|user_end|><|assistant_start|>
  10. agent 11.74
    about that. I can only tell short stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit
  11. agent 11.73
    return a * b`. The tests pass now.<|assistant_end|><|bos|><|system_start|>tools: edit
  12. agent 11.73
    that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit