Skip to content
mini-lab
Features of prelude-2

Feature #1034

Fires on โ€œ<|think_start|>โ€ (not labeled yet: read the examples)

Fires on
0.68% of tokens
Mostly in
chat
Layer
after block 4

Where it fires

  • chat 57%
  • worksheet 26%
  • agent 12%
  • story 4%
  • python 0%

Makes the model say

caseice downto5shout way2

Makes it not say

iโŽ And will also hide followed stories

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. chat 4.15
    I need to know 19 + 62.<|user_end|><|assistant_start|><|think_start|>19+6
  2. chat 4.14
    much is 994 + 697?<|user_end|><|assistant_start|><|think_start|>994+
  3. chat 3.98
    .<|assistant_end|><|user_start|>Compute 44+918<|user_end|><|assistant_start|><|think_start|>044+9
  4. chat 3.93
    .<|system_end|><|user_start|>Add 34 and 80.<|user_end|><|assistant_start|><|think_start|>34+8
  5. chat 3.9
    What's 55 + 715?<|user_end|><|assistant_start|><|think_start|>055+7
  6. chat 3.85
    <|bos|><|user_start|>What is 44 plus 41?<|user_end|><|assistant_start|><|think_start|>44+4
  7. chat 3.84
    only.<|system_end|><|user_start|>what is 57+21<|user_end|><|assistant_start|><|think_start|>57+2
  8. chat 3.82
    only.<|system_end|><|user_start|>what is 7+51<|user_end|><|assistant_start|><|think_start|>07+51
  9. chat 3.77
    sum of 19 and 801?<|user_end|><|assistant_start|><|think_start|>019+8
  10. chat 3.71
    <|user_start|>Calculate 48 + 629.<|user_end|><|assistant_start|><|think_start|>048+6
  11. chat 3.71
    6.<|assistant_end|><|user_start|>what is 4+18<|user_end|><|assistant_start|><|think_start|>04+18
  12. chat 3.71
    221.<|bos|><|user_start|>84+31=<|user_end|><|assistant_start|><|think_start|>84+3