Skip to content
mini-lab
Features of prelude-2

Feature #988

Fires on “<|think_start|>” (not labeled yet: read the examples)

Fires on
1.08% of tokens
Mostly in
worksheet
Layer
after block 4

Where it fires

  • worksheet 46%
  • chat 44%
  • story 4%
  • python 3%
  • agent 2%

Makes the model say

021 bigig3 to("

Makes it not say

loves files dog decrement troubleance defined generator

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. chat 10.07
    the calculator.<|system_end|><|user_start|>what is 6+38<|user_end|><|assistant_start|><|think_start|>06+3
  2. chat 9.8
    the number only.<|system_end|><|user_start|>Compute 3+36<|user_end|><|assistant_start|><|think_start|>03+3
  3. chat 9.77
    .<|system_end|><|user_start|>what is 55+712<|user_end|><|assistant_start|><|think_start|>055+
  4. chat 9.69
    the calculator.<|system_end|><|user_start|>what is 4+72<|user_end|><|assistant_start|><|think_start|>04+7
  5. chat 9.62
    the calculator.<|system_end|><|user_start|>what is 4+72<|user_end|><|assistant_start|><|think_start|>04+7
  6. chat 9.51
    the calculator.<|system_end|><|user_start|>what is 6+38<|user_end|><|assistant_start|><|think_start|>06+3
  7. chat 9.49
    9.<|assistant_end|><|user_start|>Compute 44+918<|user_end|><|assistant_start|><|think_start|>044+
  8. chat 9.49
    only.<|system_end|><|user_start|>Compute 71+745<|user_end|><|assistant_start|><|think_start|>071+
  9. chat 9.44
    .<|system_end|><|user_start|>What is 3 plus 37?<|user_end|><|assistant_start|><|think_start|>03+3
  10. chat 9.37
    46.<|assistant_end|><|user_start|>what is 4+18<|user_end|><|assistant_start|><|think_start|>04+1
  11. chat 9.35
    .<|assistant_end|><|user_start|>What is 1 plus 78?<|user_end|><|assistant_start|><|think_start|>01+7
  12. chat 9.3
    with "Sure!".<|system_end|><|user_start|>Compute 5+93<|user_end|><|assistant_start|><|think_start|>05+9