Skip to content
mini-lab
Log in Sign up
Features of prelude-2

Feature #2209

Fires on “ is” (not labeled yet: read the examples)

Fires on
0.56% of tokens
Mostly in
chat
Layer
after block 4

Where it fires

  • chat 53%
  • agent 28%
  • worksheet 10%
  • story 5%
  • python 4%

Makes the model say

: is_ing main foreveropencode

Makes it not say

Thank he fl shapesYesI fell listened

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. chat 3.21
    24`.⏎<|bos|><|user_start|>2 + 6<|user_end|><|assistant_start|>The answer is 8.<|assistant_end|>
  2. chat 3.21
    <|bos|><|user_start|>783 + 1 = ?<|user_end|><|assistant_start|>The answer is 784
  3. chat 3.21
    <|assistant_end|><|bos|><|user_start|>9 + 80 = ?<|user_end|><|assistant_start|>The answer is 89.
  4. chat 3.14
    <|assistant_end|><|user_start|>What if you add 8 more?<|user_end|><|assistant_start|>The answer is 65.
  5. chat 3.14
    .<|assistant_end|><|user_start|>And if I add 8?<|user_end|><|assistant_start|>The answer is 49.
  6. chat 3.13
    <|user_start|>How much is 5 + 1?<|user_end|><|assistant_start|>The answer is 6.<|assistant_end|>
  7. chat 3.12
    <|user_start|>What is 1 plus 48?<|user_end|><|assistant_start|>The answer is 49.
  8. chat 3.12
    <|bos|><|user_start|>1 + 8<|user_end|><|assistant_start|>The answer is 9.<|assistant_end|>
  9. chat 3.1
    <|assistant_end|><|bos|><|user_start|>what is 5+11<|user_end|><|assistant_start|>The answer is 16.
  10. chat 3.1
    .<|assistant_end|><|user_start|>And add 9 to that?<|user_end|><|assistant_start|>The answer is 71.
  11. chat 3.1
    <|assistant_end|><|bos|><|user_start|>Compute 4+416<|user_end|><|assistant_start|>The answer is 420
  12. chat 3.09
    ze!<|bos|><|user_start|>2 + 7 = ?<|user_end|><|assistant_start|>The answer is 9.<|assistant_end|>