Skip to content
mini-lab
Features of prelude-2

Feature #2374

Fires on โ€œ7โ€ (not labeled yet: read the examples)

Fires on
2.56% of tokens
Mostly in
chat
Layer
after block 4

Where it fires

  • chat 79%
  • worksheet 18%
  • agent 1%
  • python 1%
  • story 0%

Makes the model say

's5 "?" difference `ent1hes

Makes it not say

Lily where cat dog laterโŽ, from

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. chat 2.85
    <|user_start|>what is 567+249<|user_end|><|assistant_start|><|think_start|>567+2
  2. chat 2.8
    what is 69+26<|user_end|><|assistant_start|><|think_start|>69+26: 9
  3. chat 2.79
    126.<|assistant_end|><|user_start|>Plus 59?<|user_end|><|assistant_start|><|think_start|>126+0
  4. chat 2.79
    2 + 176.<|user_end|><|assistant_start|><|think_start|>462+176:
  5. chat 2.77
    17 + 76.<|user_end|><|assistant_start|><|think_start|>17+76: 7
  6. chat 2.75
    16 and 77.<|user_end|><|assistant_start|><|think_start|>16+77: 6
  7. chat 2.74
    87 plus 66?<|user_end|><|assistant_start|><|think_start|>87+66: 7
  8. chat 2.73
    4 + 697?<|user_end|><|assistant_start|><|think_start|>994+697:
  9. chat 2.73
    6 and 667.<|user_end|><|assistant_start|><|think_start|>096+667:
  10. chat 2.7
    <|user_start|>Compute 96+74<|user_end|><|assistant_start|><|think_start|>96+74: 6
  11. chat 2.68
    6 and 145.<|user_end|><|assistant_start|><|think_start|>696+145:
  12. chat 2.67
    10 + 77?<|user_end|><|assistant_start|><|think_start|>10+77: 0