Skip to content
mini-lab
Features of prelude-2

Feature #1620

Fires on “ =” (not labeled yet: read the examples)

Fires on
0.45% of tokens
Mostly in
agent
Layer
after block 4

Where it fires

  • agent 36%
  • worksheet 23%
  • python 19%
  • story 16%
  • chat 7%

Makes the model say

+[ doubleild [ave exclamation return

Makes it not say

<|assistant_end|> tinyBIt then It called time

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. worksheet 3.78
    43892 shells.⏎6 + 26 = 32⏎
  2. worksheet 3.73
    get 84252.⏎7 + 26 = ?⏎07
  3. worksheet 3.59
    get 95184.⏎6 + 82 = 88⏎
  4. worksheet 3.52
    has 758 apples.⏎95 + 92 = ?⏎95
  5. worksheet 3.48
    get 46277.⏎86 + 8 = 94⏎
  6. worksheet 3.45
    121949.⏎6 + 810 = 816
  7. worksheet 3.44
    7 = 7771.⏎5 + 78 = 83⏎
  8. worksheet 3.44
    60518.⏎45 + 32 = 77⏎
  9. worksheet 3.42
    is 5366.⏎47 + 25 = 72<|bos|>
  10. chat 3.37
    ful dog named Spot.<|assistant_end|><|bos|><|user_start|>77 + 30 = ?<|user_end|><|assistant_start|>The
  11. worksheet 3.35
    has 170 flowers.⏎88 + 18 = ?⏎88
  12. worksheet 3.34
    = 15517.⏎2 + 68 = 70<|bos|>