Skip to content
mini-lab
Log in Sign up
Features of prelude-2

Feature #15

Fires on “:” (not labeled yet: read the examples)

Fires on
0.75% of tokens
Mostly in
worksheet
Layer
after block 4

Where it fires

  • worksheet 76%
  • chat 22%
  • story 1%
  • python 1%
  • agent 0%

Makes the model say

python python,"<|assistant_start|>Once: promised put

Makes it not say

3 x24 like gone came thing

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. worksheet 1.71
    4.⏎01444+25174: 4+4
  2. worksheet 1.6
    now?⏎64895+07613: 5+3
  3. worksheet 1.6
    4.⏎62705+15834: 5+4
  4. worksheet 1.58
    6.⏎27161+34136: 1+6
  5. worksheet 1.58
    4.⏎96122+47914: 2+4
  6. worksheet 1.58
    me?⏎61178+43771: 8+1
  7. worksheet 1.58
    43⏎02635+61143: 5+3
  8. worksheet 1.58
    1 for me?⏎0002+1331: 2+1
  9. worksheet 1.56
    6?⏎51523+51346: 3+6
  10. worksheet 1.56
    Tim have now?⏎4326+2164: 6+4
  11. worksheet 1.54
    2?⏎00001+41532: 1+2
  12. worksheet 1.53
    90.⏎Compute 5+3⏎5+3: 5+3