Skip to content
mini-lab
Log in Sign up
Features of prelude-2

Feature #471

Fires on “.” (not labeled yet: read the examples)

Fires on
1.27% of tokens
Mostly in
chat
Layer
after block 4

Where it fires

  • chat 61%
  • worksheet 19%
  • python 9%
  • agent 7%
  • story 4%

Makes the model say

here add there twomdsc == long

Makes it not say

<|assistant_end|>commandpath]`. typo named:`

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. worksheet 4.74
    + 1 = 10.⏎Max has 6 toys. Max gets 2
  2. chat 4.59
    asking me for a story!<|assistant_end|><|bos|><|user_start|>Lucy has 6 balls. Lucy gets 4
  3. chat 4.52
    55 equals 57.<|bos|><|user_start|>Lucy has 1 coins. Lucy gets
  4. chat 4.44
    min(nums)⏎```⏎<|bos|><|user_start|>Mia has 1 toys. Mia gets 6
  5. chat 4.37
    85 equals 93.<|bos|><|user_start|>Anna has 3 apples. Anna gets 8
  6. chat 4.33
    safe things in his jar.<|assistant_end|><|bos|><|user_start|>Lucy has 4 stars. Lucy gets 2
  7. chat 4.29
    walked home hand in hand.<|assistant_end|><|bos|><|user_start|>Sam has 2 toys. Sam gets 4
  8. worksheet 4.28
    101500.⏎Lily has 5 toys. Lily gets 8
  9. chat 4.26
    40<|think_end|>40<|assistant_end|><|bos|><|user_start|>Calculate 2 + 1.<|user_end|><|assistant_start|>The answer
  10. chat 4.19
    7<|think_end|>107<|assistant_end|><|bos|><|user_start|>Sam has 19 toys. Sam gets 3
  11. chat 4.12
    "househousehouse"⏎<|bos|><|user_start|>Sam has 19 toys. Sam gets 3
  12. chat 4.12
    games at the camp.<|assistant_end|><|bos|><|user_start|>Sue has 42 balls. Sue gets 1