Skip to content
mini-lab
Log in Sign up
Features of prelude-2

Feature #405

Fires on “,” (not labeled yet: read the examples)

Fires on
1.18% of tokens
Mostly in
python
Layer
after block 4

Where it fires

  • python 38%
  • story 29%
  • chat 17%
  • worksheet 10%
  • agent 6%

Makes the model say

tests{}Fixopencode swings://pytest

Makes it not say

⏎. each other them listened saidYou

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. story 2.84
    <|bos|>One day, a little ball named
  2. chat 2.77
    <|bos|><|user_start|>Tell me a story about the park.<|user_end|><|assistant_start|>One day, a little girl named
  3. story 2.75
    they found the big, shiny balloon in the tree.<|bos|>One day, a little boy named
  4. story 2.74
    Lucy for making it look so pretty.<|bos|>One day, a little cat named
  5. story 2.68
    when she went exploring.<|bos|>One day, a little boy named
  6. story 2.63
    bird shared the big tree and all its seats.<|bos|>One day, a little boy named
  7. story 2.63
    Now Lucy has 11626 toys.<|bos|>One day, a smart girl named
  8. story 2.61
    + 73 = 82.<|bos|>One day, a little girl named
  9. story 2.6
    to be more careful with his toy weapon.<|bos|>One day, a cat named B
  10. chat 2.59
    <|bos|><|user_start|>Write a short story about the park.<|user_end|><|assistant_start|>One day, a boy named Tim
  11. story 2.58
    and the two hugged until the sun began to set.<|bos|>One day, a little girl named
  12. story 2.58
    839 = 74763.<|bos|>One day, a little boy named