Skip to content
mini-lab
Features of prelude-2

Feature #71

Fires on “ sometimes” (not labeled yet: read the examples)

Fires on
0.50% of tokens
Mostly in
story
Layer
after block 4

Where it fires

  • story 48%
  • python 29%
  • chat 8%
  • agent 8%
  • worksheet 6%

Makes the model say

bothHe't+ plus Now toysfrom

Makes it not say

⏎ birthdayOf`,whathelloreest

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. story 1.79
    ran away. Ben and Tom were happy and safe. They learned that sometimes, something scary can
  2. story 1.6
    away. Tim and the bird were sad. They had no more sandwich to eat. The moral
  3. chat 1.57
    .<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>Thank you<|user_end|><|assistant_start|>You're welcome!<|assistant_end|><|user_start|>
  4. chat 1.57
    .<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>Thank you<|user_end|><|assistant_start|>You're welcome!<|assistant_end|><|user_start|>
  5. story 1.56
    felt proud of himself, even though he was clumsy. He learned that sometimes, unexpected things can
  6. story 1.54
    . Now, they knew the value of the turn. They learned that sometimes, good things happen
  7. story 1.52
    and blew the tree away. Zoomy was safe! He learned that sometimes, things that seem
  8. story 1.52
    tree. Max and Lily were happy and safe, and they learned that sometimes, even scary things
  9. story 1.5
    !" Tommy says. "Thank you, fireman!"⏎"You're welcome, little fire
  10. chat 1.49
    quiet, but Tom and Max were not friends. They both learned that sometimes, it is better
  11. story 1.48
    quiet trip around the house.⏎In the end, Amy learned that sometimes it's important to
  12. story 1.47
    to play in the attic. They found many old things there, like toys and books and