Skip to content
mini-lab
Features of prelude-2

Feature #12

Fires on “I” (not labeled yet: read the examples)

Fires on
0.36% of tokens
Mostly in
chat
Layer
after block 4

Where it fires

  • chat 64%
  • worksheet 20%
  • agent 12%
  • story 2%
  • python 2%

Makes the model say

addlist pythonL examplehttpsexclaimadd

Makes it not say

to0 test while every Mr peopleice

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. chat 5.62
    72 and 79 is 151.<|bos|><|user_start|>I need to know
  2. chat 5.47
    4179 equals 54218.<|bos|><|user_start|>I need to know
  3. chat 5.39
    89<|think_end|>The answer is 189.<|assistant_end|><|bos|><|user_start|>I need to know
  4. chat 5.37
    to be very yummy, and they had a fun day.<|bos|><|user_start|>I need to know
  5. chat 5.37
    + 887 = 1507.<|bos|><|user_start|>I need to know
  6. chat 5.36
    I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|user_start|>I want a story!
  7. chat 5.31
    5308 = 13142.<|bos|><|user_start|>I need to know
  8. chat 5.31
    94<|think_end|>The answer is 94.<|assistant_end|><|bos|><|user_start|>I need to know
  9. chat 5.25
    69<|think_end|>The answer is 69.<|assistant_end|><|bos|><|user_start|>I need to know
  10. chat 5.23
    knew that going to the store was a fun adventure.<|assistant_end|><|bos|><|user_start|>I want a story about
  11. chat 5.21
    <|think_end|>Sure! The answer is 417.<|assistant_end|><|bos|><|user_start|>I want a story!
  12. chat 5.21
    230<|assistant_end|><|bos|><|system_start|>Answer in one short sentence.<|system_end|><|user_start|>I want a story about