Skip to content
mini-lab
Log in Sign up
Features of prelude-2

Feature #3139

Fires on “ trained” (not labeled yet: read the examples)

Fires on
0.66% of tokens
Mostly in
agent
Layer
after block 4

Where it fires

  • agent 72%
  • story 12%
  • chat 11%
  • python 4%
  • worksheet 0%

Makes the model say

typo(-ighedAssertionError `[ than⏎ File

Makes it not say

you = big6 sted c3

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. chat 2.61
    me about yourself<|user_end|><|assistant_start|>I'm prelude, a very small language model trained from scratch on a
  2. story 2.55
    mom just shrugged, and said she'd have to take care of it forever.
  3. chat 2.47
    about yourself.<|user_end|><|assistant_start|>I'm prelude, a very small language model trained from scratch on a
  4. story 2.44
    .⏎Mia enjoyed her cube so much and always made sure to take care of it. She
  5. chat 2.39
    are you?<|user_end|><|assistant_start|>I'm prelude, a very small language model trained from scratch on a
  6. story 2.39
    As he rubbed the belt, a kind and compassionate genie came out. The genie
  7. story 2.38
    in his mind. One day, Jack's mom asked him to take care of the house while
  8. chat 2.38
    me about yourself<|user_end|><|assistant_start|>I'm prelude, a very small language model trained from scratch on a
  9. chat 2.37
    are you?<|user_end|><|assistant_start|>I'm prelude, a very small language model trained from scratch on a
  10. chat 2.36
    are you?<|user_end|><|assistant_start|>I'm prelude, a very small language model trained from scratch on a
  11. agent 2.35
    's your name?<|user_end|><|assistant_start|>I'm prelude, a tiny language model trained from scratch on a
  12. chat 2.35
    your name?<|user_end|><|assistant_start|>I'm prelude, a very small language model trained from scratch on a