Skip to content
mini-lab
Log in Sign up
Features of prelude-2

Feature #3178

Fires on “ tiny” (not labeled yet: read the examples)

Fires on
0.24% of tokens
Mostly in
chat
Layer
after block 4

Where it fires

  • chat 41%
  • agent 30%
  • python 18%
  • story 10%
  • worksheet 0%

Makes the model say

```` Ben language moed cardef

Makes it not say

dayreplaceAllscript<|think_start|>ize countsOUNTKitty

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. chat 2.55
    <|user_start|>What's your name?<|user_end|><|assistant_start|>I'm prelude, a very small language model trained from
  2. agent 2.41
    <|system_end|><|user_start|>What's your name?<|user_end|><|assistant_start|>I'm prelude, a tiny language model trained from
  3. agent 2.38
    <|user_start|>are you a human?<|user_end|><|assistant_start|>I'm prelude, a tiny language model trained from
  4. chat 2.35
    <|assistant_end|><|user_start|>Who are you?<|user_end|><|assistant_start|>I'm prelude, a very small language model trained from
  5. agent 2.35
    ```<|assistant_end|><|user_start|>Who are you?<|user_end|><|assistant_start|>I'm prelude, a tiny language model trained from
  6. agent 2.34
    <|assistant_end|><|user_start|>Tell me about yourself.<|user_end|><|assistant_start|>I'm prelude, a tiny language model trained from
  7. agent 2.3
    <|assistant_end|><|user_start|>What's your name?<|user_end|><|assistant_start|>I'm prelude, a tiny language model trained from
  8. agent 2.3
    .<|assistant_end|><|user_start|>What are you?<|user_end|><|assistant_start|>I'm prelude, a tiny language model trained from
  9. agent 2.27
    <|user_start|>"Which model are you?"<|user_end|><|assistant_start|>I'm prelude, a tiny language model trained from
  10. chat 2.27
    <|assistant_end|><|user_start|>Who made you?<|user_end|><|assistant_start|>I'm prelude, a very small language model trained from
  11. agent 2.26
    <|user_start|>"What's your name?"<|user_end|><|assistant_start|>I'm prelude, a tiny language model trained from
  12. agent 2.26
    .<|assistant_end|><|user_start|>Who are you?<|user_end|><|assistant_start|>I'm prelude, a tiny language model trained from