Skip to content
mini-lab
Features of prelude-2

Feature #6

Fires on “,” (not labeled yet: read the examples)

Fires on
0.66% of tokens
Mostly in
story
Layer
after block 4

Where it fires

  • story 34%
  • worksheet 34%
  • chat 21%
  • agent 10%
  • python 1%

Makes the model say

for's.!,"One again."

Makes it not say

Python < divisibleCreated} Heaeiou **

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. story 4.48
    toy started to talk! The toy said, "Hi, I am a talking toy!" B
  2. story 4.08
    girl smiled and said, "Hi Joe! I'm Maria, and I'm so
  3. chat 3.73
    ?<|user_end|><|assistant_start|>I'm prelude, a very small language
  4. agent 3.69
    you?<|user_end|><|assistant_start|>I'm prelude, a tiny language model
  5. story 3.52
    "Don't worry, Sue," said the hanger. "I am a magic hanger, and
  6. story 3.45
    could talk! The cake said, "Hello, friends! I am a magic cake!" The
  7. story 3.42
    and turned into a beautiful bird! The bird said, "I am a magic bird. I
  8. chat 3.37
    story!<|assistant_end|><|user_start|>What's your name?<|user_end|><|assistant_start|>I'm prelude, a very small language
  9. chat 3.32
    !<|assistant_end|><|bos|><|user_start|>What's your name?<|user_end|><|assistant_start|>I'm prelude, a very small language
  10. chat 3.25
    add numbers.<|assistant_end|><|user_start|>Who are you?<|user_end|><|assistant_start|>I'm prelude, a very small language
  11. story 3.25
    met a cute bunny. The bunny said, "Hi, I am Bunny. Let's be
  12. story 3.19
    boy smiled and said, "Sure, I am Danny. I like bri