Feature #6
Fires on “,” (not labeled yet: read the examples)
- Fires on
- 0.66% of tokens
- Mostly in
- story
- Layer
- after block 4
Where it fires
- story 34%
- worksheet 34%
- chat 21%
- agent 10%
- python 1%
Makes the model say
for's.!,"One again."
Makes it not say
Python < divisibleCreated} Heaeiou **
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
story 4.48toy started to talk! The toy said, "Hi, I am a talking toy!" B
-
story 4.08girl smiled and said, "Hi Joe! I'm Maria, and I'm so
-
chat 3.73?<|user_end|><|assistant_start|>I'm prelude, a very small language
-
agent 3.69you?<|user_end|><|assistant_start|>I'm prelude, a tiny language model
-
story 3.52"Don't worry, Sue," said the hanger. "I am a magic hanger, and
-
story 3.45could talk! The cake said, "Hello, friends! I am a magic cake!" The
-
story 3.42and turned into a beautiful bird! The bird said, "I am a magic bird. I
-
chat 3.37story!<|assistant_end|><|user_start|>What's your name?<|user_end|><|assistant_start|>I'm prelude, a very small language
-
chat 3.32!<|assistant_end|><|bos|><|user_start|>What's your name?<|user_end|><|assistant_start|>I'm prelude, a very small language
-
chat 3.25add numbers.<|assistant_end|><|user_start|>Who are you?<|user_end|><|assistant_start|>I'm prelude, a very small language
-
story 3.25met a cute bunny. The bunny said, "Hi, I am Bunny. Let's be
-
story 3.19boy smiled and said, "Sure, I am Danny. I like bri