Feature #3515
Fires on “h” (not labeled yet: read the examples)
- Fires on
- 0.98% of tokens
- Mostly in
- agent
- Layer
- after block 4
Where it fires
- agent 40%
- story 25%
- chat 22%
- worksheet 9%
- python 4%
Makes the model say
ooam lose3alinsup
Makes it not say
they prints moral waited named```)`.Fix
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
story 3.43said, "Let's play a new game. It is called 'bathe the toy
-
story 3.41iting him.⏎At the party, they played a game called "match". They
-
story 2.99in his shop. He painted it with bright colors and wrote "Share" on it
-
story 2.9in her mind.⏎Sally asked, "Mom, what does 'bitter' mean?"
-
story 2.78ummy were at the beach. Joey was so excited to build sandcastles with his
-
story 2.77has a big button and a big hole. A sign says "Shrink or G
-
story 2.59the shell, Ben. But you have to help me build a sandcastle too,"
-
story 2.59be nice to each other, share our toys, and always say 'please' and '
-
story 2.55month. For example, in January they drew snowmen, in A
-
story 2.53sand.⏎"Come on, Sara, help me build a sandcastle!" he
-
story 2.49<|assistant_end|><|bos|>Once upon a time, there was a big, soft mattress. It was
-
story 2.48Once upon a time, there was a small pigeon. It hop-hop-hopped