Feature #12
Fires on “I” (not labeled yet: read the examples)
- Fires on
- 0.36% of tokens
- Mostly in
- chat
- Layer
- after block 4
Where it fires
- chat 64%
- worksheet 20%
- agent 12%
- story 2%
- python 2%
Makes the model say
addlist pythonL examplehttpsexclaimadd
Makes it not say
to0 test while every Mr peopleice
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
chat 5.6272 and 79 is 151.<|bos|><|user_start|>I need to know
-
chat 5.474179 equals 54218.<|bos|><|user_start|>I need to know
-
chat 5.3989<|think_end|>The answer is 189.<|assistant_end|><|bos|><|user_start|>I need to know
-
chat 5.37to be very yummy, and they had a fun day.<|bos|><|user_start|>I need to know
-
chat 5.37+ 887 = 1507.<|bos|><|user_start|>I need to know
-
chat 5.36I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|user_start|>I want a story!
-
chat 5.315308 = 13142.<|bos|><|user_start|>I need to know
-
chat 5.3194<|think_end|>The answer is 94.<|assistant_end|><|bos|><|user_start|>I need to know
-
chat 5.2569<|think_end|>The answer is 69.<|assistant_end|><|bos|><|user_start|>I need to know
-
chat 5.23knew that going to the store was a fun adventure.<|assistant_end|><|bos|><|user_start|>I want a story about
-
chat 5.21<|think_end|>Sure! The answer is 417.<|assistant_end|><|bos|><|user_start|>I want a story!
-
chat 5.21230<|assistant_end|><|bos|><|system_start|>Answer in one short sentence.<|system_end|><|user_start|>I want a story about