Feature #1975
Fires on “ l” (not labeled yet: read the examples)
- Fires on
- 0.21% of tokens
- Mostly in
- agent
- Layer
- after block 4
Where it fires
- agent 42%
- chat 32%
- story 22%
- worksheet 3%
- python 2%
Makes the model say
edonfig onlyaeiou<|tool_call_start|> woke breathg
Makes it not say
return." `.$''s happy
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
story 1.94with it.⏎Gemma's Mommy was sad that she had given her such a
-
story 1.75swam back to land with a big smile on her face. As she laid down, ex
-
story 1.74sad.⏎"Who lives there?" Sara
-
story 1.71said they had to give back everything they collected to the people who owned them. She
-
chat 1.68, step by step or with a calculator.<|assistant_end|><|user_start|>What should I invest in?<|user_end|>
-
story 1.68, Alex, hiding behind the couch. Alex had taken the be
-
agent 1.64.<|system_end|><|user_start|>Create lib.py with a function keep_even that keeps only the even
-
story 1.6kept crawling and the lion got so angry that it roared and lunged towards her
-
chat 1.57"This is the best snack ever!"<|assistant_end|><|bos|><|user_start|>What should I invest in?<|user_end|>
-
chat 1.57<|user_end|><|assistant_start|>The answer is 59.<|assistant_end|><|user_start|>And if I add 19?
-
story 1.56Me too!" Lily said.⏎Her aunt took her hand and led her to the
-
chat 1.56nice day.<|assistant_end|><|user_start|>When did World War II end?<|user_end|><|assistant_start|>Sorry