Feature #2648
Fires on “ the” (not labeled yet: read the examples)
- Fires on
- 0.65% of tokens
- Mostly in
- agent
- Layer
- after block 4
Where it fires
- agent 46%
- story 26%
- python 13%
- chat 11%
- worksheet 3%
Makes the model say
end out fastat colors his[::-um
Makes it not say
friend onto `HiiesMEsubhe
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
chat 2.72tools: calculator<|system_end|><|system_start|>You are a helpful assistant.<|system_end|><|user_start|>Find the sum of 9
-
chat 2.64<|system_start|>You are a helpful assistant.<|system_end|><|user_start|>Find the sum of 9
-
python 2.61⏎⏎`evens` keeps only the even numbers. For example
-
chat 2.59<|system_start|>You are mini, a small and friendly assistant.<|system_end|><|user_start|>Find the sum of 7
-
worksheet 2.58add numbers, step by step or with a calculator.<|assistant_end|><|bos|>Find the sum of 6
-
chat 2.57<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>Find the sum of 4
-
python 2.55ood.<|bos|>`only_even(nums)` keeps only the even numbers.⏎⏎
-
python 2.45.<|assistant_end|><|bos|>`keep_even(nums)` keeps only the even numbers.⏎⏎
-
worksheet 2.429153 is 9199.⏎Find the sum of 6
-
chat 2.41to do.<|bos|><|system_start|>You are a helpful assistant.<|system_end|><|user_start|>Find the sum of 0
-
chat 2.37<|system_start|>You are mini, a small and friendly assistant.<|system_end|><|user_start|>Find the sum of 2
-
python 2.37]⏎```⏎⏎`only_even` keeps only the even numbers. For example