Feature #3609
Fires on “<|system_start|>” (not labeled yet: read the examples)
- Fires on
- 1.30% of tokens
- Mostly in
- agent
- Layer
- after block 4
Where it fires
- agent 44%
- chat 29%
- story 17%
- python 7%
- worksheet 2%
Makes the model say
script<|system_start|> preludelower definesubcontent defined
Makes it not say
,??" go f She done find
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
story 3.81wheel.<|bos|>Once upon a time, there was a little dolphin named Dolly. Dolly
-
story 3.645.<|assistant_end|><|bos|>Once upon a time, there was a dolphin named Dolly. Dolly
-
story 3.22anymore.<|bos|>Once upon a time, there was a small bird named Mr Chirp.
-
story 3.01with a big hat. The man said, "Hi, I am Mr. Rich.
-
chat 2.95of `return len(s)`. The tests pass now.<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|>
-
agent 2.93of `return items[-1]`. The tests pass now.<|assistant_end|><|bos|><|system_start|>tools: edit,
-
chat 2.91return x * x * x`. The tests pass now.<|assistant_end|><|bos|><|system_start|>Answer with the number
-
agent 2.9`result = result * i`. The tests pass now.<|assistant_end|><|bos|><|system_start|>tools: edit,
-
chat 2.89of `return len(s)`. The tests pass now.<|assistant_end|><|bos|><|system_start|>Answer with the number
-
chat 2.88of `return a ** b`. The tests pass now.<|assistant_end|><|bos|><|system_start|>You are mini,
-
agent 2.87of `return min(nums)`. The tests pass now.<|assistant_end|><|bos|><|system_start|>tools: edit,
-
agent 2.87of `return s[0]`. The tests pass now.<|assistant_end|><|bos|><|system_start|>tools: edit,