When most people talk about “controlling” large language models, they’re usually talking about prompt engineering.
You rewrite the instruction.
You add constraints.
You say “think step by step.”
And the output improves. It feels like magic, doesn't it?
But prompt engineering is only the surface layer of control. Beneath it lies something much more interesting and powerful: activation steering, the ability to nudge a model’s internal representations during inference.
To understand why this matters, we need to zoom in a little.
Steering as Probability Shaping
At its core, a language model is just estimating:
P(next token | context)
Every time it generates a word, it’s selecting from a probability distribution over possible next tokens.
All steering methods, in one way or another, reshape that distribution.
Prompt engineering does it by changing the context. Decoding tricks do it by changing how we sample. Activation steering does it by changing the model’s internal state before the distribution is even computed.
That last one is fundamentally different.
Prompt Engineering: Steering from the Outside
Prompting works because LLMs are extremely context-sensitive. Small changes in wording can dramatically shift outputs.
Ask:
Explain black holes.
Then ask:
Explain black holes to a 12-year-old using simple analogies.
You’ll get entirely different responses.
Nothing inside the model changed. The weights stayed frozen. But the input context altered the trajectory of generation.
Prompt engineering is powerful precisely because it’s accessible. It requires no internal access, no gradients, no architecture knowledge. It treats the model as a black box and still manages to guide it.
But it has limits. Prompts can be brittle. They can fail under adversarial phrasing. They don’t always provide consistent behavioral shifts across diverse inputs. And when you want fine-grained control over something abstract — like reducing hallucination tendency or increasing reasoning depth — prompts start to feel blunt.
You’re steering the system indirectly, hoping the model interprets your intent correctly.
Activation Steering: Steering from the Inside
Activation steering approaches the problem differently.
Instead of modifying the words going into the model, we intervene in the hidden states produced during the forward pass.
Every transformer layer produces high-dimensional vectors — hidden representations that encode features about the current context. These vectors are not random. They capture structure: tone, intent, topic, reasoning state, even safety alignment signals.
Research in interpretability has shown that certain behavioral traits correspond to specific directions in this activation space. That means behaviors like politeness, refusal, toxicity, or step-by-step reasoning aren’t isolated modules — they’re patterns distributed across dimensions.
If you can identify a direction in activation space that corresponds to a behavior, you can add or subtract it during inference:
h' = h + αv
Here,
h = original hidden state
v = behavior vector
α = steering strength
No weights are updated. No retraining occurs. The model’s brain is untouched — but its moment-to-moment thinking trajectory is altered.
Instead of asking the model to “be polite,” you are geometrically shifting its internal representation toward a region associated with politeness.
That is a much more direct form of control.
What Does Activation Steering Look Like in Practice?
At a high level, activation steering requires access to the model’s hidden states during the forward pass.
Step one is extracting internal activations. In most transformer libraries (like Hugging Face), you can register forward hooks to capture the hidden states at a specific layer.
Step two is constructing a steering direction. One simple approach is contrastive:
- Run the model on prompts that produce “Behavior A” (e.g., confident responses).
- Run it again on prompts that produce “Behavior B” (e.g., hedging responses).
- Collect the hidden states from the same layer.
- Compute the mean difference between them.
Conceptually:
v = mean(h_confident) - mean(h_hedging)
That difference vector becomes your behavioral axis.
Step three is injection. During inference, when the model computes hidden states at that layer, you modify them:
h' = h + αv
The scalar α controls how strongly you steer. Small values subtly bias behavior. Large values can distort coherence.
That’s it.
No retraining. No gradients. Just geometric manipulation inside the forward pass.
Why This Even Works
It might sound surprising that behaviors can be represented as directions in vector space, but this is a natural consequence of how neural networks learn.
LLMs don’t encode knowledge as rules. They encode statistical structure across millions or billions of dimensions. Patterns that frequently co-occur during training become embedded as geometric relationships.
So “being sarcastic” or “refusing unsafe content” is not a switch. It’s a region in high-dimensional space.
Activation steering works because these regions are not completely entangled. They are partially separable. With the right analysis, you can isolate directions that correlate strongly with particular behaviors and nudge the model along them.
You’re not adding new knowledge. You’re reweighting existing tendencies.
Prompting vs Activation Steering
Prompting says:
“Please behave this way.”
Activation steering says:
“Shift your internal representation toward this behavioral manifold.”
Prompting modifies language.
Activation steering modifies cognition.
One is indirect and linguistic. The other is geometric and internal.
That difference matters when consistency and robustness are important. If you want a model to reliably reduce hallucinations or amplify chain-of-thought reasoning across many prompts, internal control may be more stable than surface-level instructions.
Is This Just Fine-Tuning in Disguise?
Not quite.
Fine-tuning permanently changes model weights. It rewrites parameters. It requires data and training cycles.
Activation steering happens entirely at inference time. It is reversible. It is lightweight. It doesn’t risk catastrophic forgetting or degrade unrelated capabilities.
Fine-tuning edits the model’s memory.
Activation steering temporarily biases its thinking.
That flexibility makes it appealing, especially for research and alignment experiments.
A Small Experiment: Steering Confidence Internally
To make this less abstract, I ran a small experiment on an open-weight instruction-tuned model.
The goal was simple: compare prompt steering vs activation steering along a behavioral axis — confidence vs hedging.
Instead of changing the weights, I constructed a steering vector by contrasting internal activations from:
- Confident, assertive responses
- Hedging, uncertainty-heavy responses
This gave a behavioral direction in activation space.
During inference, I injected that vector into a middle transformer layer:
h' = h + αv
Again where:
- h is the hidden state
- v is the confidence direction
- α controls steering strength
I then compared three setups:
- Baseline (no steering)
- Prompt steering ("be confident, do not hedge")
- Activation steering (vector injection)
The goal wasn’t to prove activation steering is universally better — but to explore how internal representation shifts differ from surface-level instructions.
If you're curious about the full implementation, layer sensitivity analysis, and alpha trade-offs, you can check out the complete notebook here:
Colab:
https://colab.research.google.com/drive/1zgN3ydePd4NqPxRQQ7DKRyCc5NikBMIQ?usp=sharing
Github
https://github.com/iamfaham/llm_steering
The takeaway is simple:
- Prompt steering changes what the model reads.
- Activation steering changes how the model thinks.
The Bigger Implication
Activation steering hints at something deeper about large language models: their behaviors may be navigable.
Not modular in the traditional software sense, but geometrically modular. If behaviors correspond to directions, then intelligence becomes something we can traverse — push slightly in one direction for more reasoning, pull back in another to reduce verbosity, amplify a safety signal, dampen a risky one.
Instead of retraining giant models for every behavioral tweak, we might learn how to navigate their internal landscape.
Prompt engineering was the first wave of LLM control. It taught us that context shapes behavior.
Activation steering suggests the next wave: that behavior is embedded in structure — and structure can be manipulated.
If that’s true, then steering isn’t just a trick. It’s a new way of thinking about controllable intelligence.
Connect & Share
I’m Faham — currently diving deep into AI/ML while pursuing my Master’s at the University at Buffalo. I share what I learn as I build real-world AI apps.
If you find this helpful, or have any questions, let’s connect on LinkedIn and X (formerly Twitter).
AI Disclosure
This blog post was written by Faham with assistance from AI tools for research, content structuring, and image generation. All technical content has been reviewed and verified for accuracy.



Top comments (0)