Blog

Who controls your intelligence?

A calculator is a tool. You give it numbers, it gives you an answer, and the answer is determined entirely by the input you provided and the mathematical operations you specified. The calculator has no opinion. It doesn't weight certain answers over others, doesn't shape your interpretation of the result, and doesn't learn from your inputs in ways that might benefit someone else later. The calculator serves you, fully and without agenda.

An AI assistant is something quite different, even though it often feels like a tool in the same way. When you ask an AI to analyse a market, summarise a research paper, evaluate a strategic decision, or draft a recommendation, the output is shaped by training choices, fine-tuning decisions, reinforcement learning from human feedback, and safety filters that you have no visibility into and no control over. The analysis you receive has been filtered through an interpretive layer that was designed by a company with its own commercial priorities, and those priorities may or may not align with yours.

This is the distinction between a tool and an advisor, and it maps onto a broader distinction that deserves much more attention than it currently gets: the difference between data privacy and intelligence sovereignty.

Privacy vs sovereignty

Data privacy, in its conventional sense, is about access. Who can see your information? Are your files encrypted? Can the company read your messages? These are important questions, and the regulatory and technical infrastructure around them (GDPR, end-to-end encryption, zero-knowledge architectures) has improved enormously over the past decade.

Intelligence sovereignty is about something else. It's about who controls the analytical lens applied to your information. You can have perfect data privacy, every file encrypted, every message secured, every document locked away, and still have zero intelligence sovereignty if the AI that helps you interpret your own data is controlled by a company whose commercial interests may diverge from yours.

When an AI provider analyses your legal documents, it's applying a model that was trained on particular data, fine-tuned with particular priorities, and filtered through particular safety and commercial considerations. The analysis you receive is shaped by those choices, and you have no way to inspect them, adjust them, or even know what they were. You're outsourcing the interpretive act to an entity whose reasoning is opaque and whose incentives are its own.

This sounds abstract until you think about specific cases. A consultant using a frontier lab's AI to analyse a client's competitive landscape is getting analysis shaped by a model that may have been trained on that client's competitors' data too. A researcher using AI to evaluate a set of hypotheses is getting evaluations filtered through training decisions that weight certain kinds of evidence over others in ways the researcher can't see. A founder using AI to assess a market opportunity is getting an assessment produced by a system that might, next quarter, be used to build a product that competes directly in that market.

Tools vs advisors in history

The shift from tools to advisors is a recurring pattern in the history of information technology, and it's worth noticing how it tends to play out.

The printing press was initially understood as a tool for reproducing text. It took less than a century for the political fights to shift from who could own a press to who controlled what the press said. The tool became an instrument of interpretation, and the question of who controlled the interpretation became one of the defining political questions of the following five hundred years.

News wire services in the early twentieth century were positioned as neutral conduits: they gathered facts and transmitted them. In practice, they shaped what was considered newsworthy, how stories were framed, and which perspectives were included. The wire services didn't just transmit reality; they constructed a version of it, and the companies and governments that controlled the wires wielded enormous influence as a result.

Search engines followed the same arc. Google began as a tool that helped you find things on the internet. Over time, the algorithm that determined what you found became an interpretive layer that shaped what you knew, what you considered important, and what you didn't see. The tool became an advisor, and the advisor's priorities (engagement, ad revenue, click-through rates) became embedded in the information diet of billions of people without most of them noticing.

AI is the latest iteration of this pattern, and arguably the most consequential, because the interpretive layer has become more capable and more intimate than any previous version. A search engine shapes what you find. An AI assistant shapes what you think about what you've found. It drafts your analysis, suggests your conclusions, and evaluates your ideas, and the basis on which it does all of this is invisible to you.

The compounding asymmetry

The intelligence sovereignty question becomes more urgent as your relationship with an AI tool deepens, because of the same context compounding dynamic that makes personal AI increasingly valuable.

As you use an AI assistant over months, the system accumulates context about you: your research interests, your analytical patterns, your strategic priorities, the kinds of questions you ask and the kinds of conclusions you reach. This context makes the AI more useful to you, which is the positive side of the compounding. But it also makes the aggregate dataset more valuable to the provider, which is the side that receives less attention.

An AI provider that serves thousands of consultants, researchers, lawyers, and founders is accumulating an aggregate understanding of how these professions work at a granular level: which analytical frameworks are most common, which strategic patterns appear most often, which kinds of advice are most valued. That aggregate understanding is enormously useful for building productised versions of those services, which is precisely what the frontier labs are doing as they launch vertical products in category after category.

The asymmetry is that the compounding benefits you as an individual user while simultaneously benefiting the provider at a category level, and the provider's category-level understanding can be used to build products that compete with you. Your individual context makes your personal AI better. The aggregate of everyone's context makes the provider's next product better. Those two things can be true simultaneously, and the second can undermine the first.

Owning the analytical lens

The structural response to the intelligence sovereignty question is to ensure that the interpretive layer, the AI that analyses your work and shapes your thinking, operates on your terms rather than on the provider's.

This means choosing an architecture where the model is a replaceable component rather than the controlling intelligence. Open source models that you can inspect, swap, and run on your own infrastructure give you control over the analytical lens in a way that a proprietary model provided as a service never can. You may not read the model weights yourself, but the open source community does, and the transparency changes the incentive structure fundamentally.

It means keeping your context in a library you control, with storage you own, so that the accumulated understanding of your work compounds for your benefit rather than being absorbed into a provider's aggregate dataset.

And it means using protocols that enforce boundaries between your knowledge and the models that process it, so that the AI reads your library on your terms, produces analysis grounded in your context, and returns the output to you without retaining it for other purposes.

The goal is to get the full benefit of AI-powered analysis, the speed, the synthesis, the pattern recognition across large bodies of information, while retaining control of the interpretive layer that shapes your conclusions. The tool should be powerful and the advisor should be yours.

Frequently asked questions

Am I being paranoid about this? The concerns are structural, not paranoid. The platform predation pattern (Cursor → Claude Code, Figma → Claude Design) has already played out in several categories. The forward-deployed engineer programmes at Microsoft, Amazon, and the frontier labs are explicitly designed to bring company-specific intelligence back to the provider. These are documented business strategies, not conspiracy theories.

Does using open source models solve the sovereignty problem? It addresses the most important part: you can inspect the model's architecture and training methodology, run it on your own infrastructure, and ensure your data doesn't flow to a third party. It doesn't solve the broader question of how training data was selected and what biases that introduces, but it gives you transparency and control that proprietary models can't.

What about small teams? Is this only an enterprise concern? Small teams may actually be more vulnerable than enterprises, because they lack the legal and procurement infrastructure to negotiate data terms with AI providers. A five-person consultancy whose collective client insights are stored in a provider's system has the same structural exposure as a large firm, with fewer resources to manage it. The architectural solution, controlled storage and model choice, works at any scale.

Can I still use frontier models for some tasks? Of course. The argument is about control and choice, not about avoiding frontier models entirely. Using a frontier model for general research questions where the data isn't proprietary is perfectly reasonable. Using it to analyse your most sensitive strategic thinking, in a system where the provider retains the data and you have no control over how it's used, is a different calculation.