The privacy problem with AI assistants like Instinct

The more useful an AI assistant is, the more data it needs. The question nobody's answering well enough: where does that data go, who controls it, and what happens when you leave?

The pitch for AI assistants in 2026 is compelling: an AI that manages your schedule, triages your inbox, follows up with contacts, negotiates your bills, and handles the administrative overhead that consumes hours every week. The capability is real. Early testers of Instinct, Meta's Muse, and Poke all report that the experience is transformative when the assistant has enough context to act intelligently on your behalf.

The privacy problem is equally real, and it's structural rather than incidental. The more capable the assistant, the more data it needs. The more data it has, the more exposed you are. And the terms under which that data is collected, stored, and used vary dramatically between products, often in ways that users don't discover until something goes wrong.

What the assistants need access to

The capability progression tells the privacy story:

A scheduling assistant needs your calendar. A triaging assistant needs your email. A proactive assistant needs your messages, your location, and your patterns. A fully autonomous assistant needs your financial accounts, your passwords, and the ability to act on your behalf. Each capability increase requires a corresponding increase in access. The assistant that handles everything needs access to everything.

This isn't a flaw in the design. It's the design. An assistant that doesn't know your schedule can't schedule. An assistant that can't read your email can't triage. The privacy problem isn't that assistants want access. It's that the terms, retention, and business models around that access vary from reasonable to alarming.

Three models, three risk profiles

The surveillance model (Instinct)

Instinct's launch terms granted a perpetual, irrevocable, sub-licensable licence to access, use, store, reproduce, transmit, distribute, and modify user data, including for AI model training. The data collection included screen captures, keyboard inputs, and continuous email access. Within days, testers reported Gmail data retained after access was revoked, an email sent without approval, and a prompt injection vulnerability that leaked inbox data.

The risk profile: maximum capability, maximum exposure. The perpetual licence means the data rights survive even after you stop using the product. The business model (free product, venture-backed at $2.5 billion) suggests the data is the revenue source, not subscriptions.

The platform model (Meta Muse, Grok)

Meta's Muse runs in a "dedicated secure virtual machine" with its own browser. The architecture is more contained than Instinct's, and Meta claims data protections. But Meta's track record on data handling, including a $17 billion settlement with state attorneys general over platform harms, creates a trust deficit that architectural claims don't fully resolve. Muse connects to Gmail, Google Calendar, Ticketmaster, OpenTable, and Stripe, giving the assistant broad access to personal and financial data.

Grok, built by xAI, provides AI assistance with real-time access to X (formerly Twitter) data. The privacy considerations are different: the public social data is already accessible, but the personal conversation history and usage patterns create a profile that's governed by xAI's terms.

The risk profile: the assistants are backed by large companies with resources to build security infrastructure but also with business models (advertising, data-driven engagement) that create structural incentives to retain and use personal data.

The private context model (Fabric)

Fabric's approach is architecturally different: you choose which tools to connect. The data lives in your infrastructure with bring-your-own-storage. No perpetual licence. No screen recording. No keyboard logging. The AI gets context from the sources you've chosen, stored in your cloud storage, encrypted with your keys.

The capability trade-off: no ambient screen capture or keyboard logging means the assistant knows less about your moment-to-moment activity. The context from connected tools (email, Slack, meetings, files) provides rich professional context without the surveillance layer. For knowledge work, the structured context typically produces better AI output than raw surveillance data because the signal-to-noise ratio is higher.

The risk profile: controlled exposure. You decide what the AI can access. You can revoke access. The data is in your infrastructure. The privacy posture is architectural, not policy-based.

What to look for

Before granting an AI assistant access to your data, check five things:

The licence. Does the company claim a perpetual, irrevocable licence to your data? Can they use it for training? Can they sub-licence it to third parties? These terms matter because they determine what happens to your data after you've shared it.

The retention. What happens to your data if you revoke access or delete your account? Is the data deleted or retained? Instinct's early testers found that Gmail data was retained after access was revoked. The retention policy determines whether "leaving" actually means leaving.

The business model. Is the product free? If so, the data is likely the business model. A subscription-based product has less structural incentive to monetise your data. This isn't foolproof (a subscribed product can still have aggressive data terms) but the business model shapes the incentives.

The architecture. Is your data stored on the company's servers or on infrastructure you control? Cloud-hosted data governed by the company's terms is fundamentally different from data in your own S3 bucket governed by your keys. Bring-your-own-storage is the architectural answer to the trust question.

The track record. How has the company handled data in the past? Meta's settlement history, Instinct's launch controversies, and each company's security track record are relevant signals. Past behaviour is the best predictor of future behaviour.

The structural answer

The privacy problem with AI assistants isn't solvable through better terms of service. Terms can change. Companies can be acquired. Policies can be reinterpreted. The structural answer is architectural: data that lives in your infrastructure, encrypted with your keys, accessible only through connections you've authorised, and revocable at any time.

This is the private context layer model. The AI gets the context it needs to be useful. You retain control of where that context lives and who can access it. The trust is in the architecture, not in the policy.

Frequently asked questions

Should I avoid AI assistants entirely? No. The capability is real and valuable. The question is which privacy model you're comfortable with. A private context model (Fabric) provides rich AI assistance without the surveillance trade-offs. A subscription chatbot (Claude, ChatGPT) provides general assistance with moderate data exposure. A surveillance assistant (Instinct) provides maximum convenience with maximum exposure. Choose based on your risk tolerance.

Is Meta Muse safer than Instinct? Architecturally, possibly. Meta's secure VM claim and the company's infrastructure resources suggest better security engineering. But Meta's business model is advertising, which creates structural incentives to retain and use personal data. The trust calculation involves both the technical architecture and the business incentives.

What about Apple Intelligence? Apple's approach (on-device processing, user-controlled permissions, privacy as a design principle) is the closest to the private context model among the major platform players. The limitation is that Apple's AI capabilities lag behind dedicated AI assistants. As Apple improves the AI quality, its privacy model becomes increasingly attractive.

Can I use Fabric as a personal AI assistant? Yes. Fabric's AI assistant answers questions from your accumulated knowledge. Agents handle recurring tasks (follow-ups, summaries, monitoring). The assistant doesn't manage your calendar or book restaurants (that's what Muse and Poke do) but it handles knowledge work assistance, client management, and operational tasks with full privacy.

What happens to my data if the AI company is acquired? In most cases, your data transfers to the acquiring company under the existing terms. A perpetual licence doesn't expire when the company changes hands. The acquirer inherits the data rights. Bring-your-own-storage protects against this: if the company is acquired, your data is still in your infrastructure.

Is the privacy concern overblown? For casual queries ("what's the weather"), the privacy risk is minimal. For assistants with deep access (email, calendar, financial accounts, screen recordings), the risk is proportional to the depth of access. The more capable the assistant, the more legitimate the privacy concern.

Will regulation fix this? Eventually, partially. GDPR and similar regulations provide baseline protections. But regulation trails innovation, and the AI assistant category is moving faster than regulators can respond. Architectural protections (bring-your-own-storage, user-controlled access) are available now and don't depend on regulatory enforcement.

What's the most privacy-respecting AI assistant in 2026? For knowledge work: Fabric with bring-your-own-storage. For general chat: Claude (strong privacy stance, no training on conversations by default). For personal life management: Apple Intelligence (on-device processing). The answer depends on what you need the assistant to do.

Related reading: Instinct and the case for a private context layer, What Instinct gets right and misses, Own your data, own your AI, Where does your knowledge live?. Related pages: Private and secure, Privacy.