Blog

Run a research project with a hundred agents



Spread interviews, papers and transcripts across many AI agents working on one shared folder, so each job's results are there for the next job and nothing has to be merged at the end.



The simplest way to run a research project across many AI agents is to point every job at the same shared folder. The sources sit in one place, each job reads what it's assigned and writes its findings to an output folder, and the next job can read those findings the moment they're saved. There's no copy of the data per job, and no merge step at the end.

This post walks through how that works for a study built on interviews, papers and transcripts, and what the folder has to handle once dozens of agents are writing to it.


The copy-per-job habit

The usual way to parallelize agent work is to give each job its own sandbox and its own copy of the material. A team studying how independent retailers choose payment providers might have 180 customer interviews, 60 papers and a few years of support transcripts. They split the interviews into batches, copy a batch into each sandbox, run an agent per batch, and collect the output files when the jobs finish.

It works until the second pass. Job 70 has no idea that job 12 already found a recurring complaint about chargeback fees, so it reports the same theme in different words. The outputs come back as a pile of files with overlapping names, and someone spends a day stitching them into one document, removing duplicates and deciding which version of themes.md is the right one. Meanwhile a new interview arrives, and it isn't in any of the copies.

The research was parallel. The results weren't, and the person doing the merge became the bottleneck.


One folder for the whole study

With a shared drive, every job works on the same files. A layout for the retail study might look like this:

payments-study/
  AGENTS.md
  inputs/
    interviews/
    papers/
    support-transcripts/
  results/
    interviews/
    papers/
  synthesis/
    themes.md
    open-questions.md
payments-study/
  AGENTS.md
  inputs/
    interviews/
    papers/
    support-transcripts/
  results/
    interviews/
    papers/
  synthesis/
    themes.md
    open-questions.md
payments-study/
  AGENTS.md
  inputs/
    interviews/
    papers/
    support-transcripts/
  results/
    interviews/
    papers/
  synthesis/
    themes.md
    open-questions.md

inputs/ holds the source material, results/ holds one findings file per source, and synthesis/ holds the documents that pull everything together. AGENTS.md at the top describes the coding scheme and the format every findings file should follow.

Getting material in is mostly a matter of uploading it. On a paid plan, Fabric transcribes uploaded audio and video in MP3, WAV, M4A, MP4, MOV and WebM, so recorded interviews become searchable text without a separate tool. Text in PDFs is searchable too, which covers the papers. Audio and video transcription has the details, and a user research repository covers how teams keep interviews useful after the project.


Scoping each job

A job that reads interview 143 doesn't need to delete anything, and it doesn't need to see the team's budget spreadsheet. With Agent Vault, you give the research agents access to the study folder only. Reading files and running searches across inputs/ covers what they read. Creating documents in results/ covers what they write, and the later jobs that pull themes together also get write access to synthesis/. Moving and deleting can stay off.

Access can be set for a single task, a day, a week or indefinitely. For a batch run, a single task or a day is usually right, and the access ends on its own when that period is over. If a job decides it needs something outside its scope, such as last year's study for comparison, it sends a request and waits until someone approves or denies it. Why AI agent access should expire makes the case for working this way, especially when jobs run untested code in sandboxes.

Every action is logged with a timestamp, and the log can be filtered by agent, area, period or action. When a findings file looks odd, you can check what the job that wrote it read first. Agent Vault is on the Pro and Team plans.


Results are there as soon as they're written

Because every job writes into the same folder, a finding saved at 10:02 can be read by a job that starts at 10:03. That changes how a large study can be sequenced.

A first wave of jobs reads one interview each and writes a findings file to results/interviews/. A second wave, started while the first is still running, reads the findings rather than the raw interviews and groups them into themes. A third job writes the summary. None of them wait for a human to collect files in between, and a late interview simply joins the first wave when it arrives.

Later jobs find earlier results through search. Fabric runs grep, regular-expression and full-text search on the server across the whole drive, so a theming job can look for every findings file that mentions "chargeback" or matches a coding tag like #pricing-trust without downloading the folder. Why AI agents give up searching your files explains why that speed matters when agents search over and over. Long sources can be read in part, too. A job can read one section of a three-hour transcript, or part of a large CSV of survey answers, without fetching the whole file. Let AI agents analyze large data files goes further into that.


Many writers, nothing to merge

Most of the writing doesn't collide. Each first-wave job writes its own findings file, named after its source, so two jobs rarely touch the same file.

The synthesis files are different. Several theming jobs may add to themes.md within the same few minutes. If one of them saves a change based on a version of the file that another job has since updated, Fabric refuses that write. The job re-reads themes.md, sees what changed, and reapplies its addition on top. Nothing is silently overwritten, and no themes (conflicted copy).md appears beside the original. That revision check is what makes it safe to let many agents write into the same folder at once. When two AI agents edit the same file walks through the mechanism with a smaller example.

It's still worth designing the folder so collisions are rare. One file per source in results/ and a short, append-only themes.md keep the refused writes to a handful.


Everyone follows the same rules

In a long run, consistency matters as much as speed. If job 90 codes interviews differently from job 9, the themes stop meaning anything.

The coding scheme lives in AGENTS.md, and every job reads it before starting. Fabric doesn't let agents change instruction files such as AGENTS.md, CLAUDE.md and SKILL.md, even when they have write access. A job that misreads an interview, or reads a transcript containing text that looks like instructions, still can't rewrite the rules for every job after it. When the researchers refine the scheme halfway through, they edit the file themselves and the next wave picks it up. Your agent instruction files are a security risk has more on this, and share skills with your whole team covers keeping reusable steps, such as a findings template, as a skill.


When a job gets it wrong

With enough jobs, one will eventually overwrite a good findings file with a worse one, or delete something it shouldn't. Fabric keeps earlier versions in version history, and deleted files go to a trash you can restore from. The activity log shows which job did it. What happens when an AI agent deletes the wrong file covers recovery.


Connecting the jobs

Agents in sandboxes usually connect through Fabric's MCP server, the CLI or the API. A researcher reviewing results on their own computer can use the mounted drive from the desktop app and open the findings like any folder. How AI agents connect to a shared drive compares the options.

Fabric prices per person, so the research agents don't need seats of their own. See pricing for teams.

The same idea runs through the rest of this series, starting with your AI agents need a shared drive, not a group chat. For research workflows without agents, see Fabric for research.


Frequently asked questions

Can many AI agents work on the same folder at once?

Yes. In Fabric, each agent reads and writes the shared folder directly. If an agent saves a change based on an outdated version of a file, the write is refused and the agent reapplies its change, so agents don't overwrite each other.

How do I stop research agents seeing files outside the project?

Use Agent Vault to give them access to the project folder only, with separate permissions for reading, searching, creating documents, moving and deleting.

Do I need to merge results from parallel agent jobs?

Not when every job writes to the same shared folder. Results appear as soon as they're saved, and later jobs read them directly.

Can agents analyze interview recordings?

On a paid plan, Fabric transcribes uploaded audio and video, and agents can then read and search the transcripts like any other text.

How do later agents find what earlier agents wrote?

They search the drive. Fabric runs grep, regular-expression and full-text search on the server, so agents don't have to download the results folder first.

Can an agent change the instructions other agents follow?

No. Fabric protects instruction files such as AGENTS.md, CLAUDE.md and SKILL.md, so agents can't change them even with write access.

What happens when a research agent's access period ends?

Access set for a single task or a fixed period ends automatically. You can also revoke it at any time.

Do I pay for each research agent?

No. Fabric prices per person, so agents don't need their own seats.


Related pages


The workspace that thinks with you.

Ready when you are.

The workspace that thinks with you.

Ready when you are.

The workspace that thinks with you.

Ready when you are.