Blog

The research log: why your memory is lying to you

The honesty problem

There's a specific kind of dishonesty that researchers, knowledge workers, and anyone who works with ideas is vulnerable to. It's not the kind that involves fabricating data or misrepresenting results. It's quieter than that. It's the way your memory selectively preserves the things that went right and quietly discards the things that didn't.

You run an experiment. It doesn't work. You adjust your hypothesis, run another version, and this time it does work. A month later, when you're writing up the results or explaining your approach to someone, the failed version has faded. Not disappeared entirely, but its edges have softened. You remember that you tried something that didn't work, but the specifics are gone: what exactly you expected, why you expected it, what the actual result was, and what it told you about your assumptions.

Feynman put it simply: the first person you must avoid fooling is yourself, because you're the easiest target. Your brain is extremely good at constructing a narrative that makes your current beliefs feel like they were always the obvious ones, smoothing over the turns, dead ends, and surprises that actually characterised the work.

A research log is the cheapest defence against this.

What Darwin understood

Darwin's approach to this problem was procedural and deliberate. He had a rule: any fact or observation that contradicted his theory had to be written down immediately, because he'd noticed that his memory would delete inconvenient evidence faster than the convenient kind.

This wasn't a productivity technique. It was an epistemic one. Darwin understood that his own cognitive system was biased toward preserving consistent beliefs and discarding inconsistent ones, and he designed a physical workaround. The notebook was more reliable than his brain, specifically for the information his brain most wanted to lose.

The same bias is operating in your work right now. The experiments that confirmed your hypothesis are clear in your memory. The ones that didn't are already fading. The thing you were wrong about last month has been subtly rewritten into something you were "sort of right" about. The assumption you never tested has become, in your memory, an assumption you tested and confirmed.

Writing it down, at the time, before your memory has a chance to edit, is the only reliable fix.

The format

A research log doesn't need to be complicated. For each piece of work worth documenting, capture five things:

Hypothesis. What do you believe going in? State it explicitly, even if it feels obvious. The point of writing it down is to create a record you can't retroactively adjust.

Setup. What are you doing to test this? The method, the approach, the tools, the data. Enough detail that you could reproduce it, or at least understand what you did when you read this entry six months from now.

Expectation. What do you think will happen? This is different from the hypothesis. The hypothesis is the belief; the expectation is the predicted outcome. "I believe X is true, so I expect the result to show Y." Writing this down is where the real discipline lives, because it forces you to make a concrete prediction that can be checked against reality.

Result. What actually happened? Not your interpretation of what happened. The raw result. The numbers, the observation, the outcome. Leave interpretation for the next step.

Updated belief. Given the result, what do you now think? Did the hypothesis survive? Was it weakened? Was it wrong? What will you do next?

This cycle, prediction followed by observation followed by update, is the fundamental loop of learning from experience. Without the written record, the loop gets corrupted: your memory smooths the prediction to match the outcome, and you learn less than you should from each iteration.

Why expectations matter most

The expectation step is the one most people skip, and it's the one that matters most.

When you predict a result before you see it, you create a testable record of your understanding. If the prediction is right, you've confirmed something. If it's wrong, the gap between prediction and result is where learning actually happens. That gap tells you what your model of the situation is missing.

Without a recorded expectation, every result is confirming. You see the outcome and think "yes, that makes sense." Of course it makes sense, you're explaining it after the fact, and your brain is very good at explaining things after the fact. The question isn't whether the result makes sense in retrospect. It's whether you predicted it in advance.

This is the same principle behind calibration training: predict the result, check the result, adjust. Repeated hundreds of times, it builds genuine understanding rather than the illusion of understanding. Cover a paper's results section and guess the numbers from the method. Predict what your experiment will show before you look at the data. Forecast which of this month's releases will matter in two years and check your hit rate later.

The log is where all of this lives, and the log is what makes it cumulative rather than forgettable.

The log as a thinking tool

Paul Graham makes the point that an idea can feel fully formed right up until you try to put it into words. The page finds gaps your head papers over: the assumption you never tested, the step that doesn't follow, the two claims that quietly contradict each other.

A research log does this continuously. When you sit down to write an entry, you have to articulate what you're doing and why. And in that articulation, you often discover that you don't actually know as clearly as you thought. The hypothesis you thought was crisp turns out to be vague. The setup has an uncontrolled variable you hadn't noticed. The expectation contradicts something you wrote two weeks ago.

These discoveries are uncomfortable but they're exactly the ones that save you from wasting time on work built on flawed foundations. Better to find the crack in your reasoning while writing a log entry than while writing the final paper.

Rereading previous entries compounds this. Your log from a month ago contains thinking from a person who was wrong about things you now know. Seeing exactly how and where they were wrong (they being you) is humbling and educational in a way that no external feedback matches. You can see the moment your reasoning went off track, the assumption that led you down the wrong path, the result you should have paid more attention to.

Private log, public writing

The research log is private. It's rough, honest, full of wrong predictions and abandoned hypotheses. This is part of its value: you need a place where being wrong carries no social cost, because the inhibition of looking stupid is one of the main things that prevents honest documentation.

But there's a separate case for putting some of your thinking in public, and Chris Olah and Shan Carter make it well. Their essay on research debt argues that fields accumulate undigested ideas faster than individuals can process them, and that a clear explanation of a difficult concept is a contribution in its own right rather than a service job.

A digital garden or a blog that accumulates your thinking over time serves a different purpose from the private log. The log is for honesty. The public writing is for clarity, for contribution, and for the surprising fact that explaining something to others is one of the most reliable ways to deepen your own understanding of it.

Public writing is also the strongest credential that can't be faked. A body of work that shows how you think, what you're interested in, and how your understanding has evolved over time tells potential collaborators and employers more than any CV. It's a commonplace book turned outward.

Where the log lives

The log needs to be somewhere you'll actually use it daily, and somewhere you can search it later.

In Fabric, a dated note for each day's research entries, searchable by meaning rather than by date or folder, means you can find "the experiment where I tested the batch size hypothesis" without remembering when you ran it or what you called the file. Voice notes work for quick entries when you're at the bench or the whiteboard and don't want to type. The AI assistant can synthesise across your log: "What have I learned about X over the past month?" produces an answer grounded in your own documented thinking.

The format matters less than the consistency. A plain text file updated daily is better than an elaborate system used once a week. The habit is the hard part. The tools should make the habit as frictionless as possible.

The compounding effect

A research log from your first month is mildly useful. A research log spanning a year is a resource that changes how you work.

After a year, you have a searchable record of several hundred hypotheses, expectations, results, and updated beliefs. You can see patterns in your own reasoning that are invisible in the moment: the kinds of assumptions you consistently get wrong, the experimental approaches that reliably produce useful results, the questions that keep resurfacing from different angles.

Hamming's observation that knowledge and productivity compound like interest applies directly here. The daily entries look trivial in isolation. What you predicted, what you found, what you learned. Give them a year and they produce a depth of self-knowledge about your own research process that most people never develop, because most people don't write down enough to notice the patterns.

The log is also protection against the narrative smoothing that makes experienced researchers less accurate than they think. The longer your career, the more your memory has had time to rewrite. The log doesn't rewrite. It just sits there, patiently recording what actually happened, available whenever you need to check your story against the record.

Frequently asked questions

How long should each log entry be?

A few sentences to a paragraph per experiment or piece of work. The five-part format (hypothesis, setup, expectation, result, updated belief) can be completed in five minutes. Longer entries for significant results or important changes in direction are worth the time, but the default should be brief. Consistency matters more than depth.

What if my work isn't experimental?

The format adapts. If you're doing theoretical work, document your reasoning: what you're trying to prove, why you think this approach will work, what you expect the outcome to be. If you're doing qualitative research, log your interpretive hypotheses: what you expect to find in the data, what you actually find, how your interpretation changes. The prediction-observation-update loop applies to any kind of thinking, not just quantitative experiments.

Should I log negative results?

Especially negative results. These are exactly the results your memory is most inclined to smooth over or forget. A documented negative result tells you what doesn't work and why, which is often more valuable for subsequent work than a positive result that confirms what you already believed.

How does a research log differ from a lab notebook?

A traditional lab notebook documents what you did, with enough detail for reproducibility. A research log as described here adds the expectation and updated belief steps, which turn the notebook from a record of actions into a record of thinking. The lab notebook asks "what happened?" The research log asks "what did I expect, what happened, and what did I learn?"

Can I use this approach outside of research?

Yes. The prediction-outcome-update loop is useful for any work where you're making decisions under uncertainty: product development, investing, hiring, strategy. Documenting your reasoning before you see the outcome, then comparing, is the fastest way to calibrate your judgement in any domain.