Compound learnings across experiments

Every experiment needs an entry before you can compound anything. If you have not built that yet, read Build a learnings log so the system never relearns a lesson first. This chapter starts the day after your log exists.

Keep the log in Notion, Airtable or Google Sheets. The tool matters less than having one place for it.

Tags let you see across tests. Use three:

• Audience: who saw it, such as new visitors, trial users or existing customers.
• Lever: what you changed, such as headline, offer, price, proof or speed.
• Place: where, such as landing page, email or ads.

Choose the words once and keep to them. If you write "hook" in one entry and "headline" in another, the pattern will hide.

Once a month, block forty-five minutes. Filter the log to the last thirty days. Read each entry aloud if you work alone, or take turns if you work in a team. Then answer one question: what shows up more than once?

For example, three of your four email tests with a specific number in the subject line beat the plain version. That is a pattern. A single result is only a result.

A belief is one sentence, plus the evidence. Make a table with five columns: the belief, the tests that support it, the tests that contradict it, your confidence and the date you last checked.

Use a simple scale for confidence:

• Low: one test, or tests that disagree.
• Medium: two tests that agree.
• High: three or more that agree, in at least two places.

Be honest in the contradict column. It is the part that stops you fooling yourself.

Cap it at fifteen lines. If it gets longer, merge the beliefs that say the same thing or drop the ones you no longer use. A page nobody reads is no better than the log.

Put it at the top of your backlog, so it is the first thing you see when planning.

Before you approve a new test, ask three things. Does a belief already answer this? Does the idea contradict a belief with high confidence? Could this test raise a belief from low to medium?

If a high-confidence belief already covers it, skip the test and apply the belief as your default. Use the time on a question you cannot yet answer. When you do pick a winner, follow Scale winning experiments across channels to carry it elsewhere.

Markets change. Set a rule: any belief older than six months, or any belief about an audience that has changed, goes back to low confidence until you test it again. This stops last year's answer running this year's business.

Paste the current page into the brief for anyone writing the next test, whether that is a colleague, a freelancer or an AI assistant. They then start from what you already know and spend their effort on something new.

• Writing beliefs from one result and calling it a rule.
• Leaving out the tests that failed, so the page looks neater than the truth.
• Tagging inconsistently, so you cannot filter.
• Skipping the monthly read-through when the month is busy.
• Treating high confidence as permanent.

You can state your five most reliable beliefs without looking. A proposed test gets rejected at least once a month because the answer is already known. Over a quarter, a larger share of your tests are about open questions, and a larger share of those tests win.