Experiment Design: Does Refreshing Old Posts Move Rankings? Our Pre-Registered Test

Does refreshing old posts move rankings?

The evidence we could read does not say, and this page exists to fix a test that can. The figures on the pages that rank for this topic are single before-and-after cases or correlations, and none of the text we read describes pages left untouched for comparison.

• The claim to test: pages that get a substantive refresh gain more average position than similar pages left alone, over the same 12 weeks.
• The design: eligible pages are randomly split into two arms, so season, search updates and normal decay hit both.
• The safeguard: the metric, windows, threshold and decision rule are published here before any result exists, so none of them can be chosen afterwards.
• The status: the rules are fixed, the page list, seed and start date are not yet recorded, and no results exist.

For the general method of sizing and calling a marketing test, the marketing experiments playbook gives the template this protocol follows. This page applies it to one search question.

What is a content refresh, and what counts as one in this test?

Surfer uses that definition and adds that a refresh is usually cheaper than a new page because the URL already has history, links and impressions. Siege Media separates a refresh from a content update: an update is minor revisions such as correcting errors or adding recent statistics, while a refresh also reworks insight, readability and alignment with current search intent.

That distinction matters for a test, because "refresh" is a bundle of changes, not one. This protocol tests the bundle as defined in the treatment section below. It does not test any single element, such as a new title tag, and it cannot say which part of the bundle did the work.

What does the published evidence on refreshes actually show?

It shows that refreshes are widely recommended and that decay is measurable, but not that a refresh causes a ranking gain. The table lists each figure the supplied sources give, the kind of evidence it rests on in the text we read, and what it cannot tell you.

None of the five rows has a comparison group. The closest thing to a method is the Siege Media study, which pulled the top 10 results for each keyword through the Google Search API and read the last-updated dates in the snippets. That is a careful snapshot of dates, but it is a snapshot, not an intervention.

Two numbers on the Siege Media page do not match. Its text says 17,749 keywords were analysed and its table totals 17,805, a gap of 56, and the text gives nearly 234 million monthly searches against a table cumulative volume of 283,202,230. We use neither as a precise input, and we use only its 730-day average, which the page works out as 730 ÷ 365 = 2 years.

Why is comparing traffic before and after a refresh not enough?

A before-and-after comparison is not enough because every other change in the same weeks lands in the same numbers. Seasonal demand, a search update, new links and normal decay all move a page's traffic whether or not it was refreshed.

Take decay alone. Animalz reports an average weekly decay of -1.21% in a 2018 analysis of AdEspresso data. Compounded over 12 weeks, an untouched page would keep 0.9879¹² ≈ 0.864 of its traffic, a fall of 13.6%. Adding the weekly losses without compounding gives 12 × 1.21 = 14.5%.

The table shows why that matters. Both scenarios are invented, and in both the refreshed pages are measured the same way. Only the untouched pages differ.

In scenario A the naive reading sees nothing, although the refresh held pages level while the others decayed. In scenario B it sees a win that the untouched pages beat. The same refreshed result can mean opposite things, which is why this test needs the comparison arm.

A second trap is regression to the mean. If you pick pages because they recently dropped, some of them rebound with no help, and the rebound looks like a refresh effect. Random assignment from a single pool, with decline status recorded, puts that rebound in both arms.

What does this page register, and what is still open?

This page registers the rules of the test, not yet its sample. The publication timestamp on this page, 27 October 2026 at 18:45 UTC, is the registration time for everything marked fixed.

The commitment is simple. No page is edited until the three open items are recorded here, and the fixed items are not changed afterwards except for typographical corrections, which will be listed.

Pre-registration earns its place by closing the choices that let a result be bent after the fact. The table lists each one and what this page fixes. It is the author's own list, built from the choices this particular test would otherwise leave open.

What exactly is the hypothesis, and what is the null?

The hypothesis is that pages given a substantive refresh gain more average position than comparable untouched pages over the same period. The null is that the two arms change by the same amount.

difference = median change in position (refreshed) − median change in position (untouched)

Position is read the way Search Console reports it, where a lower number is better, so an improvement is a negative change and a good result is a negative difference. The test is two-sided, because a refresh that makes things worse is also an answer.

Which pages qualify, and how are they split?

A page qualifies if it is an indexable article last substantively edited at least 12 months before the first edit, and has at least 30 organic clicks in the 12-week before window. The 30-click floor keeps the before figure from being too small to compare, and both numbers are the author's choices.

• Minimum sample: 30 eligible pages, 15 per arm. If fewer qualify when the sample is frozen, the test is not run and this page says so.
• Stratum: each eligible page is recorded as declining or stable, using the Animalz trigger of an organic traffic drop of more than 20% over 90 days. Pages are split at random within each stratum.
• Method: the list is sorted, a published random seed shuffles each stratum, and alternate pages go to each arm. The seed is posted before any edit.
• Recorded, not used to split: query type, such as how-to or comparison, because Siege Media's data shows average days since the last update running from 400 for "best" keywords to 1,420 for "calculator" keywords.

The sample is small, and that has a consequence. We have not run a power calculation, because the input it needs is the spread of page-level changes, which we do not yet have for these pages. With 15 pages per arm the test can catch a large effect and will miss a modest one, and the sample-size section of the marketing experiments playbook explains why.

What is the refresh that the treated arm receives?

The refresh is a fixed checklist applied to every page in the treated arm, with the first three items required. The list is drawn from the steps in the Animalz and Surfer guides, trimmed to what a single editor can apply consistently.

1. Replace or remove every statistic, date and tool reference that is out of date, and attach a named source to those that stay.
2. Repair or replace broken internal and external links.
3. Compare the page with the current top results for its main query and restructure it if the expected format or intent has changed, which both guides stress.
4. Align the title tag, H1 and meta description with the page as it now reads.
5. Add what the top results lack, such as an original example or data, instead of adding words. Animalz warns that padding creates copycat content.
6. Change the displayed date only if the content changed meaningfully, as Animalz advises.

Each page gets a change log. The URL does not change, and neither arm receives any other edit, including new internal links, during the before and after windows. A page assigned to the treated arm is analysed in that arm even if its refresh turned out lighter than the checklist, and the log reports how many items each page received.

Which metric decides the test, and over what window?

The deciding metric is the change in a page's average position in Search Console between the 12 weeks before the edits and the 12 weeks after. Position is chosen because the title asks about rankings.

Surfer lists impressions, clicks, click-through rate, average position and top queries as the search metrics to collect. Here, clicks, impressions and CTR are secondary: reported in full, with no power to change the verdict.

All the edits ship inside a two-week edit window. The before window ends the day before it opens, and the after window starts the day after it closes.

Average position has a known weakness, and it is the author's inference. A page that starts appearing for extra, lower-ranked queries can show a worse average position while its visibility improves. The secondary metrics are there so that case is visible, and a conflict between position and clicks will be reported as a conflict.

The choice of 12 weeks differs from Animalz, which suggests judging a strategy within 30 days. Our reasoning is that 30 days of one page's Search Console data is a thin base for an average, and the extra weeks cost only waiting. Pages with no impressions in either window have no position and are reported by arm, not silently dropped.

What is the decision rule?

The decision rule has four outcomes and is written before any data: supported, contradicted, not detected, or not run. It uses a permutation test, which reshuffles the arm labels many times to see how often chance alone produces a difference as large as the one observed.

The threshold is the smallest change we would act on, in the same spirit as the smallest lift worth acting on in the experiment template. Animalz notes that dropping from first to second can cost roughly half the traffic, so one place is not trivial at the top, though it is a design choice for pages across the range.

A verdict applies to this sample only. A supported result would mean refreshed pages moved here, not that refreshing works for any site.

What can go wrong, and what happens if it does?

Four things can disturb the test: a search update during the windows, contamination between arms, technical changes to a page, and a data gap. Each has a rule that is written now.

• A search update. It hits both arms, which is the reason for the control arm. Any update announced during the windows is listed with its date and the result is not adjusted.
• Contamination. A refreshed page can take traffic from or give links to an untouched one. Internal links between arms are logged and the interaction is reported, not corrected.
• Technical changes. A page is excluded, in either arm, only if during the windows it is redirected, merged, deleted, set to noindex, has its canonical changed or its URL moved. The number excluded per arm is published.
• Data gaps. If Search Console data is missing for part of a window, the gap is reported and the page stays in with the data it has.

What the test does not measure matters too. It does not measure citations in AI answers, which the Animalz guide treats as a second kind of decay. Nothing here queries an AI system, and a result about position says nothing about that channel.

How to read the pages that did not respond

Pages that did not move stay in. If a refreshed page falls while an untouched one rises, that is part of the spread the permutation test already accounts for, not a reason to remove either.

What is documented and what is analysis?

The sources supply the definitions, the triggers and the figures quoted, and everything about the design is the author's. The two lists keep them apart.

• Documented: Surfer's definition of a content refresh, its list of search metrics, its 6 to 12 month review advice and its positions 11 to 20 and declining-performer cohorts.
• Documented: Siege Media's distinction between an update and a refresh, its 730-day average across its keyword set, its per-type averages quoted above and its method of reading snippet dates.
• Documented: Animalz's -1.21% weekly decay and 55% traffic gain from a 2018 analysis of AdEspresso data, its 42% quarterly-versus-annual claim, its 20% over 90 days trigger and its advice on dates and padding.
• Analysis: the hypothesis, the 30-page minimum, the 12-month and 30-click eligibility rules, the two-week edit window, the 12-week windows, the 1.0-place threshold, the 0.05 level and the stratified split.
• Analysis: the compounding arithmetic, the two scenarios, the list of open choices and the reading of average position, all based on invented numbers or the author's reasoning.
• Not supplied: any controlled result on refreshes, any primary search-engine documentation, and any Kanso result. A page titled The Advanced SEO Testing Guide for 2025 appeared in the search results supplied for "seo test design". We did not read it, and nothing here draws on it.

When will results appear, and where?

Results will be added to this page, below the protocol, with their date, once the after window closes. No date is promised here because the first edit has not been scheduled.

Whatever the verdict, the published result will include the page counts per arm, exclusions, the difference with its permutation p-value, the secondary metrics and the per-page change log. If the test cannot run or ends inconclusive, that goes on this page too. A hub for experiment logs and a playbook on choosing between updating, merging, redirecting and deleting are planned but not published yet, so there is nothing to link.

In the meantime, two published pages help with the earlier steps. The striking-distance Search Console workflow covers pages ranking just off page one, and the seven-step organic traffic drop diagnostic covers what to check when a page falls.

Content refresh experiments: frequently asked questions

Sources

• Content Refresh Strategy: How to Update Old Content for SEO and AI Search, Animalz, retrieved 27 October 2026
• Content Refreshes: How Often Are They Needed? [Data Study], Siege Media, retrieved 27 October 2026
• 5-Step Content Refresh Guide for High-Performing Blogs, Surfer, retrieved 27 October 2026