research
Early signals for landing page tests: click-through finds losers in a day, time on page points the wrong way
Purchases are the right way to judge a landing page. They're also the slowest. On the paid in-app traffic we ran for a DTC apparel brand, landing pages sold to between 0.1% and 0.25% of visitors, while 3% to 8% clicked through to the store. If you wait for purchases, you wait a long time, and you keep paying for traffic on pages that are clearly losing.
So we replayed the campaign to see which early numbers agree with the purchase verdict you eventually get. A few things came out of it:
- Click-through found the losing pages fast. With a couple hundred visits per page, it named the eventual worst page 97 times out of 100. Purchases needed more than 5,000 visits per page to do that even a third of the time.
- Time on page pointed the wrong way. It correlated −0.72 with purchase rate.
- Early signals also got a pair of look-alike pages wrong. Two pages at 8.1% and 8.2% click-through ended up 60% apart on purchases, and click-through backed the wrong one at 90% confidence from day 4 to day 21.
What we wanted to know
The standard advice is to wait for statistical significance on purchases. In practice, a lot of teams check the dashboard on day three and cut whatever looks bad. Waiting spends money on pages that are clearly losing, and cutting on a handful of purchases throws out good pages by luck. I wanted to know which early reads you can actually trust, and when.
What we did
The campaign sent paid traffic from in-app ad placements to a set of landing pages for one apparel collection, with a share of visitors held back on the brand's own page as a baseline. Most of the pages had the same job, which was getting the visitor into the store to buy.
We had two sources. Session recordings gave us behavior for each visit: scroll depth, active time, taps, whether a tap hit a link, and whether the visitor clicked through to the store. The ad platform gave us every page's landings, click-throughs and purchases over the campaign.
Then we tested the early signals two ways:
- Resampling. For five pages with long histories, we repeatedly drew a fixed number of recorded visits per page (from 100 up to 5,000), computed each signal and checked how often it picked out the page that ended up with the worst purchase rate. We did that 400 times for each sample size.
- Replay. For every pair of pages that ran at the same time, we fixed the overlap window and took the purchase comparison at the end as the answer. Then we replayed the calendar. At set days from day 1 to day 35, we read each signal on everything seen so far and scored it against that answer.
Click-through finds losing pages fast
Here's how often each signal named the eventual worst page, by recorded visits per page, and how well it ranked all five pages against their final purchase rates:
| Signal | 100 visits | 200 | 1,000 | 5,000 | Rank correlation with purchase rate |
|---|---|---|---|---|---|
| Clicked out to the store | 81% | 97% | 100% | 100% | +0.64 |
| Tapped, but never hit a link (lower is better) | 50% | 63% | 96% | 100% | +0.65 |
| Visited the store within 24 hours | 0% | 0% | 19% | 45% | +0.63 |
| Purchased within 7 days | 0% | 0% | 7% | 31% | +0.54 |
| Scrolled past 20% of the page within 10 seconds | 0% | 0% | 0% | 0% | −0.12 |
| Active seconds on the page | 4% | 1% | 0% | 0% | −0.72 |
| Tapped within the first 5 seconds | 6% | 6% | 0% | 0% | −0.27 |
Click-through works because it happens 20 to 40 times as often as a purchase, and it's on the path to one. Dead taps, where someone taps something that isn't a link, were the second useful signal. The recordings explain why: most of those taps landed on the hero photo, the header and the headline text, and none of those led anywhere.
Engagement pointed the wrong way
Scroll depth, active time and early taps are what most dashboards lead with. On this traffic they ranked the pages backwards.
The clearest example was the worst page in the set. It was a selector page that asked visitors to pick an option, and each option opened a pop-up instead of taking them to the store. It had the most scrolling and the longest active time of any page. People did exactly what the page asked them to do, but the options didn't lead to the store.
Early purchase counts kept the wrong page
Early in the campaign, several pages were cut after three days with zero or one purchase each. At the campaign's purchase rate, each of them would only have been expected to get about two or three purchases in that time, so zero or one was well within luck. Their click-through was roughly twice that of a page that survived on three early purchases. That survivor later settled at the lowest purchase rate of the live collection pages.
How much traffic each signal needs
Two pages at 6.5% and 4.3% click-through separated with about one day of traffic each. Purchase rates of 0.23% and 0.10%, which is actually a bigger relative gap, needed roughly 25 times as much traffic per page. At this campaign's volume that's more than three weeks.
Over six weeks, purchases separated fewer than one in ten pairs of pages at 95% confidence. When all the pages are reasonable, most pairs can't be told apart on purchases in any reasonable amount of time.
On two pages that looked alike, the early reads were wrong for weeks
For the three pairs of pages that were very different, every early read agreed with the eventual purchase verdict on day 1, with 150 to 250 visitors per page. Purchases took anywhere from six days to three weeks to get to the same confidence.
The look-alike pair went the other way. Two collection pages had click-through of 8.1% and 8.2%. Click-through backed the second page at 90% confidence or more from day 4 to day 21, and a model of each visitor's first 30 minutes of behavior backed it from day 7. Purchases said the opposite: 2.27 against 1.42 buyers per 1,000 visitors, about 60% more on the first page, with 97.6% probability. It took 28 days of purchases to show that.
Two things explain it:
- A lot of buyers were quiet. Nine in ten visitors on every page neither clicked nor reached the store within 30 minutes of landing. That quiet group still produced a quarter to a third of the buyers on the better pages, days later. Between 41% and 63% of buyers bought after the first 30 minutes, and 18% to 38% of buyers hadn't clicked or reached the store at all in that window. Nothing you read at the end of a visit can see them.
- The clicks went to different places. The weaker page spread its clicks across nine individual product pages. The stronger one sent them to the collection.
Early reads are overconfident unless you correct for it
If you read early signals as plain statistics, they look decisive way too often. On pairs of pages that purchases never separated, click-through made a 90%-confident call on 80% to 88% of them at some point. Purchases did that on 13% to 24%.
The fix is to add an error term for the difference between pages to every early read. Across the pages with enough traffic, actual purchase rates scattered around what behavior predicted by about 15% to 22% of the predicted value, which is more than sampling noise explains. With that term in the confidence math, the big calls still settled on day 1, and the false calls on the look-alike pages never got to 90%.
The rule I'd use
- Compare like with like. Only judge a page against pages that sell the same way and are running at the same time. Traffic mix drifts: click-through for three of these pages ran 10% to 11% in their first days and 6% to 7% over their lifetime. Fixed thresholds from past campaigns would have been wrong, while the ranking between pages held.
- Cut clear losers on click-through. After about 1,000 landings, a page whose click-through is below three quarters of its peers' median, and whose purchases are at or below what its peers' purchase rate predicts, can go.
- Don't keep a page because it got one to three early purchases if its click-through is bad. And don't kill a page with good click-through on zero purchases before about 5,000 landings.
- Wait for purchases on look-alike pages. If two pages are within a point of each other on click-through, only purchases will separate them, and it'll take weeks.
- Use recordings to figure out why a page is losing. Dead-tap share, first-screen exits and which element soaks up the taps tell you whether a page is fixable (a button below the fold, a photo that isn't a link) or broken at the core (choices that lead nowhere).
Pages that sell through an on-page cart instead of a click to the store need their own early signal, like add-to-cart or checkout starts. Click-through only ranks pages that sell the same way.
A few caveats
This is one brand, one product family and one traffic source. In-app ad placements might behave differently from search, social or email traffic. Recording coverage was uneven across pages, and purchases were counted for the same visitor within seven days. The resampling results for pages that were cut early rest on small purchase counts, so the strongest statements above come from the five pages with the longest histories. How fast the early reads settled rests on three pairs of pages that were very different, and the look-alike failure rests on one pair, so all it shows is that the failure can happen.
FAQ
Can I pick a landing page winner on click-through rate?
You can pick losers. Click-through reliably flags pages that are clearly worse, usually within a day. It's not reliable for picking the winner among good pages, because pages with the same click-through can be very different on purchases.
Is time on page a good signal for landing pages?
Not on its own. In this campaign it was negatively correlated with purchase rate, because the page that held attention longest was the one that made people work without getting them to the store.
How long should a landing page test run?
It depends on the decision. Clear losers show up in about a day of traffic on click-through. Choosing between two pages that behave alike takes purchases, which took three weeks or more here. There's more on the math in how much traffic a landing page test needs.
Should I stop a page with zero sales after three days?
Check what zero means first. If you'd only expect two or three purchases in that window, zero is often just luck. Compare its click-through with the other pages to decide, and give pages with healthy click-through more traffic before you cut them.
Related
- Landing page test sample size
- Two ways people buy from an ad: why click-through can mislead when visitors aren't convinced yet
- What in-app ad traffic does on a landing page
- Lutiq Holdout Protocol
I'm Colin Behr, co-founder of Lutiq. Before Lutiq I spent years in mobile ad tech at AppLovin, Vungle and Branch. Lutiq predicts which on-brand landing page to show each paid click and measures the lift against a live holdout. The figures in this post are our own, from one brand's campaign in July to September 2026, and the brand isn't named.