@danielroe.dev I’m curious if you would be interested in making the lab data on page-speed.dev more visually prominent?
just as an example I would argue that the 56 perf on nextjs.org should have more visual weight here
Zach Leatherman 3535 replies 1717 likes Original on Bluesky35 replies
the core web vitals folk I think would very strongly suggest that lab data could be misleading compared to data from actual users
... which is why it's currently so de-prioritised (I got some very helpful advice from @tunetheweb.com before launching)
do you have a different point of view?
I should say though that any _mistakes_ in presenting the data are not barry's but mine 🙈
Yup and I stand by that advice. What users actually experience is what matters—not what some Lighthouse simulation thinks they might experience.
And if a site happens to be visited by users of slightly better devices or network, then lucky for them! And I'd argue Next's site probably fits that.
Of course there's always the risk that real users are abandoning a poorly performance site before CWV can even be recorded, but that's also difficult to measure accurately. And IMHO that would still be reflected in field metrics (even if not fully).
I also get a much higher score (85) on Nextjs.org when I test on PSI: pagespeed.web.dev/analysis/htt..., showing the variability in lab scores.
Plus I know the Next people DO prioritise performance on their site. I mean there's plenty of JS-heavy bad examples out there but this is not one IMHO.
Sure — imo Core Web Vitals measures on two axes: performance versus the relative wealth of its visitors (via the hardware they use).
You can’t use Core Web Vitals to compare the performance of two separate web sites — only Lab Data can do that — so scorecards like this are quite leaky
agreed, I don't think this is for 'comparison' vs other sites so much as 'how well am I doing at creating a performant site for my users'
yeah — right on. I’m glad you included the lab data on the page! I think both are important!
I’m curious why this result was lab-data first? page-speed.dev/fedidb.com via bsky.app/profile/dani...
No field data available:
pagespeed.web.dev/analysis/htt...this is lab-data first because it hasn't been up long enough to get CRuX data ...
ah I see — how about that design treatment for lab data when field data is available too? 😅
Regardless, a good discussion about how field data can be misleading too!
For other folks that might want more info, I did write a little bit about this here: www.zachleat.com/web/lighthou...
Arguably lab is more comparable if wanting to do that on a pure like for like comparison. But is it right to do that? A plain text page will beat a complex web app in such a perf score but to what end?
in general I couldn't be more opposed to weaponising open data like this in 'framework wars'
(or indeed, against framework wars in general)
this always needs to be about pushing ourselves to be better.
I wouldn’t say that it’s an exercise is weaponizing data — it’s more about having a stable baseline from which we can have performance discussions — we need more objective truth, not metrics that are wiggle-word friendly
no, not you!
I have seen it happen though.
comparison between frameworks on the basis of sites using them is always tricky because there are always other factors:
- type of site (app vs marketing vs personal)
- third party scripts
- business choices about what to prioritiseah, for sure — I remember the Astro blog post 😅
One benefit of lab metrics that I feel is underrated and under-discussed is the ability to understand how a site performs for devices/regions/etc that are not in its current user base. Valuable for anyone interested in broadening their reach.
Obviously field data is critical information, but candidly, a unanimously techy and wealthy audience can give almost any site "good" CWVs, while the same site might appear unresponsive on a low-end device. Field data doesn't help us answer, "would the average person think this site performs well?"
(Sorry to sealion in here :) I enjoyed the thread.)
I will never mind a little friendly backup 😅
it's a good thing that the notion of "good" in metrics doesn't vary by device or connection speed because I think generally that's true of how people perceive a service's performance. Something either feels fast or it doesn't and it's our job to aim for that good sentiment for anyone who shows up
Ohh I haven't thought of this in a long time.
It accidentally reminded me of the old but goldie study from YouTube.
TL;DR: improved perf => metrics actually worsened => folks who simply could not use YouTube before (+20mins of wait) at least now had a chance (~2mins) and started using it
a classic
On the 10yr anniversary of that blog post, Chris was my guest — and he added some details re: Feather 🪶
I had to read that twice. So a prediction of what your site experience will be like WHEN/IF you get users in APAC with their best selling device, ya?
maybe anyone outside of tech that bought a reasonably priced phone that the incredibly young clerk working at the store said was “fine” 😅
much like when we used to get clients who would say, "well that's nice but we don't actually have any users on mobile"
Offering a poor mobile experience will continue to fulfill that prophecy 😅
Oh... that sounds like the heralded Boston Globe days! AMIRYTE?
ooh, no actually I think they were unusually great about testing in all manner of scenarios. But generally in that era the sentiment was pretty common: "let's focus on the desktop experience, that's where our users are"
interesting, the performance score is a 89 for me?
page-speed.dev/nextjs.org-
result consistency isn’t a generic lab data problem — it may be a problem with this specific API but it’s not accurate to blame this on lab data
-
-
It's running on the same Vercel infrastructure as everyone else. It's a pretty simple site, although we do have some overly heavy SVGs on the homepage I'd like to replace.
