An update on review scoring

The most contentious part of any review is the numeric score that’s given to the product. which reduces our evaluation of how a product performs in what is often a very competitive space to a value on a 10-point scale. We have often talked about getting rid of our scores, but readers tell us they often look to the score first when reading a review, and rely on scores for their purchasing decisions.

As a result, we take scoring very seriously here at The Verge. We know how much weight each score has on the perception of a product and how much conversation there is about Verge scores. Reviewers pitch scores to their editors and those scores are often hotly debated before we publish a review. Our aim is to make sure that the score provides valuable information to our audience that they can use to better understand how a product performs.

As time goes on, our average score for a category of product tends to gets higher, something we call score inflation. That’s partially because of positive progress in the industry — today’s devices are better than yesterday’s — but score inflation can also change the meaning of a score. A rating of six on our scale is actually a good score — it means a product is better than average but still might have a few issues — but if readers only ever see sevens and eights, they will assume that a six is a much harsher score than it actually is. Every once in a while, we need to revisit our scoring procedures and see if we’re being critical enough across the board.

And so, that’s what we’ve done. To tamp down score inflation, we’ve started using more of the scoring scale as we’ve rated devices. That translates to seeing lower scores on reviews more frequently than you might have in recent years and devices that are better than others getting lower scores than they might have in the past. In fact, we’ve already been doing this for the past few months; if you’re a regular reader you might have noticed the change.

We’ve also moved to a 10-point, whole number scale, which we feel is simpler to understand and encourages using more of the scale than when we used a 20-point or 100-point system. Now, when we’re debating between two scores, we often will give the tie to the lower score as opposed to using a half-point compromise.

You can see our current scoring system on our How we rate page, which has been updated as well. I encourage you to read it to better understand how we review and score devices, and we will be linking to it quite often in our reviews to be more transparent about our scoring.

This does mean that some more recent scores will not align directly with older reviews. Our scoring is always a snapshot in time that compares a product to other devices that are available at the time of the review. Older devices likely would not receive the same score were they to be reviewed again months or years later.

Our commitment to you is that our reviews are based on real-world experience, leverage our extensive experience and knowledge about a device category, and provide insight into how a device functions and how it stacks up against other devices in its given category. We’ll keep making adjustments whenever necessary to make sure that we’re providing the best possible reviews to our audience.