We have to aplaud DPReview for working on a new digital camera rating system to overcome shortcommings of their existing system. As many have noticed, technological advancements eventually make it such that new cameras generally outperform older ones. Over time, the rating system gets skewed such that most new cameras get a top rating and the average camera can no longer get an average rating.
Doing better poses an interesting problem. One can’t simply rate based on a new standard each year (or any other period) because it would make comparison meaningless between generations of cameras. Imagine someone gave 5 stars to a camera like the Nikon D70 when it came out. Then when the D70S comes out, it is given 4 stars. How would you know if this is a step back, nothing has changed or is an improvment, if the rating system changed in between?
A long time ago, when Neocamera has less than 100 reviews, we re-rated all the cameras in order to comply with the goal of keeping the scale from Poor to Excellent well distributed. It turns out we discussed having this goal before but it was impossible to do again due to the number of reviews we have.
The new rating system to be established at DPReview will replace the old system using a percentage scale. It strives to keep average cameras average based on last year’s reviews and therefore is expected to progress in the same direction but at a slower pace and has room to grow upwards. This means that, as cameras get better, newer cameras will on average be better than the average score. Also, they do not plan to use the maximum 100% yet. The best current camera will receive a certain score and only an even better model will push the maximum score further. When a new model reaches a 100% score, they plan to review the system again.