I have just had a glass of a very nice
chardonnay from Limoux in southern France, and in order to convey to you how
good it was, I’m letting you know that I give it a score of 59! Fantastic! Oh,
by the way, that score is on my 46-point scale that ranges from a minimum of 12
to 66, the maximum.
It’s just wrong, isn’t it? But why? When a
newspaper or magazine wine columnist awards a wine a score of 95 points, we
tend to assume that it is a great wine. We all interpret this in much the same
way. That is, it is a wine that has been rated in the top 5% of all wines; a
wine that leaves behind all those in the 70s, 80s, and early 90s. But when was
the last time you saw a wine with a rating of even 80, much less one with a 75
score? Perhaps these wines never make the guides, magazine or newspaper
columns, or blogs. Alternatively, maybe these wines don’t exist. In a system
such as this, popularized by American wine critic Robert Parker, there is a
dramatic change in meaning as one moves below the 90s. While a rating of 90
means a perfectly acceptable wine, a rating below 80 effectively means
vinegar.
And? Well, it is important to recall that this
system is supposed to be based on 100 points, that is, percentages. Think about
marks at school or university. How would it feel if you scored 75 out of 100 in
an exam and were considered a failure as a result? Your sense of outrage would be
fuelled by the understandable belief that coming in the top quarter of
potential marks meant that you were much better than those who must have scored
below you.
Considering for a moment what scales supposed
to do, it is obvious that they provide an idea of the magnitude of quantities:
distance, weight, height, temperature. By and large, we never have problems
with these types of scales, a function perhaps of their longevity as useful
tools and the fact that the quantities they measure are perceived to be
properties of the ‘real, objective world’. It is when we start to measure the
intangible, the subjective – thoughts, attitudes, feelings, perceptions – that
things become a little trickier.
Why give things a number at all? Why can’t I
just tell you that my glass of chardonnay was “bloody good”? Does a Robert
Parker type score really tell you more than this? Of course it does, you might
argue, because it uses 100 points and that gives far more discrimination
between wines than a few categories. That’s true – or should be – but only if
the scoring system uses the whole range of the scale. In other words, when rating
wines, it is important to have wines that score 32, or 59, or 66. The use of
the whole range gives meaning to the differences between scores. In simple
terms, if the whole scale is used, you can be confident that a difference of
one point – wherever it is on the scale - between two different wines means a
1% difference. If I never use a score below 70, however, then that means that
the range of 70 to 100, or 30 points, defines the length of the scale, if it is
the case that 70 means terrible and 100 means as good as wine gets (but see
below). Now, a one-point difference between these two wines becomes a little
more than a 3% difference.
Parker actually explains his scale by
categorizing the score into deciles (groups of 10), except for the 90s, which
are grouped as 90-95 and 96-100. These categories start at 50-59, which as a
group are deemed ‘unacceptable’ wines. It is evident therefore that Parker’s
scale is even potentially only a 50
point scale or, if we exclude unacceptable wines, then a 40 point scale. Also,
consider if we then are putting wines into categories (either of 10 or 5 points),
then perhaps it is a 5 or 6 (if we include unacceptable wines) category scale.
But it also seems to be that we can rate within categories. So, any “average
wine with little distinction except that it is soundly made. In short a
straightforward, innocuous wine” (that is, 70-79) can be rated perhaps as more
(e.g., a 71) or less (e.g., a 78) innocuous.
This is not to say that a 100-point scale,
even if used as such, is perfect. Imagine a scenario in which you tasted some great wines – for example,
Burgundies from a recent good year such as 2002. Fantastic La Tâche – 96! Fabulous Romanée-Conti – 97! Superb Grands Échezeaux – 98! But then I have slipped in an older wine
from one of the great vintages of the last century. And it is Magnificent –
easily 4 points higher than the Grands Échezeaux. Oops. This is known as a ceiling effect, and at least part of the
problem is due to the scale being so compressed. If all really good wines have
to be given a score somewhere in the 90s (because apparently that’s where they
all live), then when comparing great wines, I am reduced suddenly to a 10 point
scale! And, unlike the group Spinal Tap
with their amplifiers (see: http://www.youtube.com/watch?v=XuzpsO4ErOQ), I cannot really just add a few points to my
scale if something nicer comes along. To be fair, this is a wider problem than
just wine rating (imagine rating different samples of Swiss chocolate), but
such ceiling effects are reduced appreciably by using the entire scale, irrespective
of what is being rated.
Part of the problem with wine ratings is that is
that it is commonly believed that the numbers have an independent meaning –
that is, they signify something about a wine, independent of other wines. Such
beliefs are a carry over from a style of judging based on identifying defects
often seen in past years in wine judging and still seen today with dairy
judging. In such quality control type judging, a high number often means a
product that is relatively free of defects. But this is not how scales of this
type work. It isn’t even the way Parker-type scales are applied, as wines with
obvious defects simply do not undergo the rating process (although the Wine Spectator magazine’s category from
75-79 is defined as including wines that are drinkable, yet still have some
minor flaws).
Another aspect of wine ratings as they are
commonly practiced is the view that wines can exist not only without defects, but
also as perfect examples of their type. In other words, the practice of wine ratings
clings to the idea of a Platonic, objective ideal of a perfect wine. If you are
a wine judge, you may have even encountered wines that live in the highest 90s.
But for the rest of us, we may not know a 99 if we drank a magnum of it …… even
if we did think that it was very good.
A major reason why we want to assign number to
things is that it allows comparison. In science, it frequently allows very
rigorous comparison via the use of statistical analysis. But we can only do
this if we know that our scale has certain properties. In rating food likes or
sensory properties, scales like the wine 100-point scale are often used and the
resultant data can be used to statistically compare products with one another.
We can seldom talk about these properties in the same way we talk about weight,
for example. It is difficult to make statements that one product is liked twice
as much as another using the scales that are commonly used. However, we can
usually talk about relative degrees of difference: so, a difference between 70
and 90 on a scale ought to mean the same as the difference between 50 and 70.
Without using the whole scale however, it is not certain that such judgments
could be made.
It is relatively common to compare foods from
different manufacturers, if they are of the same type. We can compare Swiss
milk chocolate with those from other countries, for example. But how do we
compare a French pinot noir with one from New Zealand or the USA unless they
are all made to be like one another? With wines, there are difficulties
comparing on an equivalent basis, or like with like: different methods of
growth, production, climate, season, intention, aging and so on. Which
particular combination produces the Platonic ideal wine?
Preference, and to some extent quality (if we
eliminate defects), is subjective. When a wine columnist gives a score of 96, they
are valuing certain aspects of the wine in question. We might all agree that
high acidity makes a wine tough to drink now, but what about astringency? A
dry, puckering sensation might be a characteristic of some high quality red
wines, but there are other wines that are judged as high in quality that are
much softer in the mouth. You might trust Parker to tell you whether a wine was
unbalanced or excessively astringent or that it will age well. You might even
let him tell you that there were peach notes, and chocolate aromas, or that a
wine was too redolent of “green bananas still on the tree”, but surely it is up
to you as a wine drinker to decide whether it is a style that suits you.
I recently saw for the first time a wine
columnist give two scores – one out of 100, which he labeled “empiric” (sic), and another out of 10 for
‘subjective’ assessment, by which he presumably meant how much he liked it. And
this belated acknowledgement that perhaps describing or rating a wine ought to
have something to do with preference highlights a major issue with wine
ratings. Do you like the same wines as Robert Parker? The high impact of such
an influential critic carries with it the implication that you ought.
At heart, wine ratings are based on the idea
that a perfect wine can be achieved, and that its perfection is independent of
what wine consumers think. At least part of the unnecessary complexity and
inconsistency of such ratings, as well as their absence of scientific rigor,
comes from this notion. This seems odd given that wines are drunk to give
pleasure and that may vary from person to person. On one level, I can therefore
make a case that a $10 chardonnay can be just a good as a $100 chardonnay if
your palate says that it is.
