Thursday, January 12, 2012

On the 100-point scale, uncertainty and types of consumer

Normally I'm bored by discussions of the 100-point scale, but I had a brief Twitter conversation earlier this week that gave me a small new insight.

One of the objections of scale haters is that a rating is treated as an immutable truth by some of the audience, while this is not the true experience of wine.

Wine tasting has so many variables. Some wines are better with food. Wines have bottle variation. I just tasted with Georges DuBoeuf, who is not the first person to tell me some wines taste better in different weather conditions. A wine I consider a 92 one day might not even be a 90 next week, but a rating is fixed and permanent.

I give consumers more credit than many scale haters; I think many people know that a 92 means "a 92 according to one person's palate." Besides, if you can't give people that much credit -- if you think consumers are that stupid -- then I don't see why you'd want to take the grades away. But I digress.

The question, then, is whether a 92 is always a 92, even to one palate. My argument is that regardless of the fact that the answer is "no," most consumers desperately want that answer to be "yes."


Wine Spectator knows this; check out its imperious notes on how its wines are carefully rated and sometimes rerated to be checked for accuracy, as if one person's ratings can truly be described as accurate.


The thing is, those of us who care enough about wine to debate the 100-point scale are one type of consumer by definition: People who care deeply about wine. That makes us a very small minority.

Most consumers with any brains at all know that ratings of anything are just one person's opinion. Ratings don't make or break Hollywood movies or music downloads.

But wine is a very different product. While "Audition" is going to be the same every time you see it -- kiri kiri kiri! -- your '09 Joe's Winery Cabernet might be very different from your '08. Heck, your '09 might be different from the last batch, an issue few critics want to address.

However, at a certain point, a buyer has to make a decision -- and quickly, if a waiter is hovering over you. You can't spend an hour online looking up differing opinions of every movie you rent.

So you have two wines, a 92 and a 93. Is the second one better? I hope that for the critic who rated it, it was, at least when it was evaluated. But is it better for the consumer?

Nobody knows. And that's a very unsatisfying answer.

Now step back and tell me that on a different day, that critic might have given that 93-point wine a 90, so the 92-pointer might actually be better.

Me and you, wine lovers, we'll have a conversation about the nature of wine, food pairings, etc.

Our rich friend, the guy who buys all the 95-pointers but doesn't devote hours each day to thinking about wine, is going to look at his watch and say, "Order something."

The United States, inventor and popularizer of the 100-point scale, is not a country that embraces change and uncertainty on a mass basis. We're slow adopters of technology; we're much slower than Europe to enact social change. We don't quickly shift trends in music. We like variety, we like progress, and we like the possibility of a little change, but not too fast.

When 100-point-scale haters talk about how they want people to think about wine, it's generally how they think (and to a large extent, how I think). We want people to embrace uncertainty -- and we want everyone in the country to spend hours thinking and talking about wine.

Good luck with that.

Follow me on Twitter: @wblakegray and like The Gray Report on Facebook.

21 comments:

Tobias Øno said...

I prefer to view myself not as a 100pt scale-hater, nor certainly a scale-lover, but as an utter indifferent. Of course any enthusiast may chronicle a wine impression as they wish with love or zeal, but when a wine-percentile is relayed to me I react with a shrug but just can't shake the possibility that the scorer is insensitive/oblivious to bottle variation and the vast other factors that vary, and suspiciously confident in their own ability to narrow down the ultimate judgement to a tiny detail, uninterestingly so. I feel it's a bit rude to the wine, but I can't care.

I think personally the four groups of Émile Peynaud is useful and sufficient, –as you know making distinctions between wines that are undrinkable, ok, very good/great and unforgettable– which allows some space for the impression memories and some mystery.

Lee Newby said...

I prefer the WSET scale:

Faulted
Acceptable
Good
Very Good
Outstanding

Some use + or – for each level to add more detail. Each level is supported with the wines attributes such as complexity, intensity, structure, balance and finish in many combinations. The system is taught with strictly blind tasting so you get good at accessing quality.

W. Blake Gray said...

Tobias: The world of wine has changed since Peynaud's day. What he called "undrinkable" was generally due to spoilage, which rarely happens now. Even "OK" wines might have been affected by spoilage.

Now, in a twist I find interesting, "undrinkable" is most often leveled at very ripe wines that are sought after by certain portions of the market.

My point is that while in Peynaud's day most tasters could agree on where almost every wine fit in the lower three rungs of Peynaud's scale, today I think you'd get little agreement and a lot of argument about styles.

David said...

I don't hate point scales. Personally, I am indifferent to them as I only buy wines I have tasted. But, living in Northern California, surrounded by good wines easily tasted, I know that isn't the case for most wine drinkers.

If I think about them, I think they are misleading. Not only are they the subjective opinion of one person's palate on a specific day, but they are of a wine from a single bottle, on one day. Wine is not static. It changes. An 84pt wine in 2010 may be a 90pt wine in 2012 and a 76pt wine in 2013.

Using a point scale as a basis for buying a wine is like using the snap shots of a stranger's vacation to pick a vacation spot. Your experience WILL vary.

W. Blake Gray said...

David: That's a pretty good analogy, actually. I absolutely hate American travel writing because it offers no perspective -- it's as if every article is a snapshot. But you do learn something from a travel article, as you do from a snapshot, as you do from a wine rating.

Doug T. said...

I agree that the hundred-point scale is basically for people who don't know much or don't care to know much about wine. They just want some certainty, and the scale provides it. It made me think of my start into wine a couple of years ago. I was actually foolish enough to go out and buy half of the WS Top 100 thinking those wines were the best of that year! I ended up liking very few of them, but I learned the lesson! So will most consumers who come to care about wine.

W. Blake Gray said...

Doug: Wow, is that true? You bought roughly 50 of the Spectator's top 100 and disliked most? Wow.

I wish I could say how many of the Spectator's top 100 I like in any given year, but I've never taken notes. Probably few of the Pinots; I have a different opinion of what good Pinot Noir tastes like. But I think I would like the majority of their top 100, because even when our preferences differ, I do respect the Spectator's ability to identify quality within its chosen parameters. Example: If you asked me if I'd like a very ripe barrel-fermented Australian Chardonnay, I'd probably say no. But I absolutely love Leeuwin Estate Art Series and so does the Spectator.

Anyway, I think it would be interesting to offer alternate tasting notes of the entire Spectator Top 100, and give my own personal opinion. I'm going to guess that I would like at least 65 of them (I keep ratcheting that number down, thinking of the Pinots) and love at least, hmm, 30. Just ratcheted that number up: they're looking for greatness on that list, and sometimes greatness isn't tied to style preference. I expect if they think a Sauternes or Champagne or Sherry is great, I'll agree.

Without actually doing the tasting, I'll never know. That said, my estimates should give a measure of my respect for a publication with different taste than mine.

Your statement is so different: You liked few. At least now their publication has predictive value for you. If they like it, you probably won't. That's still useful criticism.

Kyle W. said...

I don't understand why you people ("What do you mean you people?") are so obsessed with using the 100 point system if you admit that there are faults with it and that people don't understand it the way you want them to. Why deal with all that drama? Why not just post tasting notes, and give your overall impression "this wine smelled of blah, tasted of blah blah blah and had a blah finish. A very nice wine for its price and something I would suggest buying for a leisurely afternoon summer lunch."
"This wine delivers bleedyblee aromas with a biggidyboop mouthfeel, very complex and interesting, with plenty of years to go before reaching its apex. This is one to buy by the caseload and cellar for 10-15 years."
That, to me, is so much better than trying to use a 100 point scale.
I hope the reason each reviewer uses the 100 point scale is more convincing than "that's what most reviewers do so it lends legitimacy to my reviews" because that is a bullshit answer.

Kyle W. said...

BTW, I didn't mean to imply that's the reason I think YOU use the scale, just that I've heard other people say that in the past. I would like to know why you chose that scale, however.

Robert C. said...

Kyle,

Bleedyblee and biggidyboop? You kiss your mom with that mouth? (Grey I thought that this was a high-brow blog.)
Overall the 100 point scale is familiar and works. Now the descriptors of the wine in the journals and magazine? Some of those descriptors I have never tasted or that many for that matter. I also do not think that many consumers will recognize the flavors or aromas described. Makes my head spin a little.

Kyle W. said...

Robert, if you prefer, I can retype it with typical wine reviewer adjectives like "sweet red fruits, a distinct earthiness reminiscent of walking through the forest after a light fall rain."
To be honest, I don't even read tasting notes that often, unless it's someone I know and trust (which doesn't include any professional reviewers, although plenty of professionals). I just don't see how reading that someone you don't know and will never meet gave a wine a 92 and thought it had certain notes could have any significance to people.

Dapz said...

Describing wine, in my opinion, is to explain in objective terms subjective variables. What are the aromas of this wine? How does it taste like? Is it balanced? I feel the rating scale is an effective tool to achieve that. The consumers are not stupid and take that grades with a grain of salt.

W. Blake Gray said...

Kyle: I use the scale because it's the most widely understood rating system. But not only that.

A few years ago I did an article for that scale, and in the course of researching it I came up with a point which is now widely agreed on but not many (if anyone) had written about at the time: the Wine Advocate, in particular, uses high ratings on the 100 point scale as a competitive advantage in marketing its brand.

The reason: Wineries will brag about their highest score, and in bragging, they are broadcasting and thus marketing the name of the rating organization.

Well, I'm a brand too.

Honest enough for you?

Jay S. Miller said...

Hello Blake
The Wine Advocate would quickly go out of business simply by deleting the scores. Although I think my former colleagues generally write excellent tasting notes and Robert Park on the front page of every issue emphasizes that they are more important than the number, readers want a shorthand. The Wine Advocate has a lengthy track record of making solid recommendations (if it did not, it would fade away), and people trust it for expertise.
I know there is a group of wine enthusiasts who wish points would go away. It's not going to happen.

Cabfrancophile said...

It's the old precision vs. accuracy dart board picture. My perception is reviewers can be accurate if they are consistent over time. Most are, even when they have preferences that are rather specific. Parker doesn't love Sine Qua Non one day then wake up the next craving a Chinon, at least as far as I can tell. He accurately assess the wines he likes.

Precision is where the problem comes in. If you have one sample, what error bars do you assign to a score? Critics assume arbitrary precision with no justification. I'd love to see an experiment with a bunch of 4-6 wine flights, with one or two wines repeated in each flight. The variance of the score from flight to flight (even better if over time) will give a sense of how precise the score is. The reviewer should not know the same wines are being recycled, though a skilled taster probably will recall the flavor profile. IF the wine has little change over time and context . . . . a big IF.

W. Blake Gray said...

Jay: Thank you for coming here and saying that. I've basically been saying those things about the Wine Advocate for years but it's nice to see someone associated with the publication say them.

You can tell where I stand on the concept of scoring wines. It's more information, and I'd always rather have more information. If people don't like it, they can just ignore it.

Incidentally, I know two well-known critics who don't score wines publicly who use the 100-point scale for their personal notes, so they can remember how much they like the wines.

Kyle W. said...

Blake, definitely honest enough for me, and I appreciate the honesty regarding branding/marketing. Now it makes more sense why someone would use it, even if it isn't flawless. Still don't know how I feel about it overall, but I do appreciate the little dropping of knowledge that I didn't have before.

Chelsea said...

I read and enjoyed a lot of wine blogs as a consumer, but the debate on this topic is very, very inside baseball. And, if the purpose of the blogs is to be a forum for communication among those in the industry, then fine. But I can't help but wonder: Is it really reasonable to expect even a very interested wine consumer to spend hours, days and months researching regions, makers, importers, etc. before buying? The only kind of consumer purchases I can think of where consumers regularly do that kind of work are on big-ticket items that don't come up much: cars, large appliances, etc.

Can wine be enjoyable to research as an avocation? Sure. But I think it's unreasonable to criticize people for liking benchmarks. The movie reviewer analogy is apt: Find a reviewer or two who help you understand whether you will like the movie (or the wine).

Like most people, I imagine, the wine I like most is from places I've been. But you can't go everywhere, so I like to use CellarTracker and WS because the notes and scores are useful to me in aligning with my own tastes.
But I buy and drink way more wine than most people.

Given that the vast majority of the market is occasional and casual wine drinkers, it makes sense to me that anything - shelf notes, helpful employees, etc. - that helps them buy a bottle they like is good for the industry. I personally think anyone making a business in mass-market wine should be highlighting Jeff Siegel's choices. Or what if retailers posted signs that said: If you like x, you might also like x. Or posted signs with Q&A or flavor profiles. I know there are those in the industry who look down on that too, but consumers would like it and they would buy more wine. And a larger percentage of them would become more interested in wine generally.

W. Blake Gray said...

Chelsea: You're a savvy reader, so you'll get this.

Who consumes this blog: Consumers of wine, or consumers of wine blogs?

Chelsea said...

That is indeed the question! It really feeds into my larger musing over the business model of wine blogs. Do the blogs themselves need to have a business model or do they function primarily as brand builders to help the writer in other enterprises? In either case, how does a narrow universe of readers aid that? Or maybe it's just the universe of commenters that's small?

I ask these mostly rhetorically, but it was front of mind after following a very long back and forth on Mike Steinberger's blog over how the top of the market is faring. Very interesting to read, but I'm not always sure what's in it for the blogger from a paying- the-rent standpoint. Something, I hope.

satomitu said...

The 100-point scale evaluation does have a lot of problem.
However,it is a good quick reference for
some luxury products that you cannot try before purchase.For many customers it would be absolutely impossible to what cabernet to choose out of 100s or even 1000s of similar products.
What matters here is that customers should know it is just another one man's point of view,and those things cannot be 100% accurate.