A Closer Look at “The Wine Trials”

Yes, I know, we have some pending business. But the New York weather has been so gorgeous recently that I can’t help thinking about German riesling. Every spring, when the weather really begins to turn, I start thirsting for German riesling. This year, though, I was thinking about dry rieslings because I’ve had a fair number of really good ones in the last few months. You can read my column on dry German rieslings in the newspaper on Wednesday.

Now, let’s get back to our discussion of “The Wine Trials,’’ the forthcoming book by Robin Goldstein that has drawn so many comments in the last week. As I mentioned last time, Robin was kind enough to send me the galley and now I’ve read the book. To a certain degree I regret my original post, because I hadn’t read the book yet. But I jumped in because of the Newsweek article that grabbed my attention. I don’t regret assailing the article, which took a gotcha tone toward wine lovers, but I think the book itself is considerably more nuanced, as I suggested last time.

Now, I’m not going to review the book in depth, but I will offer a few opinions. First, the most useful component is an in-depth listing of 100 wines $15 and under that scored well in the trials. These are all bottles that should be widely available without having to find a specialized store. They represent a cross-section of mass-market wines that many people who don’t spend all that much time thinking about wine – the book calls them “everyday wine drinkers’’ – can drink happily. My favorite parts of the listings are something I’ve never seen before, design critiques of each label.

Is this useful to anybody who is more than a casual wine drinker? Not really, and that is one of the contradictions of the book. “The Wine Trials’’ makes the case that people’s buying decisions are influenced by many factors, including price, marketing, power of suggestion, etc. The book suggests that if you take away all of these factors and make buying decisions strictly on the grounds of what tastes best in the glass, everyday wine drinkers prefer cheaper wines to more expensive wines.

Of course, it’s more complicated than that. Let’s say first out, that the study shows that everyday wine drinkers prefer cheaper mass-produced wines to more expensive mass-produced wines. None of the wines considered would fall into the artisanal, natural or hand-crafted categories.

Second, as hard as the book tries to make the tastings objective, to eliminate outside influences, one unaddressed fact is the mass-tasting environment. It’s a completely unnatural way to taste and to judge wine. Rather than drink wine in a natural environment with food, wines are pitted against each other, sipped and spit, one after another. I personally think wine is experienced in a different way when it’s consumed with a meal over time, and that’s one reason why, in this blog, I do not write about wines that I’ve only sipped in tastings.

Journalists have contradictions, too. I participate in mass tastings with the wine panel even though I don’t believe they are the best ways to judge wines. It’s a compromise because it may be the only way to judge large numbers of wines as we are often called upon to do. These are old arguments that I don’t want to go through again now.

More to the point, “The Wine Trials’’ makes it clear that what appeals to everyday wine drinkers is significantly different than what appeals to wine experts, which the book defines as those who’ve had some sort of training or professional experience with wine. The experts in general preferred the more expensive bottles, and the book suggests that the tyranny of the experts may be unconsciously compelling everyday wine drinkers not to follow their own preferences and to waste money doing so.

Meanwhile, the book assails publications like Wine Spectator and Robert M. Parker Jr.’s Wine Advocate for consistently giving higher scores to more expensive wines. These publications are directed to people with more than a casual interest in wine, so is it really that surprising that these critics would have such a preference? Especially since the preference was echoed by the more experienced wine drinkers who participated in the book’s tastings?

Several other things bother me about “The Wine Trials.’’ First, I think the book is way behind the wave in overstating the power of Parker and the Spectator in shaping the sorts of wines we drink. “Fueled by the magazine critics, who often seem to reserve their highest ratings for big, concentrated wines, the New World style is poised to wipe the Old World style off the map in the realms of both expensive and cheap wine,’’ we are told.

And this: “Parker is famous for preferring Bordeaux to Burgundy, and his power is such that many top Burgundian wines are currently marginalized in the marketplace when compared with top Bordeaux wines. As a result, even in Burgundy there is now a push to make wines in a more concentrated, oaky style, which is totally antithetical to the region’s historical character.’’

Marginalized in the marketplace? Try looking in the sale bins for a good Burgundy. The Burgundians are doing great! If anybody is marginalized it’s the small vignerons all over Europe who are trying to compete against the mass-produced wines that are promoted by this book. By the way, Parker’s critic for Burgundy these days is David Schildknecht, who is very much respected in Burgundy and everywhere else in the wine world.

Frankly, the notion of a Manichean wine culture was overstated in the movie “Mondovino’’ and it’s overstated here. Of course there is tension between corporate marketers and the public. One wants to separate the other from its money, and isn’t too particular about the tactics it employs. “The Wine Trials’’ uses Dom Pérignon as its prime example of a wine that has been vastly overvalued through marketing. It’s not a bad example, yet the book misses the point by suggesting that Dom Pérignon is popular with everyday wine drinkers. Please. Dom Pérignon is well known to everyday wine drinkers, but very few have actually tasted it, unless by everyday the book means wealthy conspicuous consumers. Veuve Cliquot would be a much better example of a Champagne that is wildly popular because of advertising and marketing, but that seriously lags in quality.

In my opinion, the solution is not the $11 bottle of sparkling wine from Washington State, as good a value as it might be. The solution to the conflation of wine with luxury goods is right there in Champagne, with the momentous movement of small grower-producers who are not only changing the public perception of Champagne but changing the way some of the bigger corporations are going about their business. Similar self-generating solutions exist in many other regions.

The bottom line is that wine is a far more complex picture than the one painted in “The Wine Trials’’ – at least it is for people who love wine and are seriously interested in it. For those whose interest stops at picking up the occasional bottle for dinner, there is little need to complicate that picture.

Think of visiting a museum of modern art. Those who don’t study art, and rarely think about it, will have a much different reaction to, say, a Jackson Pollock, than somebody who’s greatly interested in modern art and knows a bit about Pollock and his contemporaries. Many people might have to do some reading and research on their own, or even take an art history course, to begin to understand what critics and art historians saw in Pollock. Some might conclude it’s all hogwash, but others might gain a deep and rewarding appreciation.

I’m not saying wine is the equivalent of art. I do say that wine can be appreciated on many different levels, but that nobody should ever feel obliged to appreciate wine on any level. In the end, the book seems to divide wine consumers into the casual buyers who are pushed this way and that by forces they don’t understand, and the wealthy conspicuous status seekers who also are not quite aware of capitalism and marketing. Unacknowledged are the serious wine lovers who are knowledgeable, experimental and passionate, and who, yes, are in control of their own destinies. The book may speak to the first two groups, but not to the third.

Comments are no longer being accepted.

If you are still thinking of German Riesling on May 7, Eric, join us for Wine Blogging Wednesday 45:

//winecast.net/2008/04/11/old-world-riesling-for-wbw-45/

Cheers!

Eric

I have not read the book. I have read the paper associated with it (published by the American Association of Wine Economists) and I’ve been thinking and looking at it with people much smarter than me in the area of statistics.

The description of the dataset used in the paper and the statistical manipulations used to analyze that data for the paper seem to indicate that *not all participants may have tasted the same wines*. 506 subjects X 523 wines = 264,638 observations, not 6175.

The authors seem to have done some statistical manipulations – to predict how certain members of the study group would have rated wines they did not taste? If so, this means they projected how certain subjects would have rated specific wines of different price points – wines that they did not actually taste (!!!).

With price, by itself, not at all being an indicator of style, and preference for one style over another being the determinant of how a subject will rate a wine, how can one accurately project how any subject will rate a wine when the only working variables are an enjoyment rating and a per bottle price? After all the question of the study was to: *“investigate the relationship between price and subjective appreciation”. *.

And, can we really say that the preferences of the casual drinkers and the ‘experts’ (12% or 60 or 61 out of 506 people) are “significantly different”? That is a misinterpretation of this study I am seeing in many discussions. There is a difference between two pieces of data being very different and the differences between them being “statistically significant”.

I propose that the subjects really pretty much split 50-50 about each wine. For the ‘experts’, the scales tipped a *little bit* (the authors use the word “slightly”) in favor of the pricier wines and for the whole combined group they leaned a *little bit* towards the cheaper wines. But it was a close call.

If you have more concrete evidence of these dramatic differences as found in the book, please share them. Repeated reading of the research paper supports my contentions.

Statistics is like a language. Just like prose, it can be employed to make a statement or frame an argument. It can also be used to mislead. I am not threatened by any of the conclusions of the Goldstein group because I have too many misgivings about the methodology of the study to go along with their conclusions.

How can you go from :”a bunch of people tasted a bunch of wines and they tended to slightly favor cheaper wines” to “[this means] that wine recommendations by experts may be poor guides for non-expert wine consumers”. Especially when your question going is was: what is “the relationship between price and subjective appreciation”? The only connection they make is that critics tend to rate expensive wines higher…. But they do not test preference of the subject group against the actual ratings these wines received. At least they make no indication of this.

To assail any critic is to be blind to the truth that the critic draws his power of influence from people who agree with them. Parker did not rise to prominence through delivering a divine revelation. He did it by validating the preferences of a large group of people. One can argue that a cult of personality evolves thereafter, but there had to be enough resonance in the public to let the critic gain critical mass of influence.

This is a Marketing and Economics study, it is not an investigation of human sensation. All things serve a purpose. So the question is: does this book serve a direct financial purpose for it author or does it push an agenda from which a third party (not named in your post) can benefit. If the answer is “yes” to the latter, who benefits from the implications, lessons and the authors’ conclusions.

Soon or later we will learn, like it happened with the Mc Nuggets, that in mass produced wines the taste is built with chemicals and that there is actually a little percentage of must in it, like there were more bones then meat in the Mc Nuggets. I would like to do another test focusing on how the participant felt the next day after they drank a bottle of manipulated wines compare to a non manipulated wine. That does not mean that all inexpensive wines are manipulated not at all but in order to get those you still have to rely on an expert because often those wineries do not advertise and are not written up by any critics.
Buona Bevuta a Tutti
de-vino.com
de-vino.blogspot.com

Agree, Art for sure!

I’ve been drinking wine now for over 60 years, as a mere consumer, and I’m still reading wine articles…. Schoonmaker to Asimov…. It’s been a pleasant and fun ‘artistic’ journey.

Still no real surprise for me here. Tastes do develop and I think learning more about how people get to a place where they enjoy more complex tastes in their beverage would be a more interesting study.

I see the development of taste in many of my friends (and self). I’m a bit of a beer evangelist, sharing and introducing people to stronger more complex beers. When I have a large party the masses will drink more pale sweet largers. When I work to introduce a quality Belgian Tripel to people many tend to stay and not go back to lighter beer. Some will try a richer beer, make a face, and never try again … staying where it’s taste safe.

If not exposed to richer tastes people will not choose the more complex offering at that moment. I think that is what I’m reading about this study.

East Village Wine Geek April 23, 2008 · 8:37 am

This book seems like a contradictory, overgeneralized mess. One more person boldly trying to figure wine out once and for all. The wine and art comparison makes sense. Does one like abstract or classical? and if you like one category over the other and one evolves quicker and with more exposure one may or may not like the general phase the public is accepting at that time. Wine is the same way. Sideways came out and everyone hated Merlot. I believe the, “serious wine lovers who are knowledgeable, experimental and passionate, and who, yes, are in control of their own destinies,” are on the rise. Parker and Spectator will always have some purpose in the general public but they will hopefully always be a stepping stone as well for the emerging third category.

EvWg

If the premise of the book was a study that compared the quality of wine to the price then it would be difficult to sell. Some controversy is required. I remember taking a Sociology class in college where the question was asked “Think of a color” and everyone wrote down their answer. Over 90% of the responses were red or blue. In wine classes we will ask the audience to think of a “Pinot Grigio” and 90% will respond “Santa Margherita!” Such is the power of branding and marketing and the unseen hand of cultural influence. When we ask if the class liked a wine or not we will usually get a mixed response and many times about 50%/50%. When asked why some will say that the wine is too sweet and others may consider it medium dry. This could easily happen in a discussion of Champagne or Rieslings. Is the wine balanced? Does it have a long finish? What are the aroma and flavor profiles? What type of food would you pair it with? These qualitative nuances are not statistical in nature but key to understanding wine. There is a new theory that reports that palates vary and describes four major palate models and their preferences. I’m sure most of you have given the advice to “trust your own palate” when looking for wines that satisfy. If only wine samples were readily available to try before you buy then we wouldn’t be disappointed with our choices. Rare is the wine that will satisfy my wife’s taste and my own and we both love red wine. She always says she likes a full bodied wine. If it’s too tannic she shakes her head no. If it has some leather or barnyard then it’s not for her. Can you now imagine both of us filling out our wine scorecards in a crowded restaurant and of course me forgetting to spit? I’ll have another glass of Blackstone Merlot please….

This on-going discussion has had me thinking for a few days about what Eric posted in the first blog about “Wine Trials” that alluded to reverse wine-snobbery. It seems timely considering the reemergence of “culture wars” in the media and the current political atmosphere. Wine is, I believe, a good indicator of “cultural capital,” and it is also interesting in discussions of taste, preference and economics.

I don’t want to comment too much about the book since I’ve not read it but the issue of wine in the price to quality to popularity ratio is something I see everyday working at wine shop. Namely: why is there such trepidation among “casual wine drinkers” when questions of knowledge and preference and price come up? The best answer I’ve come up with is that wine occupies a unique place as a commodity.

Wine is purchased in two places (wine shops and restaurants/wine bars) and for years there was little to no retail specialization. Fancier restaurants had bigger wine lists w/ more expensive bottles and better shops offered more variety and probably more “world class” wines. But the average consumer went to the same basic place as the wine enthusiast to buy a bottle for dinner. I’ve had a hard time thinking of another commodity where the high-end and low-end of prices and quality are normally, and often only, offered in the same shop.

So for decades consumers in the U.S. were left in the hands of a very few “experts” when it came to selecting wine. Unfortunately I believe many of these experts(sommeliers, retail clerks/owners) used this to their own economic advantage by insisting that price was the best way of determining quality. Obviously this has changed drastically in the last 15 years and a large percentage of the wine market is now aimed at producing and selling a consistent, inexpensive product to as many people around the world as possible.

So . . . many young, casual wine drinkers are used to getting cheap bottles that taste relatively similar and many older, casual wine drinkers have long held suspicions that the “experts” are duping them. Hence the reverse wine-snobbery and the difficulty in explaining the vast nuances in style and wine-making approach, and how prices are established and why some expensive wine is actually worth it and maybe even a value.

People who want designer label skirts or suits that are tailored for them don’t go to Target, and those who want a good everyday $20 skirt or shirt don’t go to Bergdorf Goodman. And although there is increasing specialization in some retail and restaurant wine destinations most people still have a neighborhood wine shop where they go for Tuesday dinner wine and for Christmas dinner wine.

So the best thing for consumers, as usual, is to arm themselves w/ knowledge and information. But there really is “too much” to know about wine and so we’ll always need good, honest guides, critics, journalists, sommeliers, servers and retail clerks to help out. There’s nothing wrong w/ admitting someone knows more about a subject than you, and there’s also nothing wrong with saying “but I still prefer . . .”

Let’s not gloss over the question the book raises about acquired tastes. (It’s clear that people can learn to like things that most of the rest of us find pretty distasteful.) Point is, it’s likely that the “wine experts” in the trials could easily distinguish the styles of the more popular mass market wines from the more expensive wines.

Another thing, do any wines that are not made by the thousands of cases each year make it across the Atlantic into the US wine distribution system? At that production level, do they really qualify for terms like artisanal or handcrafted? Highly trained viticulturists and enologists are employed at every level of wine production nowadays.

In response to Mr. Ruttle’s question, “do any wines that are not made by the thousands of cases each year make it across the Atlantic into the US wine distribution system?” Yes. And a wine of which a couple thousand cases are made is much different than one of which a couple million (or even much more) cases are made.

Arthur Przebinda writes: “The authors seem to have done some statistical manipulations – to predict how certain members of the study group would have rated wines they did not taste? If so, this means they projected how certain subjects would have rated specific wines of different price points – wines that they did not actually taste (!!!).”

The short answer: no, we didn’t. I suggest you read the paper CAREFULLY before you make sweeping and speculative statements about how we handle the data. Feel free to contact us directly if you have serious questions.

Beau Rapier also generalizes…”And a wine of which a couple thousand cases are made is much different than one of which a couple million (or even much more) cases are made.”…yes, but not necessarily better simply because it is from a smaller run/batch. I’ve tasted crap from small, “boutique” wineries as well as wines from larger runs by large wineries. A great example of the conundrum is the “Two Buck Chuck” at the Calif. State Fair last year. I was doubtful at first, but honestly upon tasting that wine…at the Fair itself, it was apparent that it was not only A superior chardonnay but THE superior chardonnay as judged. Assuming the questions of a “bait and switch” by Franzia/Trader Joes (I could see a Fred Franzia, as he is notorious in the industry, pulling such a stunt BUT I can’t see TJ’s going along with it) are put aside, it was clearly a very tasty wine at a mass-produced volume.

The simple fact is that each wine must be judged on its own and volume in and of itself MAY have an impact on the quality. But I’ve tasted some Napa wines with extraordinarily high reputations and price tags with 15.5% Chardonnays (will go nameless here but you WOULD recognize the name, I assure you), I wouldn’t serve as cooking wine.

I think if you understand the basic process of wine making then you’d agree that there is inherently a different way of making wine in mass quantities than in small (a million cases vs. a thousand etc.). What works for one way wouldn’t work for the other. If how the wine is made is inherently different then I’d say the final product is different as well. But I never said better. I think taste and preference are to exhaustive to get into here.

In Europe most wine laws are based on how and where the grapes are grown and in what quantity their vines produce fruit (yields). They seem to think that smaller yields and tighter restrictions on production areas (and weird stuff like hand harvesting) make for wines of higher quality. But that’s just Europe.

In California they like to market “boutique wineries” as more expensive options to TJ’s. I’m sure examples of both are available now in Disneyland’s California Adventure. And I agree wholeheartedly that a great example of Conundrum is Two Buck Chuck.

Dave Erickson (The Wine Mule) April 23, 2008 · 1:12 pm

So for decades consumers in the U.S. were left in the hands of a very few “experts” when it came to selecting wine. Unfortunately I believe many of these experts(sommeliers, retail clerks/owners) used this to their own economic advantage by insisting that price was the best way of determining quality.

Obviously this has changed drastically in the last 15 years and a large percentage of the wine market is now aimed at producing and selling a consistent, inexpensive product to as many people around the world as possible.

The first paragraph is slander, pure and simple.

The second paragraph is true, but hardly grounds for congratulations.

Johan (#11)

What is the LONG answer? Please, we have the time, humor us.

I would prefer that you answer for everyone’s benefit the following questions:

1. Did all tasters taste exactly the same wines?
2. How many wines did each taster taste?
3. Did you think to account for individual preferences for different styles of wines rather than price alone ($100 can buy two vastly different wines, stylistically speaking)?
4. If so, why are there 6175 observations for 506 subjects tasting 523 wines (and not 264,638)?
5. What ARE an “ordered probit estimator” and a “linear estimator (OLS)”, and what purposes did they serve in your study?
6. How did you account for the fact that $100 can buy two vastly different wines, stylistically, even though both may receive high scores from critics?
7. Why is it not mere conjecture on your part to say that since critics tend to rate high priced wines better and the cohort found the less expensive wines MINIMALLY more enjoyable that critics are not useful guides for consumers?
8. Why you did not account for individual preference in wine style in this study? That is a factor infinitely more closely linked to how the wines were rated than price.

In science, claims must be defended and explained if we are challenged.

If you seek to inform with your study, you are obligated to explain the methodology and the results and withstand the scrutiny of critics.

This is a suggestion for a future article. Since Costco is the nation’s biggest wine seller, it would be wonderful if Mr. Asimov could review some of the wines sold there.

Dear all,

The paper is available at this URL:
//www.wine-economics.org/workingpapers/AAWE_WP16.pdf

Most of Arthur’s questions are answered in the paper and/or in Robin’s book. The paper is technical (it refers to things like ordered probit and OLS, two standard tools for anlyzing data), the book less so.

The smartass answer to this study is: “If your experiment needs statistics, you ought to have done a better experiment.” (Ernest Rutherford)

The realistic answer is that the correlations pretty much stink, and (as the abstract states) that price is not a clear indicator of whether people like a wine or not. The positive and negative correlations discussed in the paper seem to have very little meaning compared to the overall conclusion.

Overall, the experimental design looks a bit suspect to me, as mass-marketed wines are designed to be palatable, therefore are likely to get blase OK or Good ratings. I’d be interested in what the distribution of the answers was. I’d suspect that the “Bad” and “Great” answers were somewhat rare, and the variability may be greater for the answers in outlier wines.

Aren’t there 3 types of lies: lies; darn lies; and statistics?

My personal contention is, and has been, that you are not properly addressing the fact that there is a profound difference between, say, a classic Bordeaux and a cult Napa Cabernet sauvignon – although the two may be priced identically and may (or may not) receive similar ratings from experts. However, individual preferences for either of those styles would be more closely linked to how your subjects rated wines than price. I don’t believe that Fixed Effect addresses this sufficiently.

I have tried to engage Johan to elaborate on that as well as the data, methodology and conclusions as is customary for all research papers released to a larger audince.

Instead, he tells us all “go read the paper” – as if he deems me to be some half-wit not worthy engaging in an intelligent exchange. I have read the paper – numerous times and I find his response insulting and haughty.
I have read this paper and consulted with others who are statisticians and we have questions. If we do, is it not reasonable to expect that others might as well?

I like Gabrio’s comment about asking the participants how they felt the next day — which is about as good an indication as any that a wine is sound or unsound.

More telling, however, is his last thought, that the high-quality, low-cost small wineries don’t advertise (they can’t afford it) and (therefore?) aren’t reviewed.

It’s shocking, most shocking.

Actually, I thought “go read the paper” was the most reasonable of possible responses.

It seems to me that individual preferences was not included in the study because the study was not about individual preferences. It was about the relationship of price and taste. I’m not saying it wouldn’t be interesting to see a study about whether people who prefer chardonnay like expensive chardonnay more than cheap chardonnay, just that it isn’t what the authors were studying.

Steve, are you able to answer all 8 of my questions (#15) after reading the article?
If so, that’s great! But I and several others who understand the technical stuff Johan mentions (and its nuances and implications) had several unresolved questions after repeated readings.

Enjoyment *is* very closely tied to preference. If you go to a movie and spend $10 expecting to see an action drama (because, let’s say, that that is what you prefer over all other types of movies) and the movie is a sappy chick-flick comedy, will you have enjoyed (and rated) that movie as much?

Consider the question with this assumption in mind: when given a choice between an action drama and a chick-flick comedy you will pick the action drama every time.

Arthur,

If anyone here is being insulting and haughty, it is you.

The endless stream of gotcha’s in the media stemming from experiments on humans using wine is little bit exhausting, isn’t it?

Does anyone recall the proclamation — just earlier this year — that people resoundingly prefer the more expensive wine to the cheap stuff.

Granted, it was a totally different experiment incorporating perception and marketing, but I find that traditional media can never frame scientific studies adequately enough to guide wine consumers from one “breakthrough finding” to the next “gotcha!” This blog post is admirable for stepping up (yet again).