Stemstore @ Posterous

Matt Skinner: Yes, I've recommended untasted wines

November 13, 2009

 

Stuart Peskett and Adam Lechmere

Jamie Oliver's head of wine Matt Skinner has admitted to not tasting several wines that he recommends in his latest book.

In The Juice 2010, published by Mitchell Beazley in the UK, Skinner lists New World wines from the 2009 vintage that were not bottled until months after the last deadline for the book.

Skinner, who is group wine manager for TV chef Jamie Oliver's Fifteen restaurant in Melbourne, was originally criticised by New Zealand wine writer Michael Cooper in The Listener magazine.

Cooper claimed that Skinner could not have recommended a New Zealand wine months ahead of publication, before Cooper himself had even tasted it.

In a statement sent to decanter.com Skinner said, 'It is imperative that I taste all the wines that I recommend.

'However there are some releases that are consistent from year to year, and as popular, good value and accessible wines I want to include them because I know that my readers will appreciate them.

'In order to do so I include non-specific tasting notes based on the current and previous year's vintage, focusing more on basic flavours and compatibility with food.'

There is no indication in the text that these notes are non-specific. The entry for the Vasse Felix Semillon Sauvignon Blanc from Margaret River, priced at £11.99, reads, '…the palate is fresh as a daisy and puncuated by the kind of lip-smacking acidity that makes this wine almost impossible to put down.'

The wines recommended from 2009 are Jacob's Creek Riesling 2009, Nepenthe Adelaide Hills Sauvignon Blanc, Vasse Felix Semillon Sauvignon Blanc from Margaret River, Brown Brothers Moscato, Unison Rosé, Hawkes Bay, New Zealand and Doña Dominga Old Vines Chardonnay from Colchagua, Chile.

Skinner's tasting notes are submitted every January, with a final revision date in May, before publication in October of the same year. Most of the wines in question are not bottled, or shown for tasting, until June or July.

Hilary Lumsden, Mitchell Beazley commissioning editor, said, 'For our first edition, in 2006, the feedback we got was that by the time people went out and bought the book, the wines were already off the shelves, so the book was effectively out of date.

'We either upset one side or the other. There's the side that wants the most up-to-date information, and there's Michael Cooper's side. The majority of the wines in The Juice don't rely on vintage variation. A lot of them are going to be consistent each vintage.'

However, in a recent GQ column, Skinner wrote: 'It's important to remember that every year is different and that no two years – even in the same spot – will ever be the same. That's the beauty of Mother Nature.'

Mitchell Beazley said in a statement, 'It is always our intention that the advice we give is accurate and reflective of the wines available.'

Mitchell Beazley is the UK's leading wine publisher, whose list of authors includes Hugh Johnson and Jancis Robinson (the World Atlas of Wine), Andrew Jefford (The New France), Stephen Brook, Michael Schuster and other major wine writers.

Matt Skinner admits that he recommended wines that he never tasted. As we mentioned in a previous blog article, we are more than a little suspicious about the accuracy of wine ratings. Notwithstanding that fact, the thought that authors could offer ratings for wines that they had never actually tasted was unexpected to say the least. These stories only serve to underline the need for wine enthusiasts to learn how to taste and keep careful notes about the wines that they taste.

Follow us on Twitter

By LEONARD MLODINOW

Acting on an informant's tip, in June 1973, French tax inspectors barged into the offices of the 155-year-old Cruse et Fils Frères wine shippers. Eighteen men were eventually prosecuted by the French government, accused, among other things, of passing off humble wines from the Languedoc region as the noble and five-times-as-costly wine of Bordeaux. During the trial it came out that the Bordeaux wine merchants regularly defrauded foreigners. One vat of wine considered extremely inferior, for example, was labeled "Salable as Beaujolais to Americans."

View Full Image

Tia Gemmell/California State Fair

Wines are poured at the California State Fair wine competition in June 2008.

It was in this climate that in the 1970s a lawyer-turned-wine-critic named Robert M. Parker Jr. decided to aid consumers by assigning wines a grade on a 100-point scale. Today, critics like Mr. Parker exert enormous influence. The medals won at the 29 major U.S. wine competitions medals are considered so influential that wineries spend well over $1 million each year in entry fees. According to a 2001 study of Bordeaux wines, a one-point bump in Robert Parker's wine ratings averages equates to a 7% increase in price, and the price difference can be much greater at the high end.

Given the high price of wine and the enormous number of choices, a system in which industry experts comb through the forest of wines, judge them, and offer consumers the meaningful shortcut of medals and ratings makes sense.

But what if the successive judgments of the same wine, by the same wine expert, vary so widely that the ratings and medals on which wines base their reputations are merely a powerful illusion? That is the conclusion reached in two recent papers in the Journal of Wine Economics.

Both articles were authored by the same man, a unique blend of winemaker, scientist and statistician. The unlikely revolutionary is a soft-spoken fellow named Robert Hodgson, a retired professor who taught statistics at Humboldt State University. Since 1976, Mr. Hodgson has also been the proprietor of Fieldbrook Winery, a small operation that puts out about 10 wines each year, selling 1,500 cases

A few years ago, Mr. Hodgson began wondering how wines, such as his own, can win a gold medal at one competition, and "end up in the pooper" at others. He decided to take a course in wine judging, and met G.M "Pooch" Pucilowski, chief judge at the California State Fair wine competition, North America's oldest and most prestigious. Mr. Hodgson joined the Wine Competition's advisory board, and eventually "begged" to run a controlled scientific study of the tastings, conducted in the same manner as the real-world tastings. The board agreed, but expected the results to be kept confidential.

There is a rich history of scientific research questioning whether wine experts can really make the fine taste distinctions they claim. For example, a 1996 study in the Journal of Experimental Psychology showed that even flavor-trained professionals cannot reliably identify more than three or four components in a mixture, although wine critics regularly report tasting six or more. There are eight in this description, from The Wine News, as quoted on wine.com, of a Silverado Limited Reserve Cabernet Sauvignon 2005 that sells for more than $100 a bottle: "Dusty, chalky scents followed by mint, plum, tobacco and leather. Tasty cherry with smoky oak accents…" Another publication, The Wine Advocate, describes a wine as having "promising aromas of lavender, roasted herbs, blueberries, and black currants." What is striking about this pair of descriptions is that, although they are very different, they are descriptions of the same Cabernet. One taster lists eight flavors and scents, the other four, and not one of them coincide.

View Full Image

Photo illustration by Donna Kugleman/The Wall Street Journal; Getty Images (bottle); Alamy (puddle)

A smashed red wine bottle on white background.

That wine critiques are peppered with such inconsistencies is exactly what the laboratory experiments would lead you to expect. In fact, about 20 years ago, when a Harvard psychologist asked an ensemble of experts to rank five wines on each of 12 characteristics—such as tannins, sweetness, and fruitiness—the experts agreed at a level significantly better than chance on only three of the 12.

Psychologists have also been skeptical of wine judgments because context and expectation influence the perception of taste. In a 1963 study at the University of California at Davis, researchers secretly added color to a dry white wine to simulate a sauterne, sherry, rosé, Bordeaux and burgundy, and then asked experts to rate the sweetness of the various wines. Their sweetness judgments reflected the type of wine they thought they were drinking. In France, a decade ago a wine researcher named Fréderic Brochet served 57 French wine experts two identical midrange Bordeaux wines, one in an expensive Grand Cru bottle, the other accommodated in the bottle of a cheap table wine. The gurus showed a significant preference for the Grand Cru bottle, employing adjectives like "excellent" more often for the Grand Cru, and "unbalanced," and "flat" more often for the table wine.

Provocative as they are, such studies have been easy for wine critics to dismiss. Some were small-scale and theoretical. Many were performed in artificial laboratory conditions, or failed to control important environmental factors. And none of the rigorous studies tested the actual wine experts whose judgments you see in magazines and marketing materials. But Mr. Hodgson's research was different.

View Full Image

Chris Wadden

In his first study, each year, for four years, Mr. Hodgson served actual panels of California State Fair Wine Competition judges—some 70 judges each year—about 100 wines over a two-day period. He employed the same blind tasting process as the actual competition. In Mr. Hodgson's study, however, every wine was presented to each judge three different times, each time drawn from the same bottle.

The results astonished Mr. Hodgson. The judges' wine ratings typically varied by ±4 points on a standard ratings scale running from 80 to 100. A wine rated 91 on one tasting would often be rated an 87 or 95 on the next. Some of the judges did much worse, and only about one in 10 regularly rated the same wine within a range of ±2 points.

Mr. Hodgson also found that the judges whose ratings were most consistent in any given year landed in the middle of the pack in other years, suggesting that their consistent performance that year had simply been due to chance.

Mr. Hodgson said he wrote up his findings each year and asked the board for permission to publish the results; each year, they said no. Finally, the board relented—according to Mr. Hodgson, on a close vote—and the study appeared in January in the Journal of Wine Economics.

"I'm happy we did the study," said Mr. Pucilowski, "though I'm not exactly happy with the results. We have the best judges, but maybe we humans are not as good as we say we are."

This September, Mr. Hodgson dropped his other bombshell. This time, from a private newsletter called The California Grapevine, he obtained the complete records of wine competitions, listing not only which wines won medals, but which did not. Mr. Hodgson told me that when he started playing with the data he "noticed that the probability that a wine which won a gold medal in one competition would win nothing in others was high." The medals seemed to be spread around at random, with each wine having about a 9% chance of winning a gold medal in any given competition.

To test that idea, Mr. Hodgson restricted his attention to wines entering a certain number of competitions, say five. Then he made a bar graph of the number of wines winning 0, 1, 2, etc. gold medals in those competitions. The graph was nearly identical to the one you'd get if you simply made five flips of a coin weighted to land on heads with a probability of 9%. The distribution of medals, he wrote, "mirrors what might be expected should a gold medal be awarded by chance alone."

Mr. Hodgson's work was publicly dismissed as an absurdity by one wine expert, and "hogwash" by another. But among wine makers, the reaction was different. "I'm not surprised," said Bob Cabral, wine maker at critically acclaimed Williams-Selyem Winery in Sonoma County. In Mr. Cabral's view, wine ratings are influenced by uncontrolled factors such as the time of day, the number of hours since the taster last ate and the other wines in the lineup. He also says critics taste too many wines in too short a time. As a result, he says, "I would expect a taster's rating of the same wine to vary by at least three, four, five points from tasting to tasting."

View Full Image

Tia Gemmell/California State Fair

Ribbons from the 2009 California State Fair wine competition.

Francesco Grande, a vintner whose family started making wine in 1827 Italy, told me of a friend at a well-known Paso Robles winery who had conducted his own test, sending the same wine to a wine competition under three different labels. Two of the identical samples were rejected, he said, "one with the comment 'undrinkable.' " The third bottle was awarded a double gold medal. "Email Robert Parker," he suggested, "and ask him to submit to a controlled blind tasting."

I did email Mr. Parker, and was amazed when he responded that he, too, did not find Mr. Hodgson's results surprising. "I generally stay within a three-point deviation," he wrote. And though he didn't agree to Mr. Grande's challenge, he sent me the results of a blind tasting in which he did participate.

The tasting was at Executive Wine Seminars in New York, and consisted of three flights of five wines each. The participants knew they were 2005 Bordeaux wines that Mr. Parker had previously rated for an issue of The Wine Advocate. Though they didn't know which wine was which, they were provided with a list of the 15 wines, with Mr. Parker's prior ratings, according to Executive Wine Seminars' managing partner Howard Kaplan. The wines were chosen, Mr. Kaplan says, because they were 15 of Mr. Parker's highest-rated from that vintage.

Mr. Parker pointed out that, except in three cases, his second rating for each wine fell "within a 2-3 point deviation" of his first. That's less variation than Mr. Hodgson found. One possible reason: Mr. Parker's first rating of all the wines fell between 95 and 100—not a large spread.

One critic who recognizes that variation is an issue is Joshua Greene, editor and publisher of Wine and Spirits, who told me, "It is absurd for people to expect consistency in a taster's ratings. We're not robots." In the Cruse trial, the company appealed to the idea that even experienced tasters could err. Cruse claimed that it had bought the cheap Languedoc believing it was the kingly Bordeaux, and that the company's highly-trained and well-paid wine tasters had failed to perceive that it wasn't. The French rejected that possibility, and 35 years ago this December, eight wine dealers were convicted and given prison terms and fines totaling $8 million.

Despite his studies, Mr. Hodgson is betting that, like the French, American consumers won't be easily converted to the idea that wine experts are fallible. His winery's Web site still boasts of his own many dozens of medals.

"Even though ratings of individual wines are meaningless, people think they are useful," Mr. Greene says. He adds, however, that one can look at the average ratings of a spectrum of wines from a certain producer, region or year to identify useful trends.

As a consumer, accepting that one taster's tobacco and leather is another's blueberries and currants, that a 91 and a 96 rating are interchangeable, or that a wine winning a gold medal in one competition is likely thrown in the pooper in others presents a challenge. If you ignore the web of medals and ratings, how do you decide where to spend your money?

One answer would be to do more experimenting, and to be more price-sensitive, refusing to pay for medals and ratings points. Another tack is to continue to rely on the medals and ratings, adopting an approach often attributed to physicist Neils Bohr, who was said to have had a horseshoe hanging over his office door for good luck. When asked how a physicist could believe in such things, he said, "I am told it works even if you don't believe in it." Or you could just shrug and embrace the attitude of Julia Child, who, when asked what was her favorite wine, replied "gin."

As for me, I have always believed in the advice given by famed food critic Waverly Root, who recommended that one simply "Drink wine every day, at lunch and dinner, and the rest will take care of itself."

—Leonard Mlodinow teaches randomness at Caltech. His most recent book is "The Drunkard's Walk: How Randomness Rules Our Lives."Printed in The Wall Street Journal, page W6

This article does a terrific job of pointing out some of the fundamental issues that we have always had with wine ratings here at The Stemware Store. Wine ratings have gone from being a useful tool that helped consumers compare products on a shelf to a dispositive judgment of the actual value of each and every rated bottle of wine. This wouldn't be an issue if not for the fact that the human sense of taste is so inexact and subjective.

The article makes a point of mentioning that studies have shown that while the human tongue can only differentiate four simultaneous tastes, some judges/ wine critics attribute as many as eight different flavors to specific wines. It's just hard to believe that the science around the nature of human taste is completely wrong. Instead, the more likely explanation is that wine judges are making informed, intelligent guesses that should be loosely followed.

The wine buying public either needs more proof that the scores offered by these judges have some degree of accuracy or the judges themselves should couch their decisions with an appropriate level of honest disclosure regarding the limitations of their critiques. It would be even better if wine enthusiasts started trusting their OWN sense of taste. We always recommend that customers take the numerical wine ratings and convert them into the letter grades that they use in elementary school. For example, if a wine has a rating of 97, this wine has achieved an A+. We think that converting the number into a letter grade makes more sense and allows for the subjectivity inherent in the judging process.

8Share

Yes, really. Rupert Murdoch’s crusade to blame Google for the failing newspaper business model continues today, as it emerges that News Corp has conducted talks with Microsoft about de-indexing the company’s sites from Google and (presumably) being paid to include them in Bing instead.

The concept makes sense only if you buy Murdoch’s claims that Google (

) is “stealing” content rather than simply helping people find it.

The revelation comes from the Financial Times, which has a strong track record for accurate reportage – this is unlikely to be a fluffy rumor. The piece reads, in part:

Microsoft has had discussions with News Corp over a plan that would involve the media company’s being paid to “de-index” its news websites from Google, setting the scene for a search engine battle that could offer a ray of light to the newspaper industry.

The impetus for the discussions came from News Corp, owner of newspapers ranging from the Wall Street Journal of the US to The Sun of the UK, said a person familiar with the situation, who warned that talks were at an early stage.

However, the Financial Times has learnt that Microsoft has also approached other big online publishers to persuade them to remove their sites from Google’s search engine.

News Corp and Microsoft, which owns the rival Bing (

) search engine, declined to comment.

I say go for it. So, I’m sure, do all the other web publishers who see that removing many of the major news sites from Google will provide even more traffic for the upstarts. News Corp is merrily making itself irrelevant to web consumers, while continuing to use Google as its punch bag rather than addressing the radical transition of media into the online world.

Remember when the music industry started chasing single moms and college students? Remember how you couldn't shake the feeling that it was going to end badly? We got the same feeling after reading this article. I wonder if newspapers will still exist by the time my 13 year old graduates from high school. If they keep making these kinds of decisions, I would place the odds at just about even...50/50.

Okay, more tricks involving alcohol. This is a clip of a gentleman opening a bottle of Syrah by banging the bottle against a tree. Again, one of the great uses of the Internet is sharing important, useful information. This kind of information sharing makes civilization stronger.

I liked this one SO much, we tweeted it twice. We can't help it, we just love little drinking tricks. This one is a bit obvious, but we share it with our friends on Posterous because it just seemed like the right thing to do.