It’s been a while since we had something really good to debate here. Here’s one that has been on my mind for a week now.
I was reading a story in Decanter magazine about a controversy centered around how special, highly regarded, wine critics are allowed access to taste–and rate–Bordeaux wines en primeur (in advance) of the wines being bottled and released.
(The story actually is more complicated than that. There is an official en primeur tasting of Bordeaux wines for wine writers, but a very select few are allowed access to those wines even before everyone else. They are immediately rating the wines and posting them up on their social media sites like Facebook–before the other wine writers even get to taste the wines.)
One obvious issue here (besides the fact that some writers are more privileged than others) is that ratings of these wines, prior to being released, will influence the wine producers regarding the price they charge for their wines. If James Suckling gives a First Growth Bordeaux a perfect 100 based on his en primeur tasting, you can be certain that the wine, when eventually bottled, will command a high price.
So, this got me thinking. En primeur whisky tastings are a regular privilege for the most highly regarded whisky writers. We have access to barrel samples when we tour distilleries. Even more so, we (on a regular basis) get samples of whiskies well before they are released to the public.
In this new age of social media, where we can blog, tweet, or post up on Facebook our rating as soon as we get a whisky (often months before a whisky is actually released to the public), what impact does this have on consumer demand? Equally as important, what influence does this have on the whisky producers when they decide what price to charge for their whisky?
This, in itself, is a good question. But I’ll take it one step further. Some whisky writers actually taste, rate, and publish ratings of the “new make” spirit–right off the stills and long before it is aged for three years and can legally be called whisky. Are these ratings based on the quality of the whisky as it is, or is it based on its potential?
For example, in the 2010 Whisky Bible, Jim Murray rates two “New Spirit” Kilchoman samples from Islay (one from a bourbon cask and one from a combination of bourbon and sherry casks) a 93.5 and 94, respectively. Is this based on the actual whisky that was tasted, or its potential? I ask this question because in his 2011 Whisky Bible, he rates two actual whisky releases (including the inaugural release) and gives them an 85 and an 82.5.
I understand why wine writers rate based on potential. Wine ages and changes when aged in a bottle. And with some wines, they can “close up” and become worse before they get better and peak. Theoretically, whisky doesn’t change at all when it is bottled.
If we start rating whisky based on potential and not actual flavor, imagine what would that do!
So, I have two questions for you:
1) How do you feel about whisky writers tasting and rating whiskies long before they are released (and priced)?
2) How do you feel about rating a distiller’s spirit (or whisky) based on potential?
Let the discussions begin.
From what I have heard, “potential” is French for, “You haven’t actually done anything yet.” Reckon that about summarizes my opinion on this one.
Ratings are funny. I value many, yourself included, opinions but I prefer to reserve my own opinion. I will often taste something before I’ve read either a review or the distiller’s “notes,” then see if the opinions line up with my own. But to address potential, I think a “finished” product is preferred and potential should not influence price, demand and reputation should take care of the price factor. I have had the privilege of tasting a living whisky, but it was already 40+years old! I think the bottling will be very close.
I don’t mind whisky writers tasting and rating things well before they are released. I’ve noticed that my own personal preferences line up better with some writers than with others. If Generic Whisky Reviewer X and I generally like the same whiskies, and I see he loves/hates a bottling that will be coming out soon, I know to make an extra effort to find it/not waste my time. I think the pricing is less an issue than in wine, where many buyers will take a case or more at a time. Also, it seems that in general wine writers are more influential than whisky writers are. Plenty of people won’t buy a wine rated less than a certain point score. I’ve also read about stores struggling to sell wines that haven’t received at least 90 Parker points. I haven’t observed the same problem in whisky.
As for rating on potential, I don’t think it’s a good idea for whisky. Scoring should reflect the whisky as it is tasted, not what it might be after 15 more years in oak. It’s fine to note that a whisky is likely to improve/suffer from more time in wood, but please don’t project what it might become. That sort of projection seems like it would be much easier for wine, whereas whisky is often manipulated in ways not seen in wine (special barrel finishes).
First about wine testing and en primeur. The whole Bordeaux Futures market is based on those advanced testings (I actually bought some 2009 Bordeaux Futures based mainly on wine scores) and they are critical to the wine purchase. I have to admit though that I am more interested in some “cult” American wines,
As far as whisky is concerned, I already know which whisky writers have taste preferences similar to mine so I would base my purchase on their findings. I regret that whisky writers do not have the clout of Robert Parker of Jancis Robinson. Many whisky drinkers are much more individualistic than an average wine collector
I do not think that one can judge the potential of new make spirit without knowing how (and how long) whisky is going to be matured.
“I regret that whisky writers do not have the clout of Robert Parker of Jancis Robinson.”
I dearly hope they never do.
1. If the expert pre-release ratings are influencing the actual release price (lower or higher depending upon rating) is there an ethical dilemma on behalf of those elite tasters – i.e. are they violating their supposed neutrality by influencing the initial offering price via the producer-distributor-retailer chain? Perhaps it would be better if the release price was established prior to tasting and/or releasing (to anyone including the producers) pre-release ratings.
2. Could be beyond silly or possibly interesting/fun to see how well early previews match actual maturation/bottling ratings. Need to add disclaimer that it’s a potential rating and may not match (could be worse or better) than the actual release rating. It’s done all the time in sports, stocks, and other industries…
1). Bad idea for the consumer in the long run. See how Bordeaux prices have escalated astronomically in the last 10 years? More focus has shifted to the En Primeur tastings and advanced scores. I think it has become quite absurd that release prices are now heavily based on whatever score Robert M. Parker Jr. decides to give to a barrel sample- when the barrel sample itself can have very little to do with the end product in bottle (barrels haven’t been blended yet, and sometimes Chateau owners will pick out the best barrel samples for critics to taste during En Primeur, knowing that higher scores will attract more customers). So you can see the potential for dishonesty here. And that’s why every year now at En Primeur, it can be quite a circus.
2). There can be so many variables in the aging process, I don’t really see the point of rating based on potential. Most of us know what the potential is anyway for most whiskys, given the track record of each distillery. So why complicate it further by giving it a potential score? Again, just rate the end product- the product we’re buying.
Hi John, tricky issues methinks. I for one always ask if it’s a ‘pre-production’ sample or not and I always try to taste only what comes at least from ‘commercial’ bottles, even when those aren’t available yet. I’ve already been burnt with preprod samples that were fairly better than the ‘commercial’ whiskies in the past. I’ve also checked ABVs that were higher than the commercial version’s (I have some distiller’s tools at home, such as densimeters). For our MM awards we refuse ‘sample bottles’ since a few years and only take ‘normal’ bottles that we uncork ourselves.
I’m not suggesting bottlers try to fool us, not at all, but for many of them, ‘a 12yo Glenthis’ is ‘a 12yo Glenthis’, if you see what I mean.
Happy weekend
Serge
There is even some doubt about en primeur tasting in wine circles:
http://www.jancisrobinson.com/articles/a20110406.html
(the article is available for all)
Nice story. Thanks for the link!
I think that so long as the reviewers are up front about the context of their comments, reviewing any sample at any stage of production can be valid. I would stress however that without context, that is to say that a score of 95 for new make can only be compared with other new make scores, we risk comparing apples to oranges. I would hope that Jim Murray would add this caveat to his new make scores.
I think reviewers access to a whiskey before it is released is dependent on whether what the reviewer is tasting is the finished product; e.g. tasting whiskey pulled from the barrel or after the bottling has taken place. I think if a reviewer is going to put a rating out there to the public, it should be on a product that has already completed the bottling process. Second, rating new make doesn’t really mean anything to me since there are so many other variables that go into the finished product. Type of wood, level of char, age, rick house placement, seasonal fluctuations, etc. all determine the final outcome. Rating new make in my opinion places the cart before the horse. If a reviewer wants to make some first impression comments, that’s different, but don’t rate it.
Tasting whisky before it has finished its maturation and the determination to bottle it has been made is pretty useless to me. Unless you were nosing and tasting from the same cask or set of casks on a continual basis for several years before bottling maybe you could make some determination of what the final product would be but otherwise its pure guesswork. There are too many variables in play to guess the potential (maybe Murray’s rating changes reflect that)
If, on the other hand the maturation process has been completed and you are nosing and tasting a whisky right before it is going to be bottled that might have some value. Frankly I wouldn’t pay much attention otherwise.
1) I don’t mind as long as the writer (or distiller) makes note of the fact. I think back to your first review of the wihisky (yes, no “e”) that became Maker’s 46. That was an interesting example because you actually had an input on what went into the production bottles. You and the distiller were forthcoming in the process, and the end product reflects your influence.
2) We read that two barrels filled the same day and warehoused side by side for XX years will produce slightly different whisky when the spirit is bottled. So for a writer to say that new make has this or that potential is simply a guess at best.
I must add this caveat: when you publish your notes on a young, 43%, chill-filtered whisky you often add that it might have scored higher if it had more age, was bottled at a higher percentage, and not chill filtered. That, to me, speaks of potential, and even if it’s true,I rather tire of the phrase. If the whisky earns an 82 based on what the distiller put in the bottle, so be it.
The Maker’s 46 did have a happy ending. But, I will tell you that I learned a lesson the hard way with this experience. When I was first approached by the master distiller, it was my assumption that what I was being sent was the final version. Then, when I tasted it and wasn’t overly thrilled about the whiskey, suddenly I was informed that some final “tweaking” was still possible.
Had I known that, I wouldn’t have accepted the sample. I only want to receive the final version of the whiskey.
Good man, John.
Reviewing whiskey based on potential… Considering most whiskey gets blended with many different years and barrels, from different parts of the warehouse, I think this is a ridiculous idea. Probably a ridiculous idea for wine as well, but then again, who am I to say.
When talking about Scotch Whisky, I have heard that some distillers take a mediocre whiskey and age it longer to get more maturity and barrel flavor. Maybe a preliminary tasting would help there.
Who knows, maybe this would feed the competition between “new make” spirits. Could be a boon to both white dog and micro-distillers.
Just a few thoughts.
I don’t think you can compare it. It’s like comparing grapes to barley. Grapes are highly affected by the growing conditions, and the fruit makes up 80% of the flavour of the wine (yeast and barrel being the others.) Whiskey though is more influenced in the quality of wood and the conditions aging.. probably to a reverse percentage. That being the case.. as soon as you have the juice in.. you really know if you have good potential. You can make great wine out of great grapes, but you can make awful wine from great grapes too. You can never make great wine out of bad grapes.
I am a wine maker and a grape grower (not commercial.)
I have to say, I don’t see any debate here. The purpose of reviews are to help us consumers find the good stuff and put it in context. It only makes sense that when a distiller realizes they’ve got something special, they’ll want to raise the price. That’s just good ‘ol capitalism. What’s the alternative? No advance reviews?
Regarding potential, if potential can be predicted I think it’s worth discussing it. I have yet to hear someone taste new make and predict exactly what it’s going to taste like in 30 years, my understanding is that is impossible. I could be wrong.
Yes and let’s remember that good wines are usually sold way before they should be consumed, which explains why people seek ‘advice in advance’, whilst whisky is usually sold when ‘ready’ (in theory!)
Excellent point. Most of the 2010 Bordeaux Cabernet (or even Merlot) dominant wines will not be ready to be drunk before 2017 or even much later.
Since wine ages in the bottle, generally speaking you are always buying ‘potential.’ Wine will continue to evolve until the bottle is opened, to at least a small degree every wine score speaks of the potential of other bottles that will not be opened until later. Whisky, on the other hand, is sold ready to be poured and you are buying a finished product. Bottles opened at a later date should taste very much like the one the reviewer tasted. So I have no problem with advance scoring for wine but don’t see the point for whisky.
However, with publication lead times it makes sense for the print press to get advance bottles. Samples, as Serge notes, can be unreliable, and in my opinion should not be reviewed.
Given the very important, but still relatively minor influence of the make on final flavour as compared to the barrel it is matured in; the other whiskies it is vatted with; the blending formula (for single malts as well as blends); wine finishing; etc., scoring new make as whisky is onanistic at best.
One of the big problems with rating any whisky these days is the seeming competition among scorers to be quoted on shelf-talkers. Some scorers appear to compete to be the one to assign the highest possible score without being ridiculous and thus get the producer to advertise the expertise of the scorer. This is why I went to stars and am considering abandoning those as well.
If I could add, the most useful wine scores are those that predict when the wine, properly cellared, will be at its peak.
1)I don’t mind tasters getting access to whiskys before the rest of us. Those of us who seek out and read early reviews of products we’ve not had understand what reviewers we may have similar tastes too or has a good track record of rating about where you would. But in the end the great thing about taste is that the only one that matters is your own.
2) I think it’s fine to talk about potential. It really comes down to context. If you are given a private early sampling, something that’s not the final product, I see no reason to give it a rating. You could speak of what hypothetical rating you would give it if it were the final release however. As for rating a whisky higher because it has potential, I disagree with it. Call it like you see it, and speak of the potential it has if so driven too.
This is a really good topic and as I thought about it throughout the day I came to several different conclusions, but the one that I settled on parallels a different industry:
If you reviewed a movie based on a trailer – I will ignore your opinion.
If you reviewed a movie based on a prescreening, chances are the movie will be 90-100% similar to what is released to the general public, so your opinion is going to range from “almost the same” to “exactly the same”.
If you show your movie at Sundance and everybody loves it, declaring it will make millions and win an Oscar – then the buzz is going to play into/inflate the selling price.
If a good pre-release buzz gets the booze amped up in the price department, well I can’t blame them. They’re using an expert opinion to attenuate their selling strategy – no different than any other industry really. When it comes to reviews, I think that as long as it is clearly expressed what context the drink was drunk, then I can use my own judgment as necessary in validating their opinions.
I think that raters in general have flawed rating systems, but that is another discussion entirely.
Only if they don’t change the ending of the movie…
I don’t really have a problem with ‘en primeur’ whisky tastings. I only wonder how these illustrious and privileged persons feel about being involved in a new release at whatever stage: are they whisky writers or consultants? It is, of course, perfectly possible to be both but it certainly confuses the matter of any reviews generated thereafter.
I don’t believe that whisky writers approach new make in a speculatory way – I struggle to see how such a thing would be possible given the prevalence of oak on maturation. I know that for me personally I love the opportunity to nose or taste the pure spirit when I visit a distillery but I view it as a drink in its own right – a distillate of the DNA of the distillery concerned. The new make, after all, is that which the distiller has created for its individuality, a manifestation in high-strength form of the various little nuances lent by the mash tun, fermentation policy, still shape and condensing materials before it is put into a cask any other distillery could have used. This original state is something I think Dave Broom explores particularly well in his Whisky Atlas and provides an impressive degree of insight into the essential character of the whiskies we will drink after a minimum of three years.
Your ideal reviewer:
A friend of yours with a trustworthy palate, who’s connected & gets to taste a lot of whisky, and is always ready to discuss his impressions with you and make recommendations– recommendations for the benefit of you, his friend, not for the benefit of his industry connections.
I judge reviewers by how close they come to this ideal. I’m a lot less interested in the reviews of folks who are regular and fully certified elements in the pricing process of a whisky producer.
Answers is reverse order. Rating new spirit is at best a guess. We have all been told that the barrel adds so much to the end product, so I don’t see what point is made by rating the spirit. Especially with finishing thrown into the equation, so even regular updates can be easily rendered worthless.
As to advance access, it’s a very slippery slope when reviewers serve as unpaid consultants in excahnge for special privileges. And I would not have confidence, i.e. translating into purchasing, in a distillery that can’t figure out what is required to move the product from the barrel to the bottle.
I regularly review spirits in advance of release on Drinkhacker and, on at least one occasion, I have heard that a manufacturer has actually changed the recipe for a product (not Maker’s 46, btw) based on my pre-release review. I wasn’t informed of this until the revised product hit the shelves. This surprises me, but if it leads to a better product, I’m not really opposed to it. Being part of the process isn’t really my job, but it gives writing about spirits a different and somewhat more interesting angle.
As for how my ratings shake out, once I get to most of the whiskeys I cover, I presume they are already set and ready for bottling (or are functionally at that point) — and that only the packaging is usually in question by then. But no matter what my review is always based on what’s in the bottle I’m sent, not the “potential” of said bottling. Potential is one thing, but I can only rate what I taste. Your notes about Murray dropping ~10 points on Kilchoman in the fashion you describe because, perhaps, his perceived potential didn’t pan out should alarm any reader.
1) How do you feel about whisky writers tasting and rating whiskies long before they are released (and priced)?
I’m always happy to read opinions of any whisky writers but I am aware that there is risk in life. I’m not looking for somebody to ‘protect’ me from making a buy I may later regret. It’s part of the learning process.
2) How do you feel about rating a distiller’s spirit (or whisky) based on potential?
The more the merrier…
The rating practices you describe remind me strongly of the late (I hope it’s dead!) style of rating figure skating preformances: it was done by watching prior performances, the practices, and sometimes the initial program of the three or four any competitor has to prepare.
The judging was by and large based on how the individual judges *thought* the skater or pair would do, no matter what the performance merited. It was worse with certain judges: it was a very annoying known property that took several scandals before the marking system was reformed.
Of course, it’s now completely confusing for some of us.
I do listen to folks whose tastes are similar to mine, no matter who they might be, in trying to decide where my limited malt dollars go.
Marina in Campbell CA
I don’t think it hurts to read the tasting notes, as well as “projections” of what the whisky or whiskey will taste like eventually as long as it is clear that is what the reviewer is doing. Giving a new make a rating of 94 because of potential and not making that clear in the review is misleading the reader and shouldn’t be encouraged but will ultimately hurt the writer as we will stop believing or caring what he/she writes, though in the short term may also hurt our pocketbooks.
Rating anything on potential is useless as even the best nose and palate are not able to look into the future. The only rating method that guarantees full comparability is to judge anything according to the same standards. The 94 potential of newmake will crumble to dust if it is filled into a bad cask.
Pre-release reviews are a good thing as long as they are describing the product that is actually sold.
Great topic, John. This site is fantastic. Thank you.
As difficult as it can be to locate some of these fine potations, a pre-release review can be a nice way of deciding whether or not to search high and low for a rare bird. Either way, there is no avoiding that reviews heavily influence the industry (think of Buffalo Trace’s grail project which is noble in itself but I find it puzzling they want someone else to determine when they have perfected their whiskey). Beethoven’s critics had opinions of his 9th symphony ranging from “monstrous” to “sublime”, but he knew he had composed something special—even if he could not hear it! The immense influence of reviews does raise a few other questions for me:
Is it preferable to have one individual or a panel review a spirit?
Should the critic(s) blind or partially blind taste?
And most interestingly of all to me: how are the points awarded?
Quite obviously in discussing gustatory matters there is a large aspect of subjectivity. (Is this alone a rationale for a panel such as in San Francisco, something more intersubjective?). Most reviewers use a holistic scoring method (“I give it a 95 of 100″). This score reflects an overall impression and the language surrounding the score is a description of that impression, good or bad. What about using an analytic rubric in which the score represented an analysis of the different qualities of a good whisk(e)y? You could have several categories (nose, body, finish, etc.) and add a subjective category that would account for no more than say 20% of the overall rating? My wife and I developed a similar system for wine to aid us in our Bacchic pursuits and have enjoyed this immensely. Anyhow, I am very curious to know what others think.
JDW
You have some valid points-
En Primeur tasting should be defined as to the stage of the whisky at tasting and should contain a caveat that ” the tasting notes represent current findings which may differ from the final product “.
This is necessary because of variables such as cask/s used, age of maturation, etc.
Speculation(rating a spirit based on potential) should never be included in tasting notes- to me that is a travesty. Speculation has its place in an article or blog where it is identified as such, where a case is made for the speculation.
My personal preference for a rating system is the usage of A, B ,C or D similar to the educational system grades. There is so much variation in taste that allocating a specific number to a whisky or wine for that matter, should not be an absolute. Unfortunately, this serves as a basis for “pricing” in most cases.
Derek: I could not agree with you more about speculation.
The difficulty with any grading system is the impossibility of eliminating subjectivity. Whether the critic uses numbers or letters it makes sense to have transparency about what those numbers or letters mean. That is the missing element in wine and whisk(e)y reviews: a clear basis. Given there is a process for whisk(e)y tasting that involves a few steps, why not align those steps to a set of items and build a rubric around it? Even to include a review of the nose, body, taste, and finish in each holistic assessment would be progress, in my opinion. This would make reviewing more meaningful to the extent you could then compare entries more appropriately. Of course, at the end of the day, the most important thing is whether you like the whisk(e)y or not, and this is probably just the logical part of my brain run amok.
Cheers,
JDW