The 8th annual San Francisco World Spirits Competition announced their medal winners recently. Since then, I have been getting press releases almost daily from spirits companies bragging about how many medals they won.
I respect the competition’s organizers. Ditto the 25 judges. In fact, some are my friends. Some write (or have previously written) for Malt Advocate magazine.
I’m looking at the category winners. All are deserving of their awards. They are great products.
So, what’s bugging me then?
A total of 847 spirits were entered into the competition. 749 of them were awarded a medal (Double Gold, Gold, Silver, or Bronze). If my math is correct, 88.4% of all entries got a medal. That’s nearly 9 out of every 10. And 31% of all entries were awarded a Gold or Double Gold.
So, if you represent a spirit brand (Scotch whisky, bourbon, Irish whiskey, etc.), and you pay your $400 to enter it in this competition, you have a nearly 9 in 10 change of winning a medal (and a 3 in 10 chance to win Gold or Double Gold), which explains why 847 spirits were entered.
I can’t blame the brand owners for participation. $400 is a small investment to be almost assured a medal in what is being dubbed “America’s Number One Spirits Competition.” And I have no problem with the organisers making a buck or two off the concept (847 x $400 = $338,800). A spirits competition is a good idea, we need something like this, and these are the right people to do it.
My problem with the whole thing is the number of medals awarded. Most consumers think of medals as being something rare and special. They think of the Olympics, where one person gets a Gold, one person gets a Silver, and one person gets a Bronze.
Imagine what the Olympics would be like if 88.4% of all participants won a medal! Everyone would be walking around the Olympics grounds with medals on their chest. For a little while, anyway. Then they would finally figure out that if just about everyone has a medal, it’s not very special after all, and they would just stop wearing them. They would just get in the way.
The whole purpose of a spirits competition like this is to guide the consumer in making intelligent decisions when buying spirits. They naturally assume that a medal winner is something special and worthy of consideration when making a purchase.
I think every every brand owner who advertises that their spirit won a medal should also be obligated to state what the odds were in their winning the medal. (Just like the sweepstakes are obliged to do.) That way consumers could intelligently weigh the merits of the medal.
In this current scenario, I feel like everyone wins–except for the consumer.
I want to add that I forwarded a note to one of the organizers for a reply. It’s only fair to get their side of the story.
I also want to mention that this is just my personal opinion on all this, not those of Malt Advocate or its writers. — John
John,
I agree wholeheartedly with your take on this. Unfortunately, there seems to be some intentional vagueness surrounding the intent of this competition. If one looks at the list of winners on the competition’s site, none are described by category, but upon further inspection, I find 76 distinct categories for spirits listed, many of which seem to have multiple derivations within the category.
My math says that there should be about 400 medals awarded, given the categories and derivations, so where do the additional 300+ medals come from? Vague, very vague.
I spoke with Paul Pacult, one of the organizers, and here’s what he had to say:
“I somewhat understand your viewpoint as an outsider to the situation. Unfortunately, it’s wrong and not well thought out. What’s happened is that companies send us their best stuff and when something’s good, it’s good. We don’t feel that there needs to be a cut-off for the number of medals since all products are treated in the same manner. Rather than taking the negative viewpoint that you’ve decided to take, I view it more as testament to the rise in quality in distillates over the past generation in all categories. I view the results more in the reality that distillers have taken their art to the highest levels and that the SFWSC judges are recognizing those advances in ways that are fair-minded and unbiased when faced with flight after flight of quality products.”
I understand Paul’s argument that the companies are only entering the good stuff into the competition (and that the quality of the industry, as a whole, is improving). But, using my Olympics analogy again, each country only sends their best athletes to the Olympics. Still, for each contest they enter, there is only one Gold, one Silver, one Bronze. I guess that’s the point I was trying to make.
Paul offers his thought on this issue:
“I hear what you’re saying, but that’s why we have the sweepstakes round and Best of Category awards. To bestow one gold, one silver, one bronze to the single malt category, say, when there’s a plethora of crazily wonderful malts would be short-changing the category, in our view.”
Fair enough. Anyone else out there have an opinion on this topic?
While I understand the fact that these medals are just shiny participation ribbons, I don’t think that is the true issue. The “problem” is that (in my experience) there are a LOT more good whiskies out there than bad ones. This is a good thing! As you said yourself, most, if not all, of these whiskies deserve some kind of award. That is why when I read about whisk(e)y I much prefer to read the tasting characteristics than the score. Who knows what I would score it? Maybe these companies should market their products for what they are rather than the so-called awards they’ve won. I can think of plenty of whiskies I don’t like that carry awards like a badge. Same with many of the ones I do like. But if you look up at the shelf, or leaf through a magazine and the all look the same and carry the same shiny honors, what is to differentiate? And also, how bad do the ones that do not have awards look?
I don’t which ones didn’t win medals. Hopefully, they will be included with the medal winners when the competition organizers post the medal winners up on their website.
Hello,
Actually, we have the same problems with our Malt Maniacs Awards. Bottlers and distillers send their best stuff and the global ‘medals rate’ is high (even if we give gold to only 5% of them if I recall well).
I think it’s the word ‘medals’ that’s wrong (same with our awards!) because as John said, popular belief is that only the first three ‘winners’ get a medal at any competition (OK, except in judo) whilst we give medals to any spirit that got more than X points at a blind tasting.
Maybe we should replace the medals with something like seals, thumbs up, stars (like Michelin’s, more in line with ‘infinite’ categories), thistles, whatever…
Serge
Yes, I agree Serge. If competitions changed their awards structure from medals to a numerical rating or, as you suggest, another symbol, like stars, then I think it would resolve the issue. — John
So what I’m hearing here is that there are an unlimited number of “medals” handed out at these things. If that is the case, why would anyone further dilute the pool by offering a “double gold?” Just hand out as many golds as you see fit. This all sounds very gratuitous and self-serving to me, and is somewhat deceptive to the general public. They deserve to know that potentially an immense number of spirits can be awarded a gold (or “double gold”) medal.
I stand with Serge on his view that a rating system would be much more honest, accurate, and effective than this “medals” thing.
In review, I also agree with Mr. Pacult, that “…it’s wrong and not well thought out.” However, I would apply that opinion not to John’s perception of the SFWSC awards process, but to that process itself. I still feel strongly that the way it is set up and the way those “medals” are subsequently presented to the public is confusing and misleading.
The “prestigious” San Francisco World Spirits Competition can do better.
yeah, but bowmore 18 is still a kicka** scotch.
Indeed!
The common theme here is simply this: Honesty. Straightforwardness. John did a great job of it in his column above. It was how he felt. That’s what I like in a critic. If I’m observing a fine painting, it helps to hear another person tell me about the brush strokes, and the movement the painter belonged to, and the texture of the oil paints. But what often matters most is hearing how that painting makes another man _feel_. That’s something I can easily understand and compare to my own feelings. Seeing that John Hansell is confused by the whole San Francisco thing made me feel that maybe I wasn’t so crazy myself. That whole double-gold thing has been bugging the hell out of me.
Paul’s explanation makes good sense… but also sounds a bit like “spin.” I subscribe to the Spirit Journal and respect Paul’s opinion very much. What I don’t like is that I just don’t understand how the S.F. Competition came to be in the first place. What’s the purpose? The only explanation on their website is that “It is a fabulous marketing and promotion opportunity for the top medal winners.” That doesn’t sound so great to a consumer like me. Again, I understand Pauls’ defense, but full disclosure would really go a long way. His newsletter doesn’t accept advertising (kudos)… yet his competition almost seems to pander to the distillers. Please, Paul, pull the curtain open and show us that things are on the level.
John’s doing what a lot of people in the industry don’t do — he’s being honest and he’s putting himself out there. Any critic who’s willing to challenge the very industry that he’s a part of gets a gold medal in my book… or at least 90 points.
One more man’s opinion, anyhow.
I am in charge of web content for a prominent international spirits retailing website based in the UK, and part of my job is to add product information such as independent tasting notes and medals won at international festivals. I will not be bothering to add the SFWSC results to the products on our website this year because they are essentially meaningless, a fact that is becoming widely acknowledged on this side of the pond.
I agree entirely with John that the number of medals awarded by the SFWSC is far too high. What I hadn’t realised was just how high it was – 88.4% medalwinners! And 31% Gold or Double Gold!!
It is clear that a medal awarded from the SFWSC, where over 260 of 847 entrants have received a Gold Medal (or better!), means rather less than an equivalent medal from the Malt Maniacs – 10 Gold Medals in the 2007 Awards from a field of around 200 entries (based on my possibly faulty maths and Serge’s figure of 5% of entries).
I must declare an interest here, as one of my company’s products won a Gold from the Malt Maniacs. However, I can assure you that the surprise and delight we experienced at finishing in the top 10 of the 200 products submitted was considerably more than we would have managed if we had entered the SFWSC and discovered that we were in the top 260 or so of 847.
In addition, the awards from the Malt Maniacs are rather more independent: most (if not all?) of the judges have no connection with the industry other than a love for the product, and there is no entry fee for competitors other than the price of the bottle submitted for sampling. They are not reliant on the good wishes of multinational drinks companies for their livelihoods. They even limit the amount of entries per company to a maximum of three.
This means that they can please themselves when it comes down to how the gongs are handed out, without having to worry about next years entries. This is emphatically not the case at the SFWSC, where the organising committee seems not to want to upset the companies submitting the entries (and the cheques), and has opted, therefore, to give everyone a prize simply for taking part.
It seems obvious to me that the larger companies are happy to pay $400 for what is a 90% chance of a medal, and it is therefore clearly in the interests of the organisers of the competition to make the attainment of a medal as easy as possible.
Look at it from the perspective of the companies entering the competition. If you have a 5% chance of a Gold medal and it costs $400 a time, you would only enter your very best stuff. If you have a 31% chance of a Gold (or a Double Gold!), and a 90% chance of some kind of award, then you might as well enter the whole range. In that light, it is little wonder that the SFWSC attracts so many entries each year.
In my opinion Mr Pacult, whose work I have enormous respect for, is being disingenuous in his (slightly rude) response to John’s thoughts when he says that the reason for so many awards is that the quality of distillate has gone up so much in recent years. Even if this is true (and certainly in whisky terms there are a lot of obvious examples where this is not the case), surely we should be judging today’s spirits by today’s standards rather than the standards of the past?
It is also misleading to say that the companies are only submitting their best products. As Serge says, the same companies only send the best products to the Malt Maniacs Awards (they have to, as they are only allowed three entries!). Again, even if this was true (and I have made my feelings about that clear earlier in this diatribe), it is down to the organisers of the awards to decide how strict the criteria for their awards should be. Mr Pacult’s argument in relation to this would seem to be ‘It’s not our fault that everything is so good.’ But is this a good argument? And does 847 entrants really only represent the elite of spirits?
In my humble opinion (in relation to which I am well aware that nobody will be losing any sleep, which is why I’m happy to hold forth on this matter), at any competition where practically a third of the entrants receive a Gold or better, a Silver or Bronze is definitely nothing to be proud of, and as a ‘marketing and promotion opportunity’ will become less ‘fabulous’ each year. We are a cynical bunch over here in the UK (some might say less credulous) and a lot of people in the industry stopped taking the results of the SFWSC seriously some time ago.
Unfortunately, a competition that awards so many medals devalues all similar awards and not just its own. The customer is not as stupid as some people in the industry would like to think.
Apologies for the length of this post, this is something that I feel strongly about.
TF: Thank you for taking time to offer your thoughts. No need to apologize.
I am aware of the Malt Maniac’s awards. I also know about the International Wine and Spirits Competition. What are your thoughts on IWSC? And are there any other “medal” competitions on your side of the pond? — John
As a consumer, I take a very cautious view of various medal competitions for spirits and for that matter, wines. I pay no attention at all to shelf labels announcing double-triple somersault gold-titanium medals.
IMHO, the Malt Maniacs medal derby is the most trustworthy of the bunch – as a collective group of extremely well informed enthusiasts, they have no apparent monetary motive, and my understanding is that they taste their samples in blind competition. And I’ve rarely tasted a dog for the whiskies that Mr. Hansell awards.
At the end of the day, the best way to evaluate whisky is to taste it yourself before buying. I would guess that is the main practice of anyone interested enough to read this board, so the discussion of this matter really pertains to the much larger number of consumers who rely on experts with varying degrees of association to marketers to help them choose their purchases rather than “us”.
To acknowledge Mr. Pacult, it is true, there are many high quality (and increasingly expensive) whiskies now available.
Ewan
Ewan, indeed “try before you buy” is the best way to choose a whisky you will be satisfied with. Unfortunately, not everyone has this option and must rely on other sources (books, competitions, freinds, magazines, etc.) for advice.
I think it is especially important to taste Independent Bottler whiskies first before buying, becuase the quality (and flavor) of these whiskies vary greatly. –John
One other thing in the case of the San Francisco World Spirits Competition…
How do 847 spirits get appropriately and fairly judged by 25 judges in 2 days?
If all 25 judges try each spirit, that’s around 53 samples an hour (assuming an 8 hour day.)
Since the time/space continuum proves that that doesn’t work, I’d like to understand the break-down of panel size and logistics.
Kevin, this is like for some physicians: random diagnostic…
Personally, I never buy a whisky because it has won 3 gold medals at the SF WSC or any similar competition, which are for me worthless. However, this is quite important for the companies for their advertisement campaigns….
Like any investor, if you pay a few hundred dollars and this might allow to sell a few cases more, then this is a nice return on investment?
Unfortunately, the whisky enthusiast represent only marginal proportion of the whisky drinkers…