The Vicissitudes of Cigar Ratings
With the Top Cigar List season coming to an end (it officially closes once I have published my list at some undisclosed date in the future – so check back regularly) I have been giving some thought to how cigars are rated. What really got me thinking about this was a conversation with a good friend of mine about the rating process followed by cigar reviewers. Take Cigar Aficionado’s Top 25 List, which has some surprises and some really big surprises. How exactly did they come to their conclusions? How do any of us?
In Cigar Aficionado’s case they sampled over 700 cigars over the course of the year to come up with their Top 25. What this entails is the sending of their purchasing guy all over the place (e.g. New York because when you live in New York “all over the place” is limited to the Big Apple) and he (or, I suppose, she) procures Cuban cigars somehow. How they get Cuban cigars (do they smuggle them in via cigar mule or do they take the bus up to Canada to legally smoke Cubans?) is a wrinkle I haven’t completely figured out but they have so that should suffice for now. I’m guessing the purchasing guy/gal will also search out other popular cigars but I’m not sure of that.
Anyways, after the purchasing is done the impartial purchasing man/woman will take the bands off each cigar and put new bands on for identification purposes. The tasters (side note: they have pretty cool jobs) receive the cigars and then smoke them. From what I’ve seen/read they will smoke each cigar under basically the same conditions; e.g. smoking in the office around the same time of day.
Ratings are given and then the process of whittling those cigars down begins. There are probably a few rounds of tastings that go on and this will eventually lead to a ranking of the Top 25, which has to be agreed upon by a committee of tasters. As far as I understand it that is their process.
For others, and by “others” I mean yeoman bloggers, the process is a different. Tastings aren’t blind, lists aren’t done by committee and the sample size is usually significantly smaller. Take me for example. I may smoke around 800 to 1,000 cigars in a year but I know what I’m smoking and the number of unique cigars is probably somewhere around 100 to 150 or so. Of those there are a number of “dog walk” cigars that don’t really stand a chance of winning any prizes. Taking that into consideration there are something like 60 to 90 (less are actually reviewed by me) cigars that are up for my Top 10 lists (2010 and 2009 lists for your reading pleasure).
With either system there are pluses and minuses. But that’s not what’s important here. With either system there are a number of potential pitfalls. Here’s a list:
Time Spent in Retailer’s Humidor: Even if you are the Cigar Aficionado boy/girl you have no way of knowing for sure how long cigars have been eagerly awaiting their emancipation from the B&M’s walk-in humidor. Maybe that Opus X has been resting comfortably for two years while that Fausto arrived last week. I could be wrong but I don’t think there’s a way to correct for this with much certainty.
Reviewer’s Mood: We all have good days and bad days. If I’m having a bad day that’s going to hurt the score I give to a cigar. That’s part of the reason behind my, and most other cigar bloggers, policy of smoking multiple examples of the same cigar before delivering an opinion. That doesn’t happen at Cigar Aficionado unless the cigar makes it past that first round.
Environment: Cigar Aficionado probably does a better job at controlling for this than most others but there’s no getting around the fact that the environment in which you smoke your cigars plays a significant role in what you think of that cigar. Cigars that you think are great on a warm June evening can become merely good when smoked on a brisk autumn evening. That’s especially the case if your baseball team has just missed the playoffs for the second straight year.
Storage: Even Cigar Aficionado and other such “official” cigar publications must run into this problem because there’s no other way for them to taste all those cigars other than having to store them on premises for a period of time. Once you do that there will be changes, however limited, to a cigar’s performance. If a reviewer stores their cigars in multiple humidors they will be stored in different environments. The differences may be small but there are still differences.
Cigars Previously Smoked: If you are smoking more than one cigar a day each subsequent cigar after the first cigar will be effected by the previous cigars you have smoked. You can cleanse your palate and do a rain dance but your taste buds have already been affected. At least my taste buds are affected for a couple of hours after smoking a cigar, which leads me to reviewing a cigar with as fresh of a palate as possible.
Food/Beverages Consumed: Smoking a strong cigar on a light stomach does nothing good for that cigar. Conversely, a big lunch will mute the impact of a milder cigar. Factor in the different flavors you get from food and drinks and there will be some effect on your taste buds.
Non-Blind Tastings: Cigar Aficionado and the like have this over on most cigar bloggers because they do blind tastings (that’s what they say and I believe them). Unless you are able to be completely unbiased there will be some kind of a bump given to brands we already like and demerits given to those cigars we haven’t liked before. Even though this is probably true to some extent I think we do a pretty good job of keeping our biases out of our ratings.
These were the factors that I was able to come up with but I’m sure there are others that I have overlooked. All in all, I think all of these top cigars of the year lists are true barometers of what reviewers objectively perceive as being the best cigars they smoked during the year. As cigar consumers it is our job, nay, pleasure, to find reviewers who share a liking for the same kinds of cigars that we like and use their lists and reviews throughout the year for purchasing ideas.