hugo awards

Right for the Wrong Reasons: First Thoughts on the Hugos and a Better Voting System

Last year, I wrote post titled History Will Forget The Sad Puppies. My reasoning was that the Hugo nomination chicanery was similar to the ballot stuffing at baseball's 1957 All Star Game. The TL/DR version is that the fans of the Cincinnati Reds voted early and often and got almost their entire team voted on to the All Star team. Even though no rules had been broken, Major League Baseball called shenanigans, replaced several of the players, and changed the rules to discourage this sort of behavior and the event was more or less forgotten. I thought that we'd see essentially the same dynamic at play with the Hugos. 

The people who took advantage of the broken system are also cut from a different cloth than those Reds fans in 1957 and have demonstrated that they won't go away even in the face of defeat. The Puppies were shut out of the awards last year, losing to "No Award" across the board. That brings us to another thing I was wrong about: I thought that the Sad Puppies were leading the slate-voting charge, but last year marked a transfer of power to the Rabid Puppies. This year, the Rabid Puppies again were at the forefront of gaming the nomination process and the Sad Puppies have faded from relevance.

In that one sense, I was right. History will, I think, forget the Sad Puppies, but only because they've been superseded by a more militant group.  I'm not focusing on the Puppies, though. I want to go over what's wrong with the voting process and make some suggestions as to how it might be improved.

The first question you have to ask is "What are the Hugo awards meant to reward?" Are they for the "best," most literary, or most artistic works, or for the most popular, or for the most commercially successful? Depending on the answer, you may wind up with different ideal voting systems. The question isn't answered definitively on the awards' site:

What are the Hugo Awards?

The Hugo Awards, to give them their full title, are awards for excellence in the field of science fiction and fantasy. They were first awarded in 1953, and have been awarded every year since 1955. The awards are run by and voted on by fans.

"Excellence in the field of science fiction and fantasy" is certainly something worth rewarding, but it doesn't give the voter a lot of instruction. I'm going to let the last sentence be my guide today. The emphasis on "fan" suggests that the fans are free to make up their own criteria for what constitutes "excellence." That leaves us with what amounts to a popularity contest, which is fine, but I wanted to make that clear before asking the next question. 

"Is the process for nominating and voting broken?" To say it's broken, you'd have to say that the eventual winner of the award is not the work adjudged to be "excellent" by the most fans, or, more simply, the most popular work. Last year, no award was given in five categories. I think you can reasonably conclude that there's a disconnect between the nomination process and science fiction and fantasy fans.* This year, the success of the Puppies slate tracked inversely with voter participation in the category. That is to say, the more people who voted, the worse the slate voters did. This suggests strongly that a small group of voters, working together, were able to influence the nominating process to a degree which creates an undesired outcome, regardless of the politics of the small group.**

This next one may be a little pro forma, but it's an important to go over the reasons: "Where is the process broken?" The problem lies in the nomination process. The system is set up to encourage this kind of abuse. The nominations are open, meaning that any work which meets the qualifications may be nominated. That sounds marvelous, but there are many, many more works published each year than any voter can reasonable keep track of. You can nominate up to five works, which is exactly how many will appear on the final ballot which obviously can result in a slate taking all the nominations. There's no minimum number of nominees a voter must submit, nor a requirement that they submit works in all categories, making it easier for slate votes to dominate more obscure categories. There's relatively little participation in the nomination process, which inflates the influence of a determined minority. About the only good thing that can be said of the nomination process is that it requires a WorldCon membership, so ballot-stuffing is difficult and expensive.

The result is that it's possible for a relatively small subset of voters to dominate the nominations in a way that cannot be overcome in the final voting.*** I think most people would agree that this is not a desired situation. This is true whether or not there are Puppies or communists or alien pod people trying to monopolize the nominations. It's a bad system, and so long as it stays like this, it will be open to this kind of gaming. Speaking of gaming, if this sort of nonsense were taking place in, say, an MMO, the moderators would smack the offenders with the ban-hammer. Major League Baseball simply decided to overrule the voters. The fact that the people who run the Hugo awards have not acted so highhandedly is admirable, but their good will hasn't fixed the problem.

There is a plan to try to minimize the influence of slate voting in the works. It's been dubbed E Pluribus Hugo. It not a change to how the nominations are submitted but instead to how they're tallied and how the finalists are selected. It's math-y but not difficult to understand. Every voter gets one "vote" per category, and that vote is split among all the works you nominate in a category. If you nominate 5 works, then all of your nominees get 20% of a vote. Then, the votes are added and the work with the lowest vote total is eliminated. At this point, it's all tallied again, and if one of your works is eliminated, then you have 4 remaining works, so yours get 25% of a vote each. This process goes until only five works remain.

It's an interesting and appealingly geeky way of dealing with the problem, but it doesn't actually address the structural problem with the nomination process. There are still so many possible works which could be nominated that the critical mass of popularity is still going to be low enough to be subject to gaming, especially in the categories with fewer votes. Worse, it's going to encourage voters to think about how many candidates to nominate based on how they think the system works. Voters having to consider the system, not the works they like, is the exact opposite of a desirable outcome. It encourages more gaming, not less. 

So, I the last question is: "How can it be fixed?"**** I have some suggestions based on what I think the issues are and what would reduce the impact of the flaws.

  • Reduce the number of nomination slots to three, or better, one. You're only going to vote for one work in the finals, so why are you nominating the one work you think deserves to win and four that you don't?  This might make it easier for another Black Genesis to get on the ballot, but it also makes running the whole ballot far more difficult. The goal isn't to shut out people who like different works than you do, it's to prevent anyone one set of tastes from getting all the nominations.
  • Set a sales minimum for nomination, and then ask voters to select from the list of qualifying works. I know sales numbers are a very un-sexy way to determine qualification, but the idea is to prevent a list (20-30?) of eligible works and have the voters select from the list, rather than leaving it open to literally every possible work. This makes it more difficult for slates to monopolize a category because voters are picking 5 (or 3, or 1) from a list, rather than selecting from hundreds or thousands of possible eligible works.
  • If there's no sales minimum, consider having a panel of people who work in the industry and in the specific category create a broad list of potential nominees (again, 20-30) and then do the same thing as above. It's not democratic, but it might just increase the "excellence" of the awards finalists and winners.
  • Require voters to nominate in all categories, or else only allow voters to vote in the finals in categories for which they nominated one or more works. Here's where I'm sympathetic to the slate-voters. They go to all the trouble to nominate five works in a category and then voters who weren't bothered to nominate alternatives turn around an vote "No Award." The goal here is to encourage more participation in the nominating process. 

I get that the people who are running the awards don't want to make massive changes to the process and they're bending over backwards to try to be reasonable, but "reasonable" isn't the only target here. The goal is to nominate works which will appeal to fandom in general and give them the opportunity to vote for the works they believe are worthy of a Hugo award.  Based on what I've seen, I'm skeptical that E Pluribus Hugo will get the job done. 

Et obliti sunt mei...

Et obliti sunt mei...

* Now that I know a little bit about the complaints of the Sad Puppies, the fact that a small cabal managed to get their own works nominated but that those works weren't reflective of the tastes of most fans is kind of funny. Or it would be, if it weren't so disappointing that so many worthy titles failed to make the ballot.

** Incidentally, this also suggests that the awards have not been influenced by the activities a small, motivated, but relatively unpopular group up to this point.  If it had been, we'd have seen the disconnect between nominations and final voting show up like it did last year. 

*** Yes, yes. Very much like the U.S. Presidential selection process. Someone had to say it.

**** Well, another good question would be "Is it worth fixing?" but I think the answer to that one is clearly "yes." The Puppies crisis has revitalized the Hugo awards. More people are aware of them, more people are participating in them, and they're getting far more press outside of the genre than they were before. I just wish the press was about "how awesome the awards are" instead of "gah, what's going on here?!?"

Leopold Scotch and the Hugo Awards

We watched the most literary of South Park episodes the other night, "The Tale of Scrotie McBoogerballs." There's a riff near the end where critics are fighting about whether or not Butters' second novel, "The Poop That Took A Pee," expressed a radically liberal or wildly conservative vision, the joke being that it was neither and the critics were imposing their own beliefs on Butters' story. 

South Park gets their cultural satire right more often than not, but this gag didn't work for me because it didn't ring true. I can't, off the top of my head, think of many cases where a work of art was attributed to both extremes of the political spectrum. What I do see is people* regarding works which agree with their own views as "apolitical" and works with a different worldview as "overtly political."

That's my working theory: A liberal can read a book with liberal overtones and not feel that the book is political at all, whereas if they read a book with a conservative point of view, they'll see politics throughout the work and it works the other way around as well.

Viewed in this light, the controversy over the Hugo awards and the Sad and Rabid Puppies movements makes a little more sense to me. It explains why, when protesting against awards going to works for their politics rather than for being just good, fun science fiction, the Puppy slates struck me as overtly political.  The Puppies saw their slates at free of political sermonizing since they tended to share the stories' worldviews. My own political views are on the left side of the spectrum, so the point of view of the Puppies' stories stuck out to me. 

That probably seems obvious, but it took me a while to get to it. My initial reaction, when reading my Hugo voter's packet last year, was to think that the Puppies were being disingenuous when their slates were filled with works which made such strong political statements. Now, I'm more inclined to think it was a lack of awareness of one's blind spots. It wouldn't make any difference in my voting last year**, but I will keep this theory in mind when filling out my ballot this year.

For what it's worth, I can't imagine wanting stories without a point of view. Without some subjectivity in the writing, you wind up with either dry reportage of events or random gobbledygook.  I want the artist to have a point of view because otherwise, why bother? Just make sure to make it interesting art while you're at it.

* Including yours truly. I am by almost any measure a "people."

** Slates. Gaming the system. Don't do it. Please, just don't.

Because What We Really Need Is More Hugo Awards Drama

The Sad Puppies are going to make another go of it next year. My understanding is that they are not planning on slate voting this time around, and if that's the case, I wish them well. A group working to bring more awareness to science fiction and to highlight works that might not otherwise get much attention strikes me as admirable.

I feel for the Sad Puppies, because they're in difficult-bordering-on-hopeless situation. Let's say that you're taking a poll of the "six best songs of the summer!" If fifty one percent of your voting population like opera and forty nine percent prefer rap*, then there is a very good chance that every song on your list is going to be an opera song. This is a feature of how some voting systems are set up. Please don't make me link the "Spider Jerusalem on voting" rant.

Another thing which may be working against them is the inertia of familiarity. If a reader likes a particular author's voice, they will probably prefer that author's work and cast their vote for that author even if the objective "quality" of that work, however you can objectively definite it, is not equal to that of other works. I am reasonably sure that, if a Terry Pratchett novel were nominated for literally anything, it would take a massive gulf in quality to get me to vote for other works. That's my own subjective bias. I won't argue that it's a good or a bad thing, but it is a real thing and I am certain I'm not alone in this.

To me, these two factors are a reasonable explanation of why certain authors and types of books continue to be nominated and win Hugo awards. It strikes me as far more plausible than a cabal of liberal insiders gaming the system. I'm not saying it can't exist, but rather that I haven't seen any evidence to that effect.

Those were the two problems the puppies faced at the outset. There are now two more, and they're going to turn an uphill struggle into one that I can't see them winning.

Sorry. It's impossible for me to think of "rabid puppies" and not think of Old Yeller.

Sorry. It's impossible for me to think of "rabid puppies" and not think of Old Yeller.

The first problem, obviously, is that their name has been soiled. The puppies brand is associated with slate voting, an association that will not serve them well (see below). Even worse, they're associated with the rabid puppies, and that's poison. The leader of the rabid puppies has an enormous amount of personal baggage and he has a history of taking groups with a legitimate beef and turning them into frothing partisans. Even if sad puppies 4 try to distance themselves from their earlier tactics and allies, I don't think people will swift to forget.

The bigger problem, and the one which I believe probably dooms the puppies, is that the massive uptick in voter participation at Sasquan was ruinous for their slate. While many of those voters certainly had reasons other than "not liking the books" for voting against the puppies (see above), this suggests very strongly that the puppies do not represent a silent majority. Based on the numbers I've seen, I'd expect the puppies percentage of support to scale inversely with the number of voters.

Some people have suggested that the puppy slates losing to "No Award" is incontrovertible proof that the puppies claims are objectively correct and the Hugo awards are run by a clique hell-bent on ensuring the political correctness of award winners.  That's not merely hyperbolic; it's simply not true that the voting results prove anything of the sort. It's not helpful to anyone to distort the truth in that fashion, and if this kind of rhetoric is indicative of what we have to look forward to for the next twelve months, it's possible that "No Award" will be the big winner again next year, and no one who cares even a little bit about science fiction wants to see that.

Please do not interpret this as an anti-puppy statement. I just happen to be a cat person.

Please do not interpret this as an anti-puppy statement. I just happen to be a cat person.

Brief Hugo voting update

I finally finished all of the required reading and cast my ballot. Although I said I would post my choices here, I've decided not to do that on account of the ballot being of the ranked-preference sort as opposed to a single choice in each category. This was my first vote, so I was a little unclear on how it all worked, but a long list of preferences makes for  a lousy post. Instead, I'll just note a few general impressions:

1) The reading was eye-opening and I wound up voting for some things that weren't previously on my radar. No matter how objective one tries to be, preconceptions are a real thing, but there were some outstanding entries in unexpected places this year (graphic novels, I'm looking at you. Ms. Marvel was a revelation to me.) 

2) The Sasquan folks were quite friendly and helpful in getting me set up. I'm a newbie at this and getting from point A (interest in voting) to point Z (having the ballot up on my screen) was not without a little difficulty. The folks at Sasquan provided swift and accurate help when I needed it. 

3) If you're going to submit a slate and the entire slate wins nominations, please, for the love of all that is good and right, try to make sure there's at least one worthy nominee in that slate. I dislike the tactic of slate nominations, but if there are good candidates on the slate, I won't vote against them out of spite. Slate-nominating mediocre material, on the other hand, lies somewhere between trolling and vandalism. I had no problem listing "No Award" first in instances where there were no worthy candidates.

If you've never participated before, I cannot recommend it highly enough. The price of entry is a little steep, but the reading packet is more than worth the price. 

-RK

History Will Forget The Sad Puppies

Several of you have asked me what I thought about the Hugo awards mess this year. The best response I have is "Please go read what John Scalzi has to say because he's more deeply connected to this, knows more about it, and is a fantastic, thoughtful writer as well." Seriously, go check out all of his Hugo awards neepery for a much more knowledgable take on the subject.

If you haven't been following the subject and don't feel like reading Scalzi, here's my "as I understand it" summary: A small publishing house specializing in SF books with a conservative slant has gamed the nomination process for the Hugo awards. They managed to sweep the nominations in more than a few categories. I get the sense that this is primarily a commerical play by the publishing house, but it's been cast as a political move based on the (extremely dubious) claim that the Hugo awards have been co-opted by leftists. They've called their group the "sad puppies" because...I don't know, they thought it sounded better than "racist, homophobic jerks?" Anyway, the awards are almost guaranteed to be a mess this year. Some authors on the conservative slate have removed themselves from consideration. Many voters have decided to vote "no award" for some or all categories. It's ugly.

If you want my take on the Hugos, I'll give you this:

In ye olden dayes, the players selected for baseball's all-star game were elected by public ballots. In 1957, the ballots were being printed in newspapers instead of passed out to the fans at games (as I remember from the 1970's) or online (as it's done now). The Cincinnati Enquirer decided to help the fans out a little by printing pre-filled ballots with nothing be Cincinnati ballplayers selected. As a result, the starting lineup for the 1957 National League team consisted of Stan Musial, a St. Louis Cardinal, and 7 cincinnati Reds. 

People rightly saw this as a subversion of the process. Ford Frick, the commissioner of baseball, immediately replaced two Reds outfielders, Wally Post and Gus Bell, with Hank Aaron and Willie Mays because, c'mon, Hank Aaron and Willie Mays. The remaining Reds were allowed to start the game and then almost immediatley replaced once the game started, and the game looked like an All-Star game once more.

Frick took the vote away from the fans and let managers, coaches, and players select the All Stars, a system which survived until 1970. It wasn't really the fans' fault, though. The All-Star voting system was poorly designed and had always been open to this kind of abuse. It just took an incident like this to bring these issue to the forefront and get baseball to change the way the voting worked.

The lesson here, at least as it relates to the Hugo awards, is that almost sixty years later, the players elected by this broken system, Johnny Temple, Don Hoak, Roy McMillan, Gus Bell, and Wally Post, do not seem to have derived any particular benefit from their dubious election. Likewise, Hank Aaron, Willie Mays, Ernie Banks, and Eddie Matthews are not diminished by their failure to win the vote in this flawed ballot. In the long view, merit won out over attempts to leverage a poor system into a few vanity awards, just as I imagine will be the case for the Sad Puppies and their ilk.