IGN: We trade editorial placement for exclusive reviews that benefit us

I'm really not the type to get into feuds, but I'm also not the type to abandon my positions or to let a public insult go unanswered.

Which is why, of course, I have to address IGN's response in GameDaily to my post about the ethical problems with "exclusive reviews." Particularly since their response was simultaneously so paltry and also so revealing about exactly why this practice is really troubling.

It also ties in nicely to some reporting I've been doing on this subject, which I'll be including in this post, along with responses to some other questions I've been getting.

But first, what IGN told GameDaily:

[VP of games content Tal] Blevins made sure to point out the benefits of such premiers to IGN's readers. "We really want to try to get our reviews up the day of release or preferably a couple of days before so people can read our reviews before hand and use that information to make a purchase decision."

This has nothing to do with exclusivity. IGN could get its review up on release day or a few days before without negotiating a deal to do so exclusively. How would its readers be hurt if other websites also put up their review at the same time? The only people benefiting from IGN doing so exclusively are IGN and its advertisers.

The following comments taken from GameDaily are by Hilary Goldstein, who's editor-in-chief of IGN's Xbox channel and wrote the "GTA IV" review in question:

My position as editor-in-chief in the Xbox channel is to actually try to get [exclusive reviews]. It's not like somebody kinda calls you up and says, "Hey, we have the exclusive on this." It's my job to actually secure that. And that's just constantly being in contact with PR...

If you looked at our site, that entire week was all GTA tops on IGN.com, which was something we'd never done before. So it was an entire week leading to the review. That's how we get exclusives of any kind. We have real estate which is the placement of a story and that's what we negotiate with. Whether it's news or features or reviews, our bargaining chip is to basically say, "I will put it here if you let me have this." So we basically gave them top on IGN.com for five days, which is a huge deal, and that, to my understanding, is what sold it...

For us on the 360 side we have more than twenty stories that go up every day and we only have ten spots that editorial can place them in each day. That's, basically, our bargaining chip. If you want something to be seen and the higher you want it to be seen, we want the exclusive out of it. That's pretty much how we got the review. It's not that the review wouldn't have been topped if we put it up on Sunday with everybody else, but it was more that we were going to give them a lot of exposure leading into the review.

OK, so Goldstein has just admitted two things. 1) Before writing the review, he spent six months negotiating with the company whose product he was reviewing to get an exclusive on it. Which makes the whole thing even more troubling. Even though he didn't promise a good review, he was the person most involved in horse trading with Rockstar in order to get the benefit they granted him of an early embargo on his review. So it's not just IGN overall in an ethically troubling position of negotiating for favors before a review, it's the actual person who reviewed the game.

2) IGN trades editorial placement in order to get exclusive reviews. Goldstein admitted it with, apparently, no shame. So when you go to IGN.com and you see content at the top of the page, don't be under the mistaken impression that it's necessarily what the editors think is most newsworthy or interesting. It could just be something they gave away so that they could get an early embargo and a traffic boost over the competition on something else down the pike.

To address an obvious criticism somebody will probably bring up: Yes, of course, all publications including Variety tend to give better placement to exclusive news stories. And we all as reporters endeavor to keep our scoops exclusive. A reporter has some interest in keeping a story exclusive as a reward for his or her hard work in finding out the news. But no legitimate publication or professional journalist I know of ever guarantees placement. Even in the occasional situation when I get "offered" an exclusive by a source, if I choose to take them up on it, the story ends up where the editors choose to put it.

Back to Goldstein:

To me, my problem with online journalism in general is that nobody does their due diligence. Nobody from Variety called us and said, "Hey, would you like to comment about this?"

Because it was an opinion piece. Not a reported piece. I knew all the relevant information and was offering my perspective. As is evidenced in this interview, there is nothing IGN could have told me that I didn't already know.

So if Variety didn't get the game early then you're looking at somebody, I don't know, who had a grudge on his shoulder because he didn't even have the game yet and we'd already put out the review. He says in blog post, "If I had the game right now I would have broken the embargo." To me that goes against your ethics.

Surely this is a joke. But for the record: As a mature adult, I am capable of making arguments that are not driven by personal grudges. It would be nice if those responding to me were mature enough to address my actual point without making a personal accusation.

And my ethic, by the way, is that I only agree to an embargo if it's the same embargo that every other publication is adhering to. Once somebody else goes, I go. It's that simple. I don't know any other professional journalists who disagree. You may piss off a source for a little while, but you get more respect from readers (which, incidentally) ultimately makes you a publication that sources are more interested in working with anyway).

Goldstein again:

And of course we gave it a 10. But so did everyone else. There's not a person out there, even in the complaints, nobody said "this game is awful and IGN's giving it a 10." Everybody said that this game is brilliant. So what? Instead of telling people, "get the game" we were telling them "really get the game." It's sort of nit picky.

OK, so, obviously this guy missed the point. To repeat myself: "I'm not saying that 'GTA IV' doesn't deserve a 10." I have a problem with the concept of an exclusive review, regardless of what the conclusion is.

And guess what? Turns out I'm not the only one. I checked in with the folks at Ziff Davis (1UP/EGM) and here's what James Mielke, editor-in-chief for videogames, told me in an email:

[W]e do not actively pursue exclusive reviews with publishers, despite the windfalls we may reap in terms of online traffic or newsstand sales. This is something we did, admittedly, at one point pursue, but have decided to withdraw from as of the last couple of years for various reasons...

As the years passed, more and more publishers started getting bolder, offering us exclusive reviews of certain games they held in high regard, but only if we could guarantee scores of a certain grade or higher. We politely declined. The downside, however, was that certain competing publications accepted these offers, which not only undermined what every other enthusiast publisher like Ziff-Davis was doing in regards to our editorial integrity, but were also very sloppy in keeping that information quiet. Once word got out that various gaming publications were essentially 'for sale,' it became impossible for us to negotiate deals of this kind any more, for fear of guilt by association. Occasionally, a publisher will let us run a review online earlier than other websites, but there are never any guarantees or promises made as a result. Plus, we never go after them.

What about IGN's other big competitor, GameSpot? Here's what Justin Calvert recently wrote on their reviews blog:

GameSpot doesn't do exclusive reviews.

This has been the case here for as long as I can remember, and should come as no surprise to anyone who's ever taken the time to read our Review Guidelines. Not only could agreeing to an exclusive review invite a perceived conflict of interest [emphasis mine] where scores are concerned, but it would lock us into posting our review on a certain day, probably at a certain time, and almost certainly before we've had an opportunity to spend as much time checking out any online features as much as we'd like. Even ignoring the score stuff that's a problem, because we post our reviews when they're ready, and not before.

So, EGM/1Up and GameSpot and (to the extent anyone cares) Variety agree. I think the question now is: How do gamers feel? Are they annoyed that the publications they trust for news and information trade away their editorial space in exchange for exclusive reviews that benefit them and not their readers? Is it at all disturbing that the same individual who spends months engaging in that exact horse trading is then the one who sits down to write a review that's supposed to be objective and unbiased? Do they think they would benefit if any and every publication had the same embargo and just competed on how good their reviews are?

If the answer is "yes," I think gamers should let their opinions be heard by IGN and GameInformer and whoever else engages in these practices. That's my opinion as a gamer, anyway. I've obviously done so (perhaps at greater length than anyone cares).

Finally, to address some questions/criticisms that have been sent my way:

-What's your problem with Rockstar? Are you just mad that you didn't get a copy earlier? And after they gave you all that time to talk to Dan Houser... How ungrateful!

This has nothing to do with Rockstar. Its publicists have every right to give copies of their game and set embargoes in the way that best serves their goals. It's not Rockstar's, or any publisher's, job to defend journalistic ethics. That's a job for journalists and readers.

And for the record, Rockstar was nice enough to get me a copy last Friday, which was perfectly adequate to get my review done by Tuesday (even if I didn't sleep a lot in the 3.5 intervening days).

-Variety and its partner Reelz Channel ran a video review of "Iron Man" that they advertised as "first." Hypocrite! Hypocrite!!!

There is a difference between "exclusive" and "first." Variety has a long tradition of running reviews well before films open because our readers are traditionally industry professionals, not regular people wondering whether it's worth seeing (though that's changing somewhat on the Web).

But we don't do so by negotiating with a studio for an early embargo. In fact, the Hollywood Reporter typically runs its movie reviews on the same day as Variety. Sometimes they even beat us. Consumer media, on the other hand, typically don't want to run a movie review before opening day, because that's when their readers are most interested (obviously some weekly TV shows, like "Ebert and Roeper," go a little earlier). We compete with them by trying to attend every festival and screening that we can, not by trading favors with one of the companies that we cover. Sometimes, one of our film critics just told me, we break embargoes when we feel like we have a good reason. And yet, somehow, Variety survives.

Comments

[John]

"There's not a person out there, even in the complaints, nobody said "this game is awful and IGN's giving it a 10.""

........this game (Grand Theft Auto Four) is awful, and IGN gave it a ten. There goes any respect I had for them.

Seriously.

(yes, I've played it, yes, I think it's really not a ten.)

[michael_urayama94]

Mitch, Mr. Fritz is criticizing IGN for trading EDITORIAL placement for early access to games, developers, what have ya. Ads are a completely separate story. Just like a magazine like EGM gives Capcom a cover in exchange for revealing Bionic Commando exclusively first, IGN trades cover placements for exclusive access. I can cite numerous cases where Variety has done this as well. Mr. Fritz also makes it sound like exclusivity automatically dictates placement, not user interest or quality. It's a bit of a fallacy as publications don't go after exclusives that are not popular or not for high quality products. I've seen IGN champion games like Zack & Wiki or Shadow of the Colossus, neither of which are bestsellers or traffic-drivers.

Mitch: I don't think you got my point. Mr. Fritz is accusing IGN of lack of editorial integrity, questioning its ability to provide fair, trustworthy and accurate reviews for titles that they have given exclusives to (Fritz only mentions reviews, but I don't see why we should ignore exclusive previews or trailers). Looking at the site's history, I don't see evidence that IGN is unable to give its honest opinions on games DESPITE running exclusive, or first, reviews of titles. This fact is glossed over.

What is also glossed over is how IGN spent a long time playing the game before offering its evaluation and had multiple editors play the title and discuss the review and score before publishing it. Why isn't IGN lauded for going this extra mile to make sure its reviews are accurate?

Instead, people are reposting and backing up the opinon of a writer who has clearly NOT finished the game (that GamerTag data is up-to-date) yet makes judgments about the overall story and quality of the title.

If integrity and editorial processes that ensure trustworthy reviews are at the core of this argument, I don't think we should ignore that the accuser has not demonstrated them.

Why, for example, would IGN's ability to play the game ahead of the Variety editor cloud their writers' judgment, yet the Variety editor's acceptance of a free copy of the game or (rare) access to interviewing the head of Rockstar be factors that can't have the same effect?

[mister slim]

Why the fuck is editorial selling ad placement? If R* wants ads they can buy them. Why is Hilary so proud of brown-nosing R* until they accepted free ad copy from IGN?

[Mitch]

"Pointing fingers at the big boys like IGN has become a popular sport on blogs, followed by the usual impressionable responses of people who take every opinion as face value. "I knew IGN was paid off to like the game."

"It's the lowest form of journalism to use someone else's editorial and repost the opinion to cast doubt on an outlet's editorial integrity - especially when dealing with a competitor."

Okay, then tell that to Kotaku and the other blogs who are re-posting the editorial, not here.
"What I didn't see was anyone actually asking whether anyone else's review process actually compares to IGN's -- or even ask once: if IGN's reviews are tainted, why do so many people trust them explicitly?"

Because they're 'exclusive'. That brand gives the preview some false sense of authenticity and truth that simply isn't there. And since when did public opinion dictate quality?

But when it comes down to it, I think the biggest reason is that a lot of people are willing to, oh, how did you put it again? "take every opinion as face value"?

"Why don't the reviews differ radically from other publications (except for in negative cases, like Assassin's Creed)."

So wait, they have journalistic integrity because their review scores are really similar to everyone else's?

"As a member of the industry, I've seen these guys go to extreme lengths to make sure their reviews are accurate and they got the maximum playtime before writing the review."

No, you've seen them go to extreme lengths to make sure their reviews get the most clicks. What 'extreme lengths' are you talking about? Whoring out what little journalistic integrity they have for a few thousand more page views?

"Mr. Fritz is quick to point out that he felt his time with GTA IV (which was provided to him for free by the publisher) was enough for him to formulate an opinion and publish."

What does him getting the game for free have to do with anything? Is he supposed to be grateful and inflate the score? By that logic, early reviews of Superman 64 shouldn't have gone below a 5/10.

"I decided to look up his GamerTag: http://live.xbox.com/en-US/profile/profile.aspx?pp=0&GamerTag=kalter

Sorry, but not getting to the second island doesn't qualify as "enough time" for me. I don't know the IGN reviewer's gamertag, but judging from his description of the review process, it sounds like he finished the game and tested the multiplayer mode."

Alright, this is just a straw man argument. Just because he didn't finish the game doesn't make IGN have any more journalistic integrity. Maybe he didn't finish the game, but at least he didn't whore himself out and lose any credibility he had just so he COULD finish the game.

Oh, and take another look at his gamertag. The last time he was online was Sunday. He wrote his review on Tuesday. Methinks he played the game in between, don't you? ;)

[michael_urayama94]

Pointing fingers at the big boys like IGN has become a popular sport on blogs, followed by the usual impressionable responses of people who take every opinion as face value. "I knew IGN was paid off to like the game." I was even more surprised to see this opinion piece highlighted on every blog and gaming news site from Kotaku to Next-Gen. It's the lowest form of journalism to use someone else's editorial and repost the opinion to cast doubt on an outlet's editorial integrity - especially when dealing with a competitor.

What I didn't see was anyone actually asking whether anyone else's review process actually compares to IGN's -- or even ask once: if IGN's reviews are tainted, why do so many people trust them explicitly? Why don't the reviews differ radically from other publications (except for in negative cases, like Assassin's Creed). As a member of the industry, I've seen these guys go to extreme lengths to make sure their reviews are accurate and they got the maximum playtime before writing the review. I've never received a score in advance of a posted review.

Mr. Fritz is quick to point out that he felt his time with GTA IV (which was provided to him for free by the publisher) was enough for him to formulate an opinion and publish.

I decided to look up his GamerTag: http://live.xbox.com/en-US/profile/profile.aspx?pp=0&GamerTag=kalter

Sorry, but not getting to the second island doesn't qualify as "enough time" for me. I don't know the IGN reviewer's gamertag, but judging from his description of the review process, it sounds like he finished the game and tested the multiplayer mode.

[Shane]

So I guess that I'm supposed to take movie journalists more seriously than video game journalists. I'm mostly just sensing a lot of whining on both sides. Can both editors just be slapped for not being able to get over themselves?

[Chris]

Hilary Goldstein is hilarious. I love IGN.

[cornholio]

the reason why people agree to exclusive reviews is so that the reviewer gives a good score in exchange for the exclusive

a review is an opinionated position of a game
you can't make a review exclusive because the exclusive reviewer may have strong opinions compared to others and many people buy games based on reviews

[Merus]

I don't know how far back your history with the game press goes, Ben, but this quote here is telling for those of us with long memories:

"so people can read our reviews before hand and use that information to make a purchase decision"

Exclusive reviews have had something of a reputation of being inflated compared to the non-exclusive reviews later. I think it was Driv3r, quite a poor game, whose exclusive review gave it a 9/10 based on, we were told, bugs that the developers assured the reviewers were fixed in the final version. The review score was prominently displayed on the box packaging.

So when they say 'read our reviews and make a purchase decision' they might well be saying 'read our reviews and go purchase the game'.

[Bill]

The problem is that GTA IV *isn't* a 10/10 game. I personally gave it a 5/10 on my blog, but after playing it some more, I feel that's a bit harsh - 7/10 feels better now.

The handling of the cars is worse than in any GTA game to date, the storyline is merely so-so, the street layouts are very badly done, the camera has issues (it tends to hide behind the car making it hard to see where you're driving), and the world just isn't very interesting to explore.

[ldonyo]

Danny seems not to have read the whole post, if he rad any of it at all. He certainly didn't read the interviews you pulled the quotes from, Ben.

Game reviews are typically honest and unbiased on fan sites that do not derive revenue from advertising beyond Google Ads. The writing isn't very polished, but the sentiments are plain and given in no uncertain terms. Once a site starts depending on the publishers whose games are reviewed for their financial existence, ethics slowly, but surely, find themselves being compromised and abandoned.

If there were a publisher willing to try the Consumer Reports path (no advertising or freebies and all operating funds are given as donations from individuals) to reviewing games, then and only then will we have a shot at getting the unvarnished truth in the form of polished reviews.

[Danny]

All the people who have been complaining about IGN getting the exclusive review are coming across as bitter because IGN got an exclusive and they didn't. Whether that's how they truly feel is a different matter, but it is certainly coming off that way.

First, if you are doubting IGN's integrity in their actual review, and saying they gave it a higher score than they felt it deserved, then say that. Quit implying it in a roundabout way. Besides, IGN wasn't the only company to give it a perfect, not by a looong shot. So this point would just sound crazy.

Second, if you think IGN getting the exclusive review was some kind of marketing ploy on the part of Take 2 and IGN, well congratulations sir, you've just stated the obvious. Achievement Unlocked! Of course there's aspects of IGN's marketing that we aren't aware of, but as long as it's not content of the website itself (see item 1), there's no reason to really complain.

[GW]

You're absolutely right to take this position and frankly, it's laughable to see IGN trying to defend it.

I worked as a games journalist for six years, and to hear anyone trying to defend it as either 'real journalism' or 'ethics-driven journalism' is laughable.

While there is some excellent writing in the world of games journalism, the kind of ethics routinely displayed by publications like Variety, Time, Newsweek, Wired etc never, ever enters into the vast majority of print and web-based games journalism.

Publications regularly jockey with their rivals for exclusive reviews (and more often, previews, which are written as thinly-veiled puff-pieces). PR folks regularly berate journos for 'bad' coverage, and will happily stop them from getting review copies of games, access to developers, etc.

Quite simply in games journalism, for the most part, the games publishers have journalists by the balls and that situation is never going to change whilst games publications are still desperately fighting for a fickle 10-15 year old readership.

If you want integrity you generally need to look at a few places where for whatever reason the source has not been 'bought'. That isn't Gamespot and 1Up - while I'm not saying they're bad, they haven't always been good. Generally you have to look for writers and reviewers and sites that are so under the radar they haven't registered yet; like Penny Arcade was five years ago. With nothing to lose, sites like that can say what they want. (Ironically, they then tend to gain huge audiences for being honest, and that's when things start to blur, adverts start to appear, freebies start to get thrown around and so on.)

I don't think this situation is going to change any time soon, either. Games journalists invariably get paid crap because the people paying them know there's an army of kids out there who'd love to do their 'dream job'. Being paid crap, those journalists happily accept every perk that comes with the job, and will find it very difficult to say 'no' to the PR who flew them to some far-flung paradise for a press junket last month. And all the while, the desperate need for hits, or readers, will push the magazines and the sites to go after exclusives which are given out deliberately to the sites or magazines which the publisher feels are most beneficial to their product at whatever stage it's currently at.

(Want to look like your game is cutting edge and cool? Secure the cover of EDGE or Game Informer. Want to get the most readers to hear about your heavily hyped game? Go to IGN. Desperately need coverage for your middling-to-bad game and none of the 'big boys' are interested? There are a million aspiring games sites and journos out there who'll write nice things because frankly, they can't believe they're getting a free copy of a game.)

It's a corrupt system and it'll take some far-sighted journalists (some of whom exist) and some honest publishers (not as many exist, and who can blame them with budgets these days) to make it change in any way.