Coffee Talk #4: Game Review Scores and You

Posted on October 15th, by raymond padilla in Coffee Talk, PC Gaming, Today's Specials, Videogames. 257 comments

Welcome to Coffee Talk! Let’s start off the day by discussing whatever is on your (nerd chic) mind. Every morning I’ll kick off a discussion and I’m counting on you to participate in it. If you’re not feelin’ my topic, feel free to start a chat with your fellow readers and see where it takes you. Whether you’re talking about videogames, whether the Motorola Cliq will turn the company around, why the hell it’s raining in Los Angeles, or bellybutton lint, Coffee Talk is the place to do it.

In Coffee Talk #2, reader rbee90 brought up the topic of game reviews, which led to a conversation about review scores. The discussion started to get interesting and reader RRODisHere suggested that I write about the topic in Coffee Talk. Well here it is!

I have a ton of problems with the way most — not all — game reviews work. Scoring is a huge pet peeve of mine. 100-point scales are just stupid. I’d love for someone to (intelligently) explain the one-point difference between a game that gets an 87 and one that gets an 88. Five-point scales — which I like a whole lot better — are a problem because of the way the business uses scores and how some consumers interpret them. Here’s a pro tip for you — three stars out of five is not the same as 60 percent. Yet that’s the way a three-out-of-five is treated by review aggregators (most of the time). What’s worse is that some publishers base royalties on aggregate review scores, which is completely unfair to developers.

Borat Thumbs Up Coffee Talk #4: Game Review Scores and You

Personally, I think there should only be three review scores — buy it, rent it, eff it, symbolized by thumbs up, thumbs in the middle, and a thumbs down (or Megan Fox’s thumbs). Isn’t purchasing, renting, or passing what it all comes down to anyway? I pushed for this system when I worked at GameSpy, but nobody was buying it. Oddly enough, my boss at GameSpy eventually went to Crispy Gamer, which uses a scale like the one I suggested. Anyway, the bottom line is that scores have become so important that the words behind them are often overlooked and sometimes ignored.

Then there’s the way some games are reviewed. Some publishers send code to reviewers days before they’re allowed to publish their reviews. For competitive reasons, everyone wants to get the review up the second the embargo lifts. This has the reviewer cramming a pint glass of gameplay into a shot glass of time. Another practice that bugs the hell out of me is when publishers have reviewers play the game off site. In these cases, a reviewer has to commute to a hotel suite or a conference room to play the game for a few days before writing the review. Again, the short amount of time introduces a problem, but it’s compounded by having to play the game in a completely unnatural setting. My issue here is that reviewers have to play games in a way that few consumers would. Do most people play 50-hour games in three days? Of course not. Do most people make daily commutes to play games in a conference room? No.

Okay, I’m getting angry about the whole deal. What I’d like to know from you is what you expect from game reviews. Do you like like 100-point scales or do you prefer five-star systems? What information is most important to you in a game review? Do you think that the unnatural way reviewers have to play games leads to an unnatural view of the game? Leave a comment and let me know (please)!

Coffee Talk PC Gaming Videogames

Hrolf

@ Rbee90

RE5 does suck in my opinion, BUT Sessler and his crew gave Fable II GOTY award. Can you REALLY listen to someone that gives such a flawed and rushed glitch-fest of a game such high praise.

If anything, read at least 5 reviews and go on the reviewers pros and cons of the game. you get a better idea of WHY the game got the score and you can make a better choice at that point.
RRODisHere

@Smartguy

I bought Uncharted 2 and yes, I settle for Brutal Legend for free. I say I settled for it because I would never have spent $60 for it. Toys R Us didn't have Demon's Souls and I had a free game to pick so I had to pick something. I wasn't going to give up a free game. I haven't even taken it out of the plastic yet. I'll get to it eventually.
Hrolf

@ R-Pad

Your system is meant to be read, and not as a simple buy it,rent it, or eff it. You are talking about actually talking about WHY the game should be rented, bought, or eff'ed right?
- http://rpad.tv raymond padilla
  
  @Hrolf Yup yup. That's the idea.
  @Everyone I'm starting to feel weird about posting comments in bed. Would this be like talking to you in bed? Like if we were all in a giant bed together. Ewwwwwww!!!
Smartguy

@RROD

same with UC2 for me.
RRODisHere

@Smartguy

As far as Saints>Falcons. Just like a game is what the reviews say they, a football team is what their record says they are. So I'll admit that the Saints are better than the Falcons……..for now.
Thundercracker

just found this on vgchartz

Naughty Dog Dev, "ReklissAbandon", slipped up with the following comment when asked about spectating the enemy team in Uncharted 2:

"I think they wanted people to have fun and ejoy the show since all dead players can talk to each other no matter what team you are on. But I will suggest it anyway. Not to mention that the next PS3 firmware update is going to allow cross game chat. So if players use that instead of in game communication then they would also be able to communicate enemies positions."

The comment itself has since been removed.

yeah cross game voice chat, xbl still worth 50 bucks?

i love flame wars, ill never stop
Empyu

I have two problems with the way games are reviewed.

People take reviews to be facts. There are no two people who've played a game that had the exact same experiences with it. You can't just look at a metacritic score and have a number tell you whether you're going to like a game or not. I bet there's someone out there right now who thinks Darkest of Days is the best game he's played in a while – and someone else who played Street Fighter 4 for 10 minutes and never picked it up again. What good is metacritic to these people?

If reviews are accurate portrayals of how good a game is, why are there so many people saying games like CoD and Halo are overrated?

The second problem I have is that all reviews in every industry are done by one person. I feel reviews would benefit if they were done Ebert and Roeper style where people either agree or disagree on the subject.

It frustrates me when I watch G4 and they say something like, "X-play gave Brutal Legend a 4 out of 5." That's rediculous because if they had gone with a different reviewer, it could've gotten a 3 or maybe a 5. They had one person represent X-play on a subject. Remember the controversy that was sparked when Mr. Sark reviewed ODST?

Game Informer pretends to have two people review games, but it's pointless because the two reviews almost always agree with each other. For any game, there's going to be people who like it and people who dislike it. Any review that doesn't represent both sides is only giving half the story and doing a disservice to the people who depend on this information for their purchasing decisions.
RRODisHere

@Hrolf

Exactly. X-Play lost all credibility for me when they gave Fable 2 G.O.T.Y.
Thundercracker

also

rip

captain lou albano

he died saturday
Shockwave562

I certainly woulnd't have given Fable 2 Game of the year. mostly because I felt like Fallout 3 deserved it. but it was a very fun game. I don't remember all the glitches that you mentioned though Hrolf

@RROD, well I know Raymond and trust his opinion… so that's a different story. I guess when you read the same persons reviews/articles you get a good idea of what they like in games. I don't have time to build a new editor relationship :)
Hrolf

@ Thundercracker

can we PLEASE keep flamewars off this site? there is no need for it. save it for the FEED forums where we can all piss others off. Im really enjoying the banter back and forth from everyone and were all gamers here with a love for playing games. no need to bash our systems of choice.
Shockwave562

I think this whole problem with the review process comes down to the bottom line. unfortunately if that's the only line you read you're not going to be fully educated on a game. I'm more worried about spending $60 on a game that I don't actually play more than anything.

If I know the style of the game and have read a few reviews to get an idea of what the game is about then I don't have a problem taking the plunge. There have been plenty of games that I bought and only spent an hour or 2 on though… these things happen because my gaming taste is different from everyone else
Empyu

@Thundercracker

I hate flame wars… but you're not actually starting one. It could be seen as starting a valid debate on value vs reward for using XBox Live over PSN.

It's only a flame war when people start calling each other trolls and kids and other things that usually get censored. Hey does this board filter cursing?
Hrolf

@ shockwave

My wife had to restart her game 3 times and I had to restart the game 2 times. SHE finally beat the game, after getting to the 3rd hero each time before glitches killed the game, I am just past getting the third hero and cant leave the map I am on without the game freezing. instead of some stupid DLC that was obviously meant to further rip money out of peoples hands, they should have fixed the problems first.

lol, I wont even get into how badly I want to punch Peter Molynoux's face in for the iles he tells about his games.
oOcornflakeOo

@ray,

kinda creepy..
Shockwave562

@ Hrolf…. Here here. Sony and MS fans debating together without hate FTW!!
- http://rpad.tv raymond padilla
  
  @Shockwave "Knowing" reviewers is part of the deal too. The idea is that enthusiast gamers get to the point where they say, "This guy Fran at IGN. I like the games he likes for the same reasons he likes them. I'm going to listen to him," or, "This guy **** is nuts for thinking so much of Mabel 2 (not the game's real name). I'm not listening to his opinion anymore. "
  
  Some people are Eberts. Others are Ropers.
RRODisHere

@Empyu

Games are too expensive to do trial and error. The reviews will give you a good enough idea of what to expect.
Shockwave562

wow, sorry Hrolf I didn't know about any of that. no problems with mine. maybe there was a disc problem?
http://www.gamerswishlist.com/ Nightshade386

Ironically, I posted an article on this subject titled "Why Review Scores Matter….And Why They Don't" on the start up site I write for. If you want to check it out, here's the direct link:

http://www.gamerswishlist.com/news/207-Why-Review…

The emphasis for this was that we'd just given a somewhat low score to Guitar Hero 5, (the reviewer felt the songs offered should have been DLC and that there wasn't enough there to justify a full $60 release.), and I had the Halo 3: ODST review about to go up. So we figured it was as good a time as any to explain our scoring system.
tokz_21

@Rpad

hahaha Only you….

This review scale is getting out hand. look what you started Rpad.
Thundercracker

yeah i know, that wasnt a flame war i was starting there

if cross game chat is included in the next firmware update like has been speculated, is there really a reason to still consider xbl that much better than psn?

all ive ever wanted is my music to be able to play while im playing a game…i guess i answered my own question
Hrolf

"i love flame wars, ill never stop" kinda implies he wants to start one.
Thundercracker

@ray

i havent felt like that since my time at neverland

you arent gonna give us any jesus juice are you dude?
Hrolf

lol yes there was a disc problem, they went gold on a game that should have been in testing for 6 more months
Shockwave562

The nice thing for me Hrolf is that I can have both PSN and XBL for $50 per yer. if they both cost money I'd probably have to choose one or the other but thankfully I don't
Empyu

@RRODisHere

I'm not saying to go by trail and error, but there are other resources available other than just reviews. Don't you think that someone who's watch previews, gameplay videos, and tried the demo has more information than a person who's just looked at a metacritic score?

@Shockwave562

"There have been plenty of games that I bought and only spent an hour or 2 on though… these things happen because my gaming taste is different from everyone else."

This was my point exactly. Metacritic tries to fit everyone into a cookie cutter and assumes everyone will have the same experience with every game. It just doesn't work that way.
- http://rpad.tv raymond padilla
  
  Also, no way I could have written my last few comments so quickly on WordPress for iPhone.
Shockwave562

I know i know but I meant your disc, as in if you got a new one your problems wouldn't have happened. I don't recall seeing any articles about a major fable 2 glitching problem
oOcornflakeOo

@thundercracker

nuh uhh!!! xbox live it soooo worth my $60 on top of the fact that i have to pay for broadband on the side!!

(joke)
Shockwave562

#67 was at thunder, oops
Thundercracker

@hrolf

i will never stop

you want me to lie and tell people that there are plenty of good reasons to own an xbox?

ok….
Thundercracker

wow, that xbox exclusive lineup is looking good!
Empyu

@Thundercracker

"all ive ever wanted is my music to be able to play while im playing a game…i guess i answered my own question"

You know what console did that the best? The original XBox. I was so surprised when the 360 and PS3 couldn't handle music files the same way the first XBox did.
Shockwave562

@ Thunder, agreed , Mass Effect 2, Splinter Cell and Crackdown are very high on my list right now.

same with Heavy Rain and GOW 3
Empyu

@Ray

All I've been saying is that reviews would be better, and more interesting if the two sides got together more often to do their reviews.
Shockwave562

ya Ray I will admit iPhone wouldn't worth with the length of this page
Thundercracker

@shockwave

damn straight, the xbox kicks ass…reliable, quiet AND such a versatile and robust library of exclusive software released this year!
Shockwave562

#79 worth – work**
RRODisHere

@Empyu

A game is what it's Metacritic average says it is. Demos can be misleading. Devs often give you the best part of the game in the demo and the rest of the game is less.
Shockwave562

I'm glad you're finally coming around. I think Mass Effect 2 will be my GOTY next year for sure. I'm just thankful I had the ability to get both systems so I wouldn't miss out on Uncharted 2 or Mass Effect. I know it will come to PS3 eventually but I'm sort of a here and now guy. especially since I have 3-4 characters from ME 1 to import
SlickyFats

I like the "rent it, buy it, pass" system that DVDuesday uses. I don't know why others couldn't do the same. I mean it would have to be utter crap to say pass. But nonetheless I tend to rely on renting or discount games if I am on the fence about a title. I can read reviews all day but that still doesn't mean it is going to be a good game once I start playing it. Heck I read Sark's review of ODST and many others, I still didn't care for it. Reviewers opinions have become less of a priority for me. Heck i trust the people on here more. I am still looking for Demon Souls because it was suggested.
tokz_21

@rpad

I don't have that problem with posting comments with my iphone using safari. do you have to do something different since your the administrator or keeper of the blog.
Hrolf

Just because YOU think the xbox is crap does not mean everyone does. there are plenty of games that are Xbox only that are great. not to mention some of the cross platform games look like garbage on the PS3. Regardless, we all agree we like to play games, I dont see any reason to attack someone for their system/systems of choice, unless they ONLY own a WII and think the games way better then anything Xbox or PS3 can do.
oOcornflakeOo

the price to play online is a bitter/sweet topic because for some they still pay to play while others can join online right outa the box. its not flaming, some appreciate their consoles simply because of that while others become livid for being called out on their senseless spending. its no wonder why gamers become so polarized and flame.
http://www.gamerswishlist.com/ Nightshade386

@Shockwave: I have 6 characters to import. On for each character class. It's a sickness with me. I love me some Mass Effect…..
Shockwave562

@ RROD. is it just me or is it crap games that release demos? It always seems like the games I know will be good don't even do a demo until a few months after release.

I've liked maybe 2-3 demos (not including Beta) out of all the ones I've played on PS3 and 360
Shockwave562

@ Tokz… sure but don't you have to scroll forever on safari to get to the bottom?
tokz_21

@Shockwave

only after i refresh the page and there are 100 plus comments which we are getting to.

Other than that i don't have any issues.
Shockwave562

@nightshade, haha, that is determination my friend. I generally do one good and one evil but I also added in some female, and another character class with Mass Effect
Shockwave562

I hear ya, I mean I wouldn't mind posting on mine but after a while the scrolling would be a nightmare. they should do something about that I think
Empyu

@RROD

That's not exactly true. Most developers want to hide a lot of what's in the game, so they won't always put their best foot forward for the demo

What you can take from a demo is how the game feels and how much you might enjoy its core gameplay. If you don't trust the content in a demo, there are plenty of gaming journalist sites that post their gameplay videos of the game. And those are definitely not provided by the game's developer.

Newer Comments »« Older Comments

Coffee Talk #4: Game Review Scores and You

Sponsors

Categories

Recent Comments

Support RPad.TV (please)!

Today's Poll

Buy RPad.tv Merchandise (Please!)