Coffee Talk #4: Game Review Scores and You

Posted on October 15th, by raymond padilla in Coffee Talk, PC Gaming, Today's Specials, Videogames. 257 comments

Welcome to Coffee Talk! Let’s start off the day by discussing whatever is on your (nerd chic) mind. Every morning I’ll kick off a discussion and I’m counting on you to participate in it. If you’re not feelin’ my topic, feel free to start a chat with your fellow readers and see where it takes you. Whether you’re talking about videogames, whether the Motorola Cliq will turn the company around, why the hell it’s raining in Los Angeles, or bellybutton lint, Coffee Talk is the place to do it.

In Coffee Talk #2, reader rbee90 brought up the topic of game reviews, which led to a conversation about review scores. The discussion started to get interesting and reader RRODisHere suggested that I write about the topic in Coffee Talk. Well here it is!

I have a ton of problems with the way most — not all — game reviews work. Scoring is a huge pet peeve of mine. 100-point scales are just stupid. I’d love for someone to (intelligently) explain the one-point difference between a game that gets an 87 and one that gets an 88. Five-point scales — which I like a whole lot better — are a problem because of the way the business uses scores and how some consumers interpret them. Here’s a pro tip for you — three stars out of five is not the same as 60 percent. Yet that’s the way a three-out-of-five is treated by review aggregators (most of the time). What’s worse is that some publishers base royalties on aggregate review scores, which is completely unfair to developers.

Borat Thumbs Up Coffee Talk #4: Game Review Scores and You

Personally, I think there should only be three review scores — buy it, rent it, eff it, symbolized by thumbs up, thumbs in the middle, and a thumbs down (or Megan Fox’s thumbs). Isn’t purchasing, renting, or passing what it all comes down to anyway? I pushed for this system when I worked at GameSpy, but nobody was buying it. Oddly enough, my boss at GameSpy eventually went to Crispy Gamer, which uses a scale like the one I suggested. Anyway, the bottom line is that scores have become so important that the words behind them are often overlooked and sometimes ignored.

Then there’s the way some games are reviewed. Some publishers send code to reviewers days before they’re allowed to publish their reviews. For competitive reasons, everyone wants to get the review up the second the embargo lifts. This has the reviewer cramming a pint glass of gameplay into a shot glass of time. Another practice that bugs the hell out of me is when publishers have reviewers play the game off site. In these cases, a reviewer has to commute to a hotel suite or a conference room to play the game for a few days before writing the review. Again, the short amount of time introduces a problem, but it’s compounded by having to play the game in a completely unnatural setting. My issue here is that reviewers have to play games in a way that few consumers would. Do most people play 50-hour games in three days? Of course not. Do most people make daily commutes to play games in a conference room? No.

Okay, I’m getting angry about the whole deal. What I’d like to know from you is what you expect from game reviews. Do you like like 100-point scales or do you prefer five-star systems? What information is most important to you in a game review? Do you think that the unnatural way reviewers have to play games leads to an unnatural view of the game? Leave a comment and let me know (please)!

Coffee Talk PC Gaming Videogames

N8R

In all honesty, I don't pat attention to reviews at all. I check out the previews and the demos, and even then I usually stand in front of the rack completely indecisive.

I like to hear what the community says about stuff first. Honestly, alot of the people here and my online friends have helped me decide most of the games I've gotten over the past few years. I rarely pre-order stuff unless it's a sequel to a game I loved or made by a developer that I'm a fan of.

Bottom line, scores mean nothing compared to the words of people I trust to be just as much of a gamer as me. That's my opinion.
Thundercracker

i really wanted demon's souls because of all the fantastic reviews, but i just watched a half hour long video review of it on giantbomb.com

it doesnt look like my kind of game, really

ill get it, but ill get it for like 30 bucks
Thundercracker

hmmm
Thundercracker

dude, it wont let me post anything
Hrolf

A numerical value does little for me. What I like are peoples likes and dislikes about a game. It gives me a chance to help better decide which game I will purchase, but honestly, if it is a big name game, I already have a good idea if I will like it or not. Wet is a great example of how a demo works for me. Reading about the game, and knowing Bethesda had something to do with it almost guaranteed a purchase. Thank the gods I played the demo because after 3 min. I knew that for me the game blew chunks. It reminded me of John Woos Stranglehold which got tired fast. between the slo mo, mini game button mashing, and filtered graphics.

Reviews are just a tool for someone to use to better decide their purchase, but it should not be the only thing a person goes by.
Thundercracker

i just watched a half hour review of demons souls on giantbomb.com, and it seems to me that im gonna wait to buy that one used. It just strikes me like its not….whats the word im looking for?

fun
Thundercracker

ok, so i cant post long comments today…went to giantbomb.com

watched 30 minute video of demons souls

despite good reviews ill be buying it used
Thundercracker

you are great

come to village stop

infamous el guapo

put on show

100,000 pesos
Thundercracker

im having technical difficulties

i just watched a half hour review of demons souls on giantbomb.com, and it seems to me that im gonna wait to buy that one used. It just strikes me like its not….whats the word im looking for?

fun
Thundercracker

im having technical difficulties guys
N8R

@ Thundercraker

3 Amigos.
Thundercracker

i just watched a half hour review of demons souls on giantbomb.com, and it seems to me that im gonna wait to buy that one used. It just strikes me like its not….whats the word im looking for?

fun!
Thundercracker

i have long comment, but long comment no work for me today, must make shorter

fire bad
tokz_21

I like your buy it, rent it, or eff it rating.

That's all i really want to know, it if it's worth all my money or is it just experiencing the game for a bit and be done with it.
Thundercracker

i just watched a half hour review of demons souls on giantbomb.com, and it seems to me that im gonna wait to buy that one used. It just strikes me like its not….whats the word im looking for?

fun..
Thundercracker

giantbomb.com, demons souls video review, not impressed that much, i buy used
Thundercracker

i buy demons souls used, not impressed
Thundercracker

saw 30 minute review on giantbomb.com. no look fun
redtailman

Uncharted 2 is the perfect game, at least in my book
Smartguy

With Ray's system there is still a flaw. Games are liked or disliked based on the person playing the game. What Ray deems as a buy it game might be a rent or a no thankyou for me. For example, RRODishere keeps saying he settled for Brutal Legend. I think that game is an emphatic buy. I buy that game for a few reasons: 1) I think it is good 2) I want to support developers who do things a bit unorthodox and succeed. Since there is relative value in all titles I think a system should be developed that creates benchmark standards and then the intangibles are what separates the game in the review process. (For what it is worth I like the first Kingdom Hearts because it had great style..BL is the first game this generation that gives me the same feeling)

Defining those standards and evolving them incrementally per generation of course is easier said than done. Let's take gfx for example. Should all games be based on the same scale no matter which type of animation style they use or which game engine is used? Is it fair to compare the id engine to the Unreal engine? Cellshaded vs something like UC2? I think each engine should be subject to benchmark tests and then judged by how well the developers did with utilizing what is available. Then the intangible arguments of how well sound was used (not sounds), story is told, voice acting, etc.
Smartguy

@Thundercracker

Really? What don't you like? I like the dungeon crawler aspect of it. The difficulty adds great tension to the gloomy atmosphere.
Thundercracker

looks frustrating….i cant post long comments today for some reason

im still getting it, just not for 60, when theres so much else right now
Thundercracker

i like giantbomb.com for game reviews, they seem to like the same games i like, they hated demons souls, and id rather get arkham asylum right now than take a chance
Thundercracker

lol

i figured it out, im trying to use another websites name in this comment

g

i

a

n

t

b

o

m

b

.c

o

m

anyways, that website has a review of demons souls that is scaring me off for now

ill just get Arkham asylum
Smartguy

The learning curve in the first "real" dungeon better known as Boletaria Palace can be frustrating. However afterwards you will ask yourself why it was so hard. The game is quite rewarding though.

I hear ya on the fact there is so much out right now.
Thundercracker

cant go wrong getting arkham asylum instead
http://twitter.com/Sandrock323 Sandrock323

My only problem with your buy/rent/eff it ratings is that you are telling someone what to do. A game you say is a rent could be a buy for some and a game you call a buy might be an eff it to others. Good, bad, and great are better words for it I think. All in all, I've pretty much stopped reading reviews. (but to be fair, it's not like anyone has to read any this year) When Fallout 3 came out, I preordered it and bought it based on reviews. Long story short, I forgot how much I hated ammo management in RPGs, until I played that game.
RRODisHere

@N8R

If you don't pay attention to reviews them I'm sure you've wasted $60 on bad games before.

Reviews are important for me. The reviews ARE accurate. Wet go bad reviews because it's a bad game, Uncharted 2 got great reviews because it's a great game. Anyone who say they don't listen to reviews have bought a lot of bad games.
Thundercracker

i actually enjoy metacritic

i dont take it as gospel though

if a game gets an 85 or above, its usually a damn good game. The people at ign, gamespot, gamespy, etc, etc cant ALL be wrong

im much more concerned with movie reviews than game reviews because im at least somewhat confident that game reviewers are more my age and appreciate the same things i do
Shockwave562

I've grown used to a 10 point system. but that's probably because I am used to IGN as I've been reading their reviews most for about 10 years or more. I think I do look at the score because I generally know what that means to IGN.

What it comes down to if it's a game that I'm really on the fence on I'll read the whole review. usually a good reviewer will indicate whether or not the game is worth renting buying, or just not trying it at all.

When it comes down to it my game time has really decreased in my adult life. I don't have time to play as many games as I'd like anymore and the rent it ones for me are just not worth my time (such as Banjo Kazooie and Mirror's Edge). I really just want to know which games I should buy
RRODisHere

I can't tell you the difference in an 87 and an 88 score, but I CAN tell you the difference between 2 games that get a 5/5 score like Uncharted 2 and Brutal Legend.

I think the best rating system is the 10 point scale with halves. How many games that got 5/5 would get 10/10?
Shockwave562

@ Thunder, I was on the fence about that game. what don't you like about it? Is it the difficulty or is it the style? Based on the videos I didn't think I would like it but the reviews have been really good so I thought there had to be some reason
Thundercracker

i totally agree there shockwave, there is enough material out there on the web for someone to get a really good idea on whether or not they would enjoy a game. Thats why i like metacritic, because it takes into consideration so many different opinions. I do research before i drop my 60 bones
tokz_21

All reviews have flaws that's why there are different ratings. I figure that it would work for, let's say i'm on the fence about a game but Rpad or someone else recommends it to buy it. I'd go rent the game or go play at a friends house if they have it and make my decision from there. Or if it was a labeled as a rent i would rent it and see if it was worth my money.
Thundercracker

@shockwave

you and smartguy both telling me that i would like that game means more to me than a single review from anywhere, but i dont have a lot of time anymore for games ( i wish i still did). I still need to get fight night 4, arkham asylum and uncharted. By the time im somewhat tired with those games, ill be able to get demons souls for 20 bucks..and i will still get it
Shockwave562

I agree RROD, I think 10 point is a lot better than a 5 point. though I never really liked G4s scoring I did however enjoy some of the actual reviews.

Sorry Thunder I thought you said you did buy it. that game is going to have to wait for me too. it sounds like it is worth trying but I'll get it maybe when there's a lag next year sometime. although winter/spring are shaping up nicely with some good releases
Shockwave562

haha, no Thunder I'm still on the fence, sorry bout that
Hrolf

@ RROD

Uncharted 2 and Brutal legend are 2 completely different styles of game though, is it fair to compare them together?
RRODisHere

@shockwave

I think Metacritic is the best place to go. It averages all the review scores. At the very least, you can read all of the reviews because they post links to all of them. Don't ever just go with one review. IGN is the worst to me because they give 9s to every game that has just a little hype. I think IGN rewiews on hype. IGN are the only ones who gave Brutal Legend a 9. I ignore IGN.
Shockwave562

@ Hrolf, I agree but all games are compared when it comes to their score. especially when both those scores came from the same source. it is definitely a difference of opinion maybe but on a 5 point scale Uncharted 2 should be like a 6.

Brutal Legend could be a great game but perfect? I don't know
Shockwave562

ya RROD I hear you, generally I find though that IGNs score will be close to the metacritic average. I've learned that a lot of the big places will have similar scores.

I really go to IGN because I enjoy reading their reviews. like I said it isn't quite as much about the score as the words to me
RRODisHere

Look up Uncharted 2 on Metacritic. There are 62 reviews and none of them are under a 9 and they all average out to a 96%. Everyone agrees that Uncharted 2 is great.
- http://rpad.tv raymond padilla
  
  Guys, one of the points of buy it, rent it, eff it is to emphasize the words of the review. SO much of this is subjective! Using a 100-point system, the same game will be a different score to two different reviewers and two different consumers. With the system I prefer you have a simple guideline with the words backing up the reasoning.
  
  @RROD My big issue with the ten-point system is that many outlets use it differently, but it's treated the same by aggregators. Some use 5 (EGM) as the average, while others use 7(Game Informer) as the average. Using the GI system is stupid because you waste so much of the scale.
  
  @Everyone I'm posting this in bed from WordPress for BlackBerry. Frickin' mosquito bit me and woke me up. He gets an "eff it" for the reason stated in the last sentence, as well as my two new itchy spots.
Shockwave562

I only trust the big names on Metacritic though. I hate seeing some obscure name on their with a vastly different score than everyone else. Truth be told I am usually more interested in the negatives of a game than the positives but I think a lot of reviews that end up on metacritic are biased
Smartguy

@shockwave

Brutal Legend is a great game. It isn't perfect. Perfection is very rare. I can only name a few that I think are perfect.

Warcraft 2 and 3. Diablo 1 and 2. Ocarina of Time, Windwaker, Twin Snakes, Snake Eater, Soul Calibur (DC version). That's about it.
RRODisHere

@Hrolf

No, it's not fair to compare them but everybody keeps doing it. Yesterday on the Feed, Patrick Klepek posted a topic titled "Uncharted 2 or Brutal Legend, which one did you buy?".
Smartguy

@RROD

I bought both. No remorse lol.

Saints > Falcons
rbee90

why i hate reviews sooooooo damn much is because, if one dude says a bad thing about this game, the consumer will probably but most probably would not buy that game. do you guys remember when sessler was ranting on RE5? i totally believed him, and on that same day, my brother bought RE5, and it was a $#!t load of fun. I was that close of not buying an amazing game, because this game website or some dude said it sucked. i never trusted scoring systems since then. one other review site i really loathe is IGN. the reviewers are kinda stupid. some people would think if that game scored a 10 or a 9, its like the best game in the world…thats BS…dead space scored an 8, but it was one of the best original IP's released that year. same thing goes for mirror's edge. AND the most underrated game i ever played last year…VALKYRIA CHRONICLES. now that was an awesome game. people just *meh'd* it because its *NOT* final fantasy or something. F-tards.
oOcornflakeOo

really, the most natural review system for all of us SHOULD be the 1-10 scale.. we all have a great understanding of the diff between an 8 or a 9 and a 10.. i think its when we get into the "__ .5" business that the scores become a bit arbitrary. and even though someone's 7 may not be the same as anothers, i can interpret that the reviewer, at the very least, couldn't rate it an 8-10 so theres gotta be reason as to why not and thats when my judgment comes in. people should really read reviews though to have some sort of understanding as to what the reviewer was grading against. I'm all for the 1-10 scale.. sorry Ray, but the buy it rent it eff it is just too vague and would need a serious list of pros and cons to accompany it for me to really give this type of scoring a moment of my time.. =/
Hrolf

Well, I am not sure if there is such a thing as a "perfect" game. looking back at some of my most loved and favored games in the past, there were things about all of them i would have changed, Including Fallout 3, which I have put more time in so far then just about any game I have ever played, minus Ultima 6, 7, and 7 part 2.

Uncharted 2 looks AMAZING, too bad I dont have the cash for a PS3 right now but thats probably for the better, next year i really think sony is gonna FINALLY play catch up and kick some serious ass.

Brutal Legend looks really good too, but in a completely different way. Uncharted looks to take itself as a more serious style of game while Brutal Legend is alot more comedic and unusual
RRODisHere

@Shockwave

What's wrong with the small sites? The IGN guy's opinion is no more important than the small site guy's opinion. Are you telling me that Raymond Padilla's opinion on a review doesn't matter anymore because he's no longer working for G4?

Newer Comments »

Coffee Talk #4: Game Review Scores and You

Sponsors

Categories

Recent Comments

Support RPad.TV (please)!

Today's Poll

Buy RPad.tv Merchandise (Please!)