Are video game reviews in need of a massive structural overhaul? I began pondering this after the launch of Marvel Heroes. It launched out of beta in a less than perfect state. The UI was ugly, the drop rates weren’t balanced, and there wasn’t any endgame content. The core combat system worked but it lacked that “just one more level” hook that so many point and click RPGs are based on. Flash forward a year and the game is indistinguishable from its original build. Every aspect the game has been polished and improved upon. Yet, any review will still be about the mediocre launch version. Its metacritic is an unimpressive 58. Gazillion decided that with one of it’s latest larger updates it would also change the name of the game. The metacritic for Marvel Heroes 2015 is an 81, though it has a third of the amount of reviews. The introduction of patches and DLC on consoles has changed video games from a static medium to an evolving one, and reviews will need to do the same to stay relevant.

Video games were originally static products, much like other forms of media. Once a game was released very little, if any, changes were made since that would require a ton of work and money to get the updated versions into arcades and homes. Bugs, glitches, and exploits were usually never patched because of this. A side effect of these games being stuck in a fixed state was that reviews would eternally be accurate. Just like an old record or film, playing Pac-Man today is no different than playing Pac-Man two decades ago. On the other side of the spectrum we have MMO style games. Games like Marvel Heroes and World of Warcraft see new content and patches years after they were launched. The best example of this is Everquest, originally released in 1999, is still receiving content 15 years later. Even in the non-MMO AAA sphere we have seen a shift in game design based around longevity. Season Passes, map packs, more weapons, additional missions/story content, and increased level caps are all added post launch in an attempt to keep gamers playing a game after the first week.

My first reaction to this was the abolition of reviews.  I haven’t read one in a few years, instead relying on recommendation from friends, and I haven’t regretting any purchases. I’m pretty sure I’m in the minority with this though. The obvious quick fix is to adapt the reviewing process to the medium. This requires reviews to be updated when they no longer provide accurate critiques about the game. This simultaneously preserves the benefits of having reviews when a game launches and ensures validity later as content is patched in. There are two issues with this style of review though. At what threshold of accuracy must a review be amended and and at what point do we stop updating a review. Early Access games and MMOs see frequent updates and while most of these individual updates feature small changes they add up to something greater than their whole. Much like living with a pet or child you rarely notice how much they grow until you see a picture of them a year ago. Going back to Marvel Heroes, each update felt like baby steps towards making a better game but comparing the current state with the launch version is a drastic difference. When engrossed in a game it becomes difficult to tell when the little updates have created an experience that differs from an earlier version, and thus it becomes difficult to tell when a review should be updated. Should we as journalists develop and hone this skill we run into the next issue, when do we stop updating a critique? Games such as World of Warcraft and Everquest will require review upkeep decades later. The current style of looking at expansion packs individually works pretty well but the shift to digital and free to play means we rarely see expansion packs in this fashion. Both Marvel Heroes and Path of Exile have seen success with this format but they don’t give journalists definitive times to check back into a game to see what is new and what has been changed. Unfortunately, I don’t currently have an answer to this question. If a journalist was required to play an MMO until the servers shut down just to ensure accuracy I doubt too many of them would ask for or take the job. That is a pretty daunting task and I feel the tone of most reviews would switch from optimistic to jaded as the journalist became weary of playing the game.

Alternatively, we could wait to write reviews until after launch. This would erase the need for updating as well as remove any issues with embargos and the inability to review multiplayer aspects. It also allows journalists to write about games they find important instead of inadvertently adding hype to titles that already have more than enough marketing. This method disrupts the current idea of journalism, however, and I’m sure there would be negativity towards it both from publishers and consumers. For many games the opening weeks tend to see the largest amount of sales and reviews help put the spotlight on them during their launch. Consumers eager to play a game as early as possible also rely on launch reviews to help determine if a game is worth buying at its most expensive price. While this style of review may aid latecomers in the purchase of a game it completely abandons the benefits of day one reviews. Ideally, both of these styles could work in tandem to create a plethora of reviews that provide information for both day one purchasers and late buyers.

The games industry grows and expands at a rapid rate but the review hasn’t changed at all and never has this been so obvious. Free to play MMOs and Early Access games are already causing issues among reviewers as they know their critique will eventually be outdated, possibly in just a couple of weeks. Unfortunately, finding a solution to this issue isn’t as easy as recognizing it. While we can simply update reviews there isn’t any clear answer as to how often or when to stop. Hopefully the solution becomes clearer as we adjust to this change in game design. Maybe in a year we will have several small updates to the reviewing process that transform it into something indistinguishable from the current state.

 

About The Author

Contributor

Bryan is fascinated with the potential of video games as a story telling medium, both through narrative and mechanics. He loves playing games with deep systems and mechanics, giving players lots of room to tinker in the games in search of optimization. This has led him to favor fighting games and RPG though he has a soft spot for fast paced FPS titles and their twitchy, reaction based skill set. Outside of video games he enjoys programming, fiction writing, and music composition and performance.

Related Posts

  • Games are just software. I think we’d all be better off if they were treated more like say the Firefox web browser. Instead of being artifacts encased in ember until the end of time they could be always changing, evolving, forking, becoming better, so to meet everyone’s tastes and so your children’s children so-to-speak will ultimately have a better version of that game/story.

    That’s how stories used to work. The idea of recording, copyrighting, closed-culture, and so on is a relatively new invention. In this brave new world all you really have is word of mouth. The only consolation is that now we have the technology to slap a version on every single iteration so that if you really want you can recommend/comeback to that same exact version.

    • Bryan Rumsey

      I like your analogy but you are missing one point. The old “word of mouth” style of telling stories usually only saw alterations in the story when it was told by a new person. In gaming this could be seen in either mods, such as Black Mesa, or in HD remakes. The gist of the game is still there but some things have been altered to appeal to the new audience. And as you say, there is a version that meets everyone’s tastes whether they are purists who love the original or newcomers who enjoy the updated graphics, story, or controls. Going back to reviewing games, should reviewers then also consider the mod community while critiquing a game? For instance, I would give Elder Scrolls: Oblivion a 4/10 in it’s factory shipped state but an 8/10 when using Oscuro’s Oblivion Overhaul, which I believe fixed a lot of the mechanics I didn’t enjoy in the base game.

      • Hey, good to see one of your journos down in the pit.

        In reply, I am generally for radical democratization of media, because it leads to better media. I’m a media fiend and so a lot of the time I am starved for good media. Personally I believe that what a game looks like (for example) should be a user preference, and that there should be standards around game formats so that its practical to move fluidly between them experiencing different stories and ideas like with traditional media instead of remapping your brain so-to-speak every time to a different “game”.

        I literally believe that everytime someone tells a story it changes. Everyone plays games and thinks that it should have been a little different this way or that, and if they actually go in and tweak that parameter to suit themselves for the time they’ll be playing the game, then there you have a mutation that can also be published. I don’t know what role there is there for journalists, except if they also really love the media they could play an important role by tracking all of the mutations and presenting them in a way that basically helps the community of authors/players make decisions. Then you can have cross pollination where the mutations are combined into single super games/media.

        This is my personal project, the analogy to Firefox isn’t so much to say that games are software… they are not necessarily, but they are digital. But they represent stories and ideas, or at leas the kind that I prefer that look like traditional media do. But to put in a more blunt way, you can say lets opensource games. I think the only barrier to that is people who hold the games dear and obsess over the minutia of them to detriment of progress. So, if you look at games that way, at the very basic level journalists I suppose can report on new features and activities within communities.

        One criticism I have of this site and many others that cover games, is they tend to behave as if they are supplementing an intense interest in games. Anywhere that games are showcased they are done so as if the onlookers are going to seriously turn around and begin researching the games then and there. Like if you go into Sony’s store, you can’t find out anything if you even get a screenshot. You the consumer are expected to do your research. I think this is a wrongheaded model, and what journalists should be doing is acting as agents that make it easier to take in and digest lots of information. Basically journalists need to be the pre-digestion system for us all. Time Magazine does a good job of that function.

        Cooking food is basically pre-digestion. It means your body can get much more out of the food because it doesn’t have to burn as many calories just to digest it. Journalists should think of themselves as cooks, and perhaps even in order to do that some will need to specialize around certain communities, and others will have to take a broader approach in order to be able communicate the overview.

        P.S. Big marketing/production companies are probably not going anywhere. They will still survey the landscape and select the most popular games (of which there could literally be an infinite number) and adapt them for whatever the mean commercial audience of the day is according to their calculations. And that’s important too because they are often the gateways into a rich media life and often drive the development of new technologies that require large material investments. But everything digital eventually goes opensource. When innovation turns inward there is nowhere left to go but open, free, and plentiful. Even games will go there. And the books and movies and music and everything else will follow games, because print (conversion into 2D static serials) is dying.

        • Bryan Rumsey

          While it is way off of the topic of reviews, I’m curious about your idea of standardizing gameplay. I can’t see this doing anything positive for the games industry. Telling developers how to make an engine and how to map controls will stifle any sort of innovation in that aspect of game design, and that is half of the process! Your idea that games should be like traditional media is, in my opinion, flawed as it attempts to remove the key concept that gaming focuses around: story told through interactivity. Not all narrative pieces are told through dialog or cutscenes. Some narrative is told through the mechanics, through the controls and the game engine/systems. Imagine a world where all hack ‘n’ slash games required the basis of fluid combat with high combo numbers akin to Devil May Cry or God of War. Dark Souls would then have been built on this idea. While the story wouldn’t have changed, the feel of the game would clash with the feel of the narrative. The FPS genre, which is the closest to your idea of a unified game experience, benefits from having innovation in mechanics. The modern ideas of regenerating health, iron sights, and mouse look would never have come about if we had stuck with how Doom did everything. Granted I prefer Doom’s style to CoD’s but the branching idea of mechanics has brought more people to this genre.

          This branching of ideas is no different than your game mutations. Along the way in the FPS genre, one group of developers added mouse look to a game. BAM, a branching idea. Along the way in the RPG genre, someone decided to map skills to a hotkey bar instead of in menus. BAM, a branching idea. You speak of games as eventually going open source but wish to impose how these games can be edited to ensure that the act of gaming, or controlling a game, is identical across the board. That seems to be in direct opposition of everything opensource stands for, which is usually power to the people to improve an idea in any way.

          With that said, I think your heart is in the right place, accessibility. What keeps games outside of the mainstream is being able to play them well. Where you focus on the mechanics, I would focus on interactivity. There is a reason why light gun games or Wii bowling can be enjoy by all, the controller and player input. Not only do modern controller have a plethora of buttons and joysticks but they don’t have any real world connotation as to what they do. Arcade games and home peripherals seek to emulate real life as a form of interacting with a game. This is where the Wii excelled. This is the key to accessibility. Just look at how accessible the motion based dancing games are versus the arcade dancing games of the past. Dance Central is infinitely easier to play than DDR and this is attributable to the input device.

          • It all depends on why you play. There are at least as many different ways to play as there is musical genres. That doesn’t mean that each way cannot or should not be standardized and any variation could be wrapped into user preferences or extensions.

            The trouble with always reinventing the wheel is it makes it nigh impossible to just make a game in the first place. Look at RPG Maker. While I’m not familiar with it myself, but it is basically doing things right. It’s standardizing somewhat around that genre/format and presenting genre/format appropriate tools that are high-level enough that anyone can use it to just make a game to get their ideas across with.

            Then if we let games evolve (on-topic enough) you can take a game made by someone that may be very good in some ways, and very raw or not good in others. People can then take that game and use their specialized abilities to improve its weak part. Then you have an asynchronous model for development, and if you are the kind of person who just likes JRPGs then you can be totally happy just playing JRPGs. I’m that way myself, I’d just assume play the same exact game always where only the story/idea content changes, because that’s what I do with books, that’s what I do with movies, and that’s basically what I do with music, so I don’t see why games should be any different. The trouble is that First-Person-Shooter is a medium every bit the way movies are a distinct medium with a distinct interface but nevertheless we lump them altogether and call FPS and Shmups games, instead of what they are.

            My point is just that if you think covering games now is untenable, just wait a decade. It’s going to become pretty much futile then unless we change the way we think, or just choose to not give coverage to large chunks of what will be happening by then.

            P.S. I’m not trolling. I just want to see this site grow, and I think its comments sections need to look alive if its going to appear to have a readership, so by posting whatever I can as a comment I’m just hoping to make others feel more comfortable to be seen.

          • Bryan Rumsey

            I appreciate your enthusiasm about this topic and I am beginning to agree a bit more, at least on the topic of asynchronous development. I think that would be an interesting game design idea but not something that the entire industry would benefit off of. I do like the idea of a standardized format to play a genre with variation available for users, but this also already exists. Going back to the FPS, which is again the most standardized, we have had pretty much the same default controls for 5 years or so. Even though keyboard+mouse and gamepad are so different, each have their own standard mappings, and most games allow the user to switch between a few different configurations or entire remap to their will. Even game design ideas have been standardized such as only carrying two weapons, iron sights increasing accuracy, and unlocking weapons in MP via XP. Of course RPGs are the most notorious for creating entirely new systems, stats, spells, etc so your genre of choice does see the most fragmentation. As you said though, everyone plays games for different reasons. You would be happy to play the “same game” if only the story would different. I tend to play RPGs for their mechanics and the ability for me to work within the system to create optimal or overpowered builds. Thus, for my gaming needs, a central game system would cause me to lose interest in the RPG genre due to stagnation in the game engine.

            I do agree about the futility of being relevant in games journalism. Just looking at Steam alone we can find an average of 3-4 games released per day. PER DAY! It is impossible to believe an outlet could keep up, and even then that the viewer base would want to read that many reviews in a day. That is what niche outlets do, like Indie Haven. We focus only on indie games and it drastically reduces the amount of news that we feel we must cover. It is still a ton of content to discover, play, and write about but it also allows us to ensure this market has a journalistic outlet. The advent of WordPress and YouTube are allowing people to do the same thing, filling whatever niche they wish. Steam reviews have created another area for users to practice a form of journalism that can cover games. While there are more games than ever to play, there are also more ways for people to rise up to cover them. Before we only had magazines and if one of the 3-4 publications didn’t cover a game nobody heard about it. Now anyone can cover games.

          • I am enthusiastic about the topic of games in general, as a person who has been ever frustrated with them my whole adult life. The medium has failed to live up to much/take a seat at the grownup table of literary media. Not every game actually is media, but that’s another story.

            This year I am trying to begin making an effort to mingle with the “indie” game scene, but I am learning that there doesn’t seem to actually be one; not online anyway. Which is fine, but different from the impression you get from its treatment in the mainstream press. I’ve foresworn mass media games (somewhere in the middle of the last round of consoles) so it’s great that I learned about this website not long ago, since it makes it where you don’t have to wade through their coverage.

            BTW: You are arguing from the position that I am proposing something that will artificially restrict things somehow, when really your position is you are going to somehow artificially restrict the forward march of technology. The games are going to be made in whatever way produces the best games for the least amount of work. I’m just telling you how things are going to be. It’s just a matter of the technologies coming into being; not even a proposal, although I don’t shy from putting ideas in peoples heads when a door opens in conversation. I work on these things for my preferred genre/format (which I think is emerging as the dominant one) and the products of my work will have to get out into the wild before too long, so mingling becomes an unavoidable part of that work.

            You guys are doing everything right here. You shouldn’t have 0 comments on every article unless there just isn’t a healthy DIY scene yet. So we are squarely in a proto-culture phase. If I could do your work and my own work I’d do yours pretty much the same way, so please don’t get discouraged; the real culture happens on the edges. I am usually happy to comment after reading something, but I think that first comment when there are 0 comments is important, because if everything is 0 comments there is a bystander effect, and most people won’t make their presence known, and you guys will never know you are being read or what.

            Journalism is of course important. People have to find out what is happening somehow. Without it we’d have to squarely choose between getting anything done ever, or not knowing what the hell is going on anywhere ever. That’s your job.