Announcement

Collapse
No announcement yet.

Valve's Fix for Combating Review Bombing is Graphs

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Valve's Fix for Combating Review Bombing is Graphs

    Review bombing is when a bunch of Steam users get together in an effort to drastically raise or drastically lower the user scores for a game. It's usually the case that a developer or publisher has done something that people don't agree with so they "get back" at them by submitting a bunch of negative reviews. Those mass spammed negative reviews are then upvoted in a mass circle jerk while any positive reviews are mass downvoted. This is all done in an effort to... do... something? I guess their end goal is just to lower a game's overall user score enough that it may dissuade people from buying it in the future. I mean, it worked out well for Grand Theft Auto V a few months ago. It worked out so well that the game still maintained the number two sales position on the NPD charts for August 2017 and was in the top 10 for the NPD every month for the past four or so years now.

    Whatever the motivations, review bombing is pretty damn stupid. Steam has a particularly nasty problem with it, so Valve decided to take some drastic action to combat it.

    They added graphs. Before arriving at their solution of adding graphs, Valve did examine other possibilities on what to do.
    So what solutions did we explore? An obvious one would be to simply remove the Review Score. Then, as a potential customer you'd be forced to read the User Reviews themselves to see if the product sounded interesting. If you saw reviews talking about something outside the scope of the game, you could decide for yourself if it was an issue that would affect your happiness with the purchase. Unfortunately, we're pretty certain that this isn't really an option - scores were added in response to player demand in the past, and that demand for a summary of some kind is likely to still be there, even if players know it isn't always accurate.

    Another idea was a temporary lock on reviewing, similar to how stock markets prevent trading on specific stocks when abnormal behavior is detected. Based on the theory that review bombs are temporary distortions, we could prevent reviews for short periods of time whenever we detect massive distortions in submissions. In the cases where the cause of the community's dissatisfaction truly affects the game's potential happiness to new customers, we're confident it would still result in the Review Score moving down after the lock period ended.

    But if we lock reviews on a product for a short period of time, what does that mean exactly? Are players no longer able to post reviews at all during that time? Or should they be able to post them, but we ignore them for the purpose of calculating the Review Score? In the end, we didn't like the way this ultimately meant restricting the ability for players to voice their opinions. We don't want to stop the community having a discussion about the issue they're unhappy about, even though there are probably better places to have that conversation than in Steam User Reviews.

    We could change the way the Review Score is calculated, focusing on much more recent data. One of the reasons a review bomb can distort a game's Review Score for a significant period of time is because the score is based on reviews over a period of 30 days for the Recent value, and all time for the Overall value. But doing this would likely result in more fluctuation and potentially less accuracy for all games, not just review bombed ones.

    Ultimately, Valve decided to not really change anything, as they explain here.
    In the end, we decided not to change the ways that players can review games, and instead focused on how potential purchasers can explore the review data. Starting today, each game page now contains a histogram of the positive to negative ratio of reviews over the entire lifetime of the game, and by clicking on any part of the histogram you're able to read a sample of the reviews from that time period. As a potential purchaser, it's easy to spot temporary distortions in the reviews, to investigate why that distortion occurred, and decide for yourself whether it's something you care about. This approach has the advantage of never preventing anyone from submitting a review, but does require slightly more effort on the part of potential purchasers.

    It also has the benefit of allowing you to see how a game's reviews have evolved over time, which is great for games that are operating as services. One subtlety that's not obvious at first is that most games slowly trend downwards over time, even if they haven't changed in any way. We think this makes sense when you realize that, generally speaking, earlier purchasers of a game are more likely to enjoy it than later purchasers. In the pool of players who are interested in a game, the ones who are more confident that they'll like the game will buy it first, so as time goes on the potential purchasers left are less and less certain that they'll like the game. So if you see a game's reviews trending up over time, it may be an even more powerful statement about the quality of work its developers are doing.

    It is the path of least resistance. And just in case anybody was curious, the graph at the top is for Grand Theft Auto V, the game that still outsold almost every other game in August except for Madden NFL 18.
Working...
X