Valve's Fix for Combating Review Bombing is a Graph

Collapse
X
Collapse
  •  

  • Valve's Fix for Combating Review Bombing is a Graph

    [ATTACH=CONFIG]18313[/ATTACH]
    Review bombing is when a bunch of Steam users get together in an effort to drastically raise or drastically lower the user scores for a game. It's usually the case that a developer or publisher has done something that people don't agree with so they "get back" at them by submitting a bunch of negative reviews. Those mass spammed negative reviews are then upvoted in a mass circle jerk while any positive reviews are mass downvoted. This is all done in an effort to... do... something? I guess their end goal is just to lower a game's overall user score enough that it may dissuade people from buying it in the future. I mean, it worked out well for Grand Theft Auto V a few months ago. It worked out so well that the game still maintained the number two sales position on the NPD charts for August 2017 and was in the top 10 for the NPD every month for the past four or so years now.

    Whatever the motivations, review bombing is pretty damn stupid. Steam has a particularly nasty problem with it, so Valve decided to take some drastic action to combat it.

    They added graphs. Before arriving at their solution of adding graphs, Valve did examine other possibilities on what to do.
    So what solutions did we explore? An obvious one would be to simply remove the Review Score. Then, as a potential customer you'd be forced to read the User Reviews themselves to see if the product sounded interesting. If you saw reviews talking about something outside the scope of the game, you could decide for yourself if it was an issue that would affect your happiness with the purchase. Unfortunately, we're pretty certain that this isn't really an option - scores were added in response to player demand in the past, and that demand for a summary of some kind is likely to still be there, even if players know it isn't always accurate.

    Another idea was a temporary lock on reviewing, similar to how stock markets prevent trading on specific stocks when abnormal behavior is detected. Based on the theory that review bombs are temporary distortions, we could prevent reviews for short periods of time whenever we detect massive distortions in submissions. In the cases where the cause of the community's dissatisfaction truly affects the game's potential happiness to new customers, we're confident it would still result in the Review Score moving down after the lock period ended.

    But if we lock reviews on a product for a short period of time, what does that mean exactly? Are players no longer able to post reviews at all during that time? Or should they be able to post them, but we ignore them for the purpose of calculating the Review Score? In the end, we didn't like the way this ultimately meant restricting the ability for players to voice their opinions. We don't want to stop the community having a discussion about the issue they're unhappy about, even though there are probably better places to have that conversation than in Steam User Reviews.

    We could change the way the Review Score is calculated, focusing on much more recent data. One of the reasons a review bomb can distort a game's Review Score for a significant period of time is because the score is based on reviews over a period of 30 days for the Recent value, and all time for the Overall value. But doing this would likely result in more fluctuation and potentially less accuracy for all games, not just review bombed ones.

    Ultimately, Valve decided to not really change anything, as they explain here.
    In the end, we decided not to change the ways that players can review games, and instead focused on how potential purchasers can explore the review data. Starting today, each game page now contains a histogram of the positive to negative ratio of reviews over the entire lifetime of the game, and by clicking on any part of the histogram you're able to read a sample of the reviews from that time period. As a potential purchaser, it's easy to spot temporary distortions in the reviews, to investigate why that distortion occurred, and decide for yourself whether it's something you care about. This approach has the advantage of never preventing anyone from submitting a review, but does require slightly more effort on the part of potential purchasers.

    It also has the benefit of allowing you to see how a game's reviews have evolved over time, which is great for games that are operating as services. One subtlety that's not obvious at first is that most games slowly trend downwards over time, even if they haven't changed in any way. We think this makes sense when you realize that, generally speaking, earlier purchasers of a game are more likely to enjoy it than later purchasers. In the pool of players who are interested in a game, the ones who are more confident that they'll like the game will buy it first, so as time goes on the potential purchasers left are less and less certain that they'll like the game. So if you see a game's reviews trending up over time, it may be an even more powerful statement about the quality of work its developers are doing.

    It is the path of least resistance. And just in case anybody was curious, the graph at the top is for Grand Theft Auto V, the game that still outsold almost every other game in August except for Madden NFL 18.
      Posting comments is disabled for guests. Login or sign up to leave a comment.

    Related Topics

    Collapse

    • Shawn Zipay
      DOOM: The Dark Ages - Launch Trailer and Reviews
      by Shawn Zipay
      It's almost time to go medieval on some demon ass.

      DOOM: The Dark Ages will arrive next week, which means it's time for the official launch trailer to drop. DOOM: The Dark Ages is out on Tuesday for those that purchase the Premium Edition can play on Tuesday, May 13. Those that get the regular edition can play on Mary 15th. Thanks to a single global release time, those in the Americas will get to play starting on May 12th at 8PM for Premium and on May 14th at 8PM for regular....
      05-09-2025, 06:40 PM
    • Shawn Zipay
      Starfield Review Averages Sit at Upper 80s
      by Shawn Zipay
      See what critics are saying about the most anticipated game of 2023.

      The first major wave of reviews are in for Starfield and, well, they aren't bad. They're not super great either but they certainly aren't bad.

      According to OpenCritic, Starfield currently sits at an average review score of 87% with 93% of all reviewers recommending the game. The scores for this one seem to range from as low as 50% to as high as 100% (or 10 out of 10 to be more specific).

      ...
      08-31-2023, 01:04 PM
    • Shawn Zipay
      Critical Consensus: Alone in the Dark
      by Shawn Zipay
      Fails to live up to others in the genre this franchise helped to create.

      The Alone in the Dark reboot was just released today and it seems like the reviews have placed it firmly in the realm of being merely ok but with some serious shortcomings. This release from Pieces Interactive and THQ Nordic currently sits at a 62% critical average on OpenCritic, with only 42% of reviewers recommending this new release.

      Interestingly, the actual scores for this game range everywhere...
      03-20-2024, 02:01 PM
    • Shawn Zipay
      Critical Consensus: Dragon's Dogma 2
      by Shawn Zipay
      An improvement over the original but with some severe performance woes.

      Dragon's Dogma 2 is expected to be released on March 22 but the reviews have already started to pour in. The reviews have been extremely favorable for Dragon's Dogma 2 so far, with most outlets saying that this sequel improves upon the original. However, some of the flaws of the first game are not addressed in this sequel and the performance across all platforms, especially on PC, leaves a lot to be desired....
      03-20-2024, 02:37 PM
    • Loading...
    • No more items.
    TGN on Threads TGN on BlueSky TGN on Facebook
    Working...