Blizzard Trying to Fight Toxicity with Machine Learning

Blizzard was one of the thirty-plus companies to sign onto the Fair Play Alliance, a group dedicated to combating toxicity in video games by combining technology, tactics, and experience. It’s not surprising to see them there; despite a relatively sunny outlook on how well-behaved the Overwatch community would be, they soon came to their senses and started more aggressively policing their players.

It hasn’t been working great so far, since Overwatch has so many players and one too many Bastions, and now it seems like Blizzard and Overwatch Director Jeff Kaplan has decided to kick things into overdrive and employ AI to solve the problem.

Or at least, to try to, according to an interview with Kotaku:

"We’ve been experimenting with machine learning. We’ve been trying to teach our games what toxic language is, which is kinda fun. The thinking there is you don’t have to wait for a report to determine that something’s toxic. Our goal is to get it so you don’t have to wait for a report to happen."

Nothing has been implemented yet, but the idea, apparently, is to use these machine learning programs to study language (exclusively language right now, as it’s far easier to track) and quickly determine when someone crosses the line from good-natured fun to evil jerkwad. Of course, this would mean that Blizzard would have to monitor every chat in every game, which is no easy feat. That said, if it would get jerks out of the game quicker, then it sounds like a good deal to me.

As for training the algorithm to detect other common jerk behaviors, like abusing Mei walls at spawn, that’ll take some more time. A natural next step. But hopefully if this works Blizzard will share it with the world. That’s what an alliance is for, after all!