The people who police chess cheats: ‘We built a crime scene analysis for every player in the world’

Did a teenager cheat to defeat the chess world champion? This question has thrown the chess universe into turmoil since 4 September, when its top player, 31-year-old Magnus Carlsen, abruptly withdrew from the $350,000 Sinquefield Cup in St Louis after a stunning loss to the lower-ranked 19-year-old Hans Niemann.

Carlsen has not explicitly accused Niemann of cheating. But chess watchers gathered Carlsen’s accusation from a cryptic meme he tweeted after the game saying he would be in “big trouble” if he spoke – fueling wild theories, including one that Niemann cheated by receiving messages through vibrating anal beads.

The uproar continued on Monday, when Carlsen faced Niemann in an online game and resigned after just one move. On Wednesday, Carlsen gave a short interview in which he declined to explain his actions, but said “people can draw their own conclusions and they certainly have”. He intoned that he was “impressed by Niemann’s play and I think his mentor Maxim Dlugy must be doing a great job” – another apparent accusation, as Dlugy is a chess master who has been accused of cheating himself.

Niemann denied cheating against Carlsen, commenting after the earlier game that the world champion must be “embarrassed to lose to an idiot like me”. But he admitted to cheating twice on the online platform Chess.com at age 12 and again at 16, which he said got him kicked from the website. The controversy deepened when the platform announced that it had banned Niemann again, citing “information that contradicts his statements regarding the amount and seriousness of his cheating on Chess.com”.

But that move contradicts other top chess arbiters, including the Sinquefield Cup’s organizers, who say they’ve analyzed Niemann’s games and found no evidence of wrongdoing. So if neither the tournament nor Magnus is explicitly accusing Niemann of cheating, why do many in the chess world think Niemann is a cheat?

Danny Rensch, a chess master and Chess.com executive, told the Guardian that chess watchers – from authorities to armchair theorists – aren’t analyzing Niemann’s performance correctly. “It’s not anal beads. The problem is our position is so different in terms of how we look at it and measure things.”

Rensch said his platform has developed an industry-leading anti-cheating model trained on a staggering trove of real-world game data from games played on its platform. “What we did that really is different than any others do – and it’s because we were a private company that was making money and were able to invest – is we went out and built what I would call DNA crime scene analysis for every chess player in the world,” Rensch said. That means Chess.com has a highly detailed model of what legitimate behavior looks like for millions of users over hundreds of millions of games, which it can use to detect discrepancies.

“Once in a while anomalies do happen. But if you have a lot of smoke, a lot of evidence, and a lot of reason to believe in the DNA of who someone is, and you walk into the room and they just say, ‘I just lifted that fridge with one arm’, you’re like, ‘Fucking bullshit, motherfucker.’”

Rensch declined to elaborate on Niemann. “I’m not going on the record on anything that I think about the over-the-board scandal with Hans or Magnus, but you can imply what you want based on what I’m saying,” Rensch said. In forum posts this week, Chess.com CEO Erik Allebest has hinted his company might soon release more information.

That could help answer one of the central questions in this controversy: what’s the best way to detect cheating in chess?

It’s important to understand how computers affect the game. The best human chess players are a mix of artist, athlete, and scientist: they not only have the creativity and mental endurance to solve highly intricate problems, they also spend thousands of hours researching previous chess games and theorizing new lines of play. The problem is that modern chess software, called chess engines, have become so powerful and widely available that even the world’s best players don’t stand a chance against software that anybody can now download for free. For the chess industry, which is enjoying a pandemic-driven explosion of interest in everything from amateur online games to live streams of top masters ,detecting cheating has become an existential challenge.

Tanya Karali is the chief arbiter, or chess referee, of the Meltwater Champions Chess Tour, the online tournament that saw Carlsen’s dramatic resignation this week. The main way that cup safeguards against cheating is through surveillance, she said. That includes requiring multiple players to set up multiple cameras that prove that they’re alone without other electronics. “At random moments, we surprise players asking them to move around with the side camera to show the whole room,” she said. The arbiters also ask the players to share their screens so they can see what programs they’re using, and point the side camera at their ears to inspect for bugs.

But the most important authentication tool Karali uses is a screening program employed by Fide, the international chess governing body. Ken Regan, a chess master and computer scientist, said he started developing the model in 2006 after a high profile cheating accusation by Bulgaria’s Veselin Topalov against Russia’s Vladimir Kramnik in their world championship game. Regan’s model analyzes the possible moves in a chess position and projects the probability that a player of a given skill level would make a move that agrees with top chess engines. “Then, through what’s really a human judgment process, one arrives at the final odds and decides whether they are extreme enough to reject the null hypothesis,” – that is, the assumption of fair play.

Because the software analyzes the moves of the game itself, it works on over-the-board games as well as online, where the cheating rate is “100 to 200 times” higher, Regan said. Sinquefield Cup officials asked Regan to run the program on Carlsen and Niemann’s game and the results were unambiguous: “I found nothing,” he said. Regan’s model showed Neimann’s performance “was one standard deviation up” on some metrics, “but by definition the standard deviation standardly happens”.

But that’s led to an apparent disagreement between believers in Regan’s model and those of Chess.com’s model, which it doesn’t seem can be resolved without more evidence being made public. “It’s Chess.com’s move,” Regan said. The platform, he suggested, needs to “divulge or explain the reasons for their further action against Niemann”.

This is just the latest installment in a decades-long drama over the role of machines in one of the world’s oldest board games. Matthew Sadler, an English grandmaster who was ranked 14th in the world in the “pre-computer” age, left professional play in 1999 when he feared that the rise of AI would “kill the game”. He’s now a researcher who has authored multiple books about chess engines. While he can occasionally outplay computers over a few moves, he says, there’s no way to match the consistency of top engines. “In a game of 60 moves, the accuracy that engines have is just on a level that’s completely impossible for humans to attain.”

Computers have an ability to perceive the totality of the game in a way that dramatically outperforms humans, Sadler said. “Engines are just incredibly good at visualizing the whole board and finding maneuvers that, for example, use three corners of the board in order to redeploy a piece and achieve a winning angle of attack. When you see people at a weaker level doing that, well, they’ve either had a moment of inspiration or there could be something a bit funny going on.”

Contrary to Sadler’s fears, technology hasn’t killed the game – it’s made it even more popular. Chess engines have become invaluable learning tools for players: they pore over game databases and run scenarios through the engines, trying to memorize the most important variations. Because even the best brains can’t memorize everything, the game has evolved into one of trying to throw your opponents off-balance with unexpected play. And for spectators, the engines provide a dramatic way to see who’s winning games in real-time.

Could it be possible for a human player to detect computer-aided play without sophisticated technological tools? Sadler says that being able to sniff out cheating comes with experience. “If an opponent has got a very complicated decision and is just taking a minute over it, whereas you’d expect, well, any normal top player would take 15 or 20, then that’s a little bit off.” Other red flags: if your opponent seems “unnaturally calm when the position is very tense”, or “if someone is going for suspiciously long walks away from the board”. But these tells aren’t foolproof: “I once had a case like that, and it was just that the poor guy was having prolonged nose bleeds, having to run to the toilet all the time.”

As for Carlsen’s accusation? Sadler says his experience leaves him in disbelief. While Carlsen is still clearly the world’s best player, “my position still is that cheating at the top level just doesn’t really happen”, he said. “There’s an awful lot to lose. And chess is one of those games where you dedicate our life to it and it’s just a little bit hard to imagine the top players would throw that all away.”