In this Reddit interview, members of Chess.com’s Fair Play Team answer dozens of questions about detecting cheaters, closing over 2,500 accounts daily for cheating.
TLDR
Chess.com’s Fair Play team is at the forefront of ensuring a fair and enjoyable experience for millions of players worldwide. With over 2,500 accounts closed daily for cheating—amounting to more than 74,000 bans each month—the sheer scale of their anti-cheating efforts is staggering. This work involves analyzing over 10 million games per day, making it one of the most sophisticated operations of its kind.
The platform sets a high bar for accusations, requiring 99.99% certainty before closing any account. This meticulous and data-driven approach ensures that decisions are based on thorough evidence, protecting both the integrity of the game and the reputation of its players. Titled players, in particular, face extra layers of scrutiny, with cases evaluated using data from both online play and historical over-the-board (OTB) performance.
Public discourse about Chess.com’s anti-cheating methods can be a double-edged sword. While the team values constructive feedback, they face frequent challenges from conspiracies and reckless accusations on social media. Despite this, their commitment to transparency and fairness remains strong, as they strive to uphold the highest standards of integrity in online chess.
Cheating on the platform follows predictable patterns, with most offenders copying engine moves for entire games. However, sophisticated cheating methods, such as selectively using engines in critical moments, are more challenging to detect. The team is actively focused on addressing these cases through ongoing research and algorithmic development.
Looking ahead, Chess.com is preparing to launch “Proctor,” a lockdown browser designed to enhance security during tournaments with prize money. Initially set to be tested in Titled Tuesday events, the system promises to raise the bar for anti-cheating technology.
Even high-profile cases involving accusations from notable figures like Vladimir Kramnik are handled with care. While the team avoids engaging in public disputes, they rely on robust statistical analysis to back their decisions, ensuring fairness for all players involved.
The team members are:
FM Dan Rozovsky: Director of Fair Play – Oversees the Fair Play team, helping coordinate new research, algorithmic developments, case reviews, and play experience on site.
IM Kassa Korley: Director of Professional Relations – Addresses matters of public interest to the chess community, fields titled player questions and concerns, supports adjudication process for titled player cases.
Sean Arn: Director of Fair Play Operations – Runs all fair play logistics for our events, enforcing fair play protocols and verifying compliance in our prize events. Leading effort to develop proctoring tech for our largest prize events.
Full Interview
How do you deal with smart cheaters? Who are good enough to play openings and most moves but resort to engine use at critical moves?
Dan Rozovsky: This is the most difficult question in fair play. High-level players that are “spot cheating” are often challenging to catch. It’s also the reason why we’re so hesitant to share some of our detection methods.
Difficulty aside, it’s the area we focus the most on in research, especially skewed towards the higher-rated players on-site. We leverage the billions of games played monthly to pick up on stats and metrics that differ significantly in “critical” positions. It often takes more games to catch these types of cheaters, but catch them we do!
How would you know if someone found an engine move by accident or with an engine?
Dan Rozovsky: No single move is enough to close an account for cheating; we all find great moves from time to time. We even play brilliant games on occasion! Our methods are built on significant data collection and analysis for each case, no matter if it’s handled algorithmically or with human experts reviewing.
Does Chess.com assign a user account a “trust” rating? Like eBay customers rate each other? That way, honest players are more likely to get paired preferentially with each other and suspect accounts more likely to get paired with each other?
Dan Rozovsky: We have something very similar to this internally that is considered in pairings and, of course, in our fair play process. Sharing something like a “trust rating” externally (on player profiles, for example) may cause some nasty comments and accusations that aren’t substantiated.
Can Chess.com release an anonymized dataset for community sourcing methods for cheat detection (e.g., Kaggle competitions), with data like profile rating, matches played, match characteristics, banned or not, etc.?
Dan Rozovsky: Good question! It’s a tough balance to strike because we don’t want to give away any “features” or statistical tests we run in a dataset like that. Having said that, we want the community to run analyses on good data — look out for something like this perhaps in 9-12 months from now.
What’s the justification for publicly closing the accounts of titled players for violating the fair play policy, but not specifying, either publicly or privately to the player, what the reason for the ban is?
Kassa Korley: We believe public closures serve as an effective deterrent and provide much-needed transparency to our community. We’ve even seen a reduction in cheating rates (specifically for titled players).
We typically do not share details regarding an account closure because we believe we’re at risk of “giving away the game/methodology” to cheaters. We hope the community can understand why citing the games & reasons for closure can make it far more difficult to catch cheaters in the future.
We’ve looked closely at how major sporting organizations like the IOC, NFL, ITIA, and others approach similar issues with PEDs. They announce the suspensions and violations, but they don’t disclose their methods and evidence for the same reason that we don’t — it makes evasion easier.
That said, I have conversations with players all the time, and we’re always looking for meaningful ways to improve the appeals process.
Is it okay for a player to use Stockfish if they suspect their opponent is using Stockfish?
Dan Rozovsky: No! Report the player on-site instead.
Why do cheaters get a second chance on Chess.com?
Sean Arn: People make mistakes. As a company, we believe in second chances. But not third chances!
What percentage of players do YOU believe cheat during Titled Tuesdays?
Dan Rozovsky: It varies from event to event, but in most events, we see it hovering around 1%. We have many projects in the works, none more pertinent than enhancing our proctoring tech, which we hope will send this number plummeting towards 0.
How confident are you that you could catch subtle cheating? For example, a player looking at the eval bar occasionally or inputting a computer move once in a critical position and nowhere else?
Dan Rozovsky: Some of our approach is based on hypothesis testing, comparing potential cheaters to a “benchmark” made up of many players in a particular rating range. For someone like Magnus or Hikaru, we use a much stronger benchmark than we use for the typical player.
It is generally true that the stronger the player, the harder it is to detect when they’re cheating. At the same time, Magnus and Hikaru have well-established playing styles that we’re able to quantify, so we’d still be able to detect when they deviate significantly from their normal levels.
Does it ever happen that people get wrongfully banned for cheating and then reinstated after the appeal? What actually proves their innocence?
Sean Arn: Absolutely. Every month, we review thousands of appeals, and we reinstate less than half of one percent of those appeals. Every case is different, but one of the most common reasons for reinstatement is an established OTB player on an underrated account playing against opposition far below their level.
How confident are you that you could catch subtle cheating? For example, a player looking at the eval bar occasionally or inputting a computer move once in a critical position and nowhere else?
Kassa Korley: Cheating like this is very subtle, and detection requires a holistic review. We analyze not only the moves but also the time spent, the frequency of exceptional moves, and overall statistical anomalies in the games. For titled players, we also consider their historical performance as part of the process.
On what basis was “Viih_Sou” (Brandon Jacobson) banned? He said he was 100% clean and didn’t cheat yet you guys banned him. Just wondering.
Kassa Korley: As with any titled player case, the Viih_Sou account closure was the product of a thorough statistical and expert review of his games. After conducting this review, we determined there was conclusive evidence of cheating. I met with Brandon regarding the case/how we got there, and as with any case, he had the opportunity to appeal.
There have been many instances during my tenure here where players insist on their innocence, and we arrive at an impasse. We are open to new information and give every player the opportunity to appeal, but getting to the point of making a closure is the product of really thorough and diligent work from our team. We have a high threshold for closure and are extremely confident when we move to close an account.
Does it ever happen that people get wrongfully banned for cheating and then reinstated after the appeal? What actually proves their innocence?
Sean Arn: Absolutely. Every month, we review thousands of appeals, and we reinstate less than half of one percent of those appeals. Every case is different, but one of the most common reasons for reinstatement is an established OTB player on an underrated account playing against opposition far below their level.
What is the arbitration process like for someone appealing a fair play ban?
Sean Arn: When an account is closed for a fair play violation, they typically receive a closure email that provides them two options: either create a second-chance account or file an appeal. Users who choose to appeal then fill out a form where they are asked to provide details relevant to their case (e.g., OTB rating, Title, and any other relevant comments). The team then reviews these appeals and makes determinations.
For serious appeals (there are very few of those, most of them are either abusive or just outright lies), we convene as a team to review the player’s stats, games, and account details in light of the newly submitted information, and then we come to a determination.
Do you find the public discourse about your work, particularly on social media, annoying, counterproductive, etc.?
Dan Rozovsky: It’s awesome that people talk about fair play — it’s an interesting topic and work that we love doing! What’s frustrating are bad faith and reckless accusations — either when they are made about players in the community or about our team.
Our team of 30+ members takes great pride in their work, and it pains me when people say that we’re not taking this work seriously. Especially when people spread conspiracies or other misinformation. Our systems review over 10 million games per day, approximately 1 million cheat reports, and we close ~2,500 of these accounts daily. That’s nearly one player every 30 seconds!
We love our community and want to keep the game we all hold dear to the highest fairness standards that exist online. Help us do that with constructive feedback and by calling out misinformation!
Why does Chess.com pair new accounts with fairly older accounts? Most of the new accounts that I’ve played against have gotten banned. Given the size of the userbase Chess.com has, isn’t it possible to do pairings of new accounts with newer accounts (if there are enough of them)?
Dan Rozovsky: Good question! The general answer is that when you’re brand new to Chess.com (or any game where ratings are calculated for the players), you need to play against players that have reliable ratings to determine your own ability. New accounts do play other new accounts at times, but more information is gained for the new player when they play against someone with an established rating.
Are you investigating only in-game content (like the moves quality, or time spent) or also some other factors, like the age of the account, the FIDE rating linked to the profile (for titled players), the downloaded extensions, the mouse movements…?
Kassa Korley: The more information/context that is at our disposal, the better. Titled players, for instance, typically have more games/data/experience, so we may also consider their OTB (over-the-board) profile/rating history when making a decision.
You mentioned a few factors related to in-game content, but there are over a hundred other in-game factors that are at our disposal when we review someone’s play, so there is much to comb through!
What do you think is the most common method of cheating (i.e., using Stockfish for every move, using it just in critical positions, using it just in the endgame, accessing an eval bar w/out Stockfish, etc.)?
Sean Arn: Most cheating is not clever. Most cheating is just copy/pasting in engine moves for the entire game.
How many players are banned a day? Could you give an estimation of what % of your userbase is/will be a cheater and are you confident you’ll catch most of them?
Sean Arn: We closed more than 74,000 accounts for cheating last month (numbers haven’t been published yet, so enjoy the early info!), which puts us at ~2,500 per day. In a given month, we close less than half a percent of our active users, and (since most of them cheat egregiously), we’re very confident in our ability to catch them.
Almost all of the public discourse around cheating is related to prize money events and top players. What percentage of the cheating on Chess.com is happening in games with titled players vs. non-titled players?
Sean Arn: The vast majority of cheating occurs in lower rating brackets. Intuitively, it is easier to catch that cheating. We devote significant resources to this level (and all levels) of cheating.
**Does your system predict the likelihood that a player is cheating, and if so, what % certainty does the system need to project before an account is banned? Are these standards different for normal vs. titled accounts?
Also, is there any plan to more aggressively deal with bad faith accusers like Kramnik (releasing stats supporting those accused by him, for example)?**
Kassa Korley: We are able to determine the statistical likelihood of a performance, and in most instances, it can help guide/inform a decision. Anomalous performances happen all the time, and in most cases, are not closure-worthy.
Our standards for determining someone is conclusively cheating (AKA closure-worthy) are greater than 99.99%. It is a very high bar and applied across our entire community.
There have been a number of bad faith “investigations” calling players’ integrity into question, and while we have sometimes thwarted some of this discourse with our own research/findings, I believe most of these don’t merit a response because that gives it more oxygen. Public accusations and targeted attacks of players do violate our community guidelines, and while we may not always administer penalties publicly, we always escalate infractions with players accordingly.
I assume this is some form of hypothesis testing being carried out to determine if a player is cheating. I’m curious about someone like Magnus or Hikaru. Do they frequently trigger the fair play alarms?
Dan Rozovsky: You’re not completely off the mark! Some of our approach is based on hypothesis testing, comparing potential cheaters to a “benchmark” made up of many players in a particular rating range. For someone like Magnus or Hikaru, we use a much stronger benchmark than we use for the typical player.
It is generally true that the stronger the player, the harder it is to detect when they’re cheating. At the same time, Magnus and Hikaru have well-established playing styles that we’re able to quantify, and so we’d still be able to detect when they deviate significantly from their normal levels.
Will the lockdown browser (or app) client that Erik said (on the Perpetual Chess pod) was in development a LONG time ago ever see the light of day? Would it become an option to use for everyone all the time or only in tournaments? If the former, would it be possible to only match with other users of the client?
Sean Arn: This project (called Proctor) is still very much alive, and we will have news here soon.
We will initially begin testing with a limited pool of players in Titled Tuesday, and once we are confident that it works well, we will gradually roll Proctor out to all players competing in prize events.
There is no intention to use it outside of prized play at this time.
Do you pay attention to the advice that Kramnik gives?
Dan Rozovsky, Kassa Korley and Sean Arn: If we took all of Kramnik’s advice, we’d have no members left.