Reducing Toxicity

Task

Reduce the level of toxicity on the platform, decrease the number of support requests related to negative comments, and lower the share and volume of negative statements under posts and photos.

Research

According to research, negative posts provoke anger in more than 40% of respondents. Users express a need for safe and respectful communication. One third of all complaints on the platform are related to negative comments, with 80% of support requests concerning comments aimed at combating toxicity.

Process

The platform was already using an ML model to hide offensive words and 18+ content. A decision was made to leverage the same model to gradually reduce negativity and help users develop a habit of respectful communication.

The task was split into two sub-tasks:

1. Block a comment at the point of submission.
2. Temporarily restrict the ability to send messages and comments for repeated violations.

Comment Blocking

At the point of submission, each comment is checked for toxicity, offensive language, and other unwanted content. A comment flagged as toxic is not published. After the failed attempt, the sender receives a notification explaining that the comment cannot be posted as it is considered toxic. The notification is visible only to the sender and disappears upon page reload.

Temporary Restriction

Users can receive an automatic restriction for repeated violations of chat rules or as a result of reports from other users. The restriction may apply to sending messages, comments, or both. A user who has been restricted can view detailed information in a dedicated slide-up panel, which includes reminders about the site's rules and guidelines for respectful communication in comments and messages.

Results