Authors
Ruby leads editorial, operations and initiatives at Newschecker. In her former avatar at New Delhi Television (NDTV), India’s leading national news network, she was a news anchor, supervising producer and senior output editor. Her over a decade-long career encompasses ground-breaking reportage from conflict zones and reporting on terror incidents, election campaigns, and gender issues. Ruby is an Emmy-nominated producer and has handled both local and international assignments, including the coverage of Arab Spring in 2011, the US Presidential elections in 2016, and ground reportage on the Kashmir issue since 2009.
UPDATE: Rest of World retracted its article “Fake news verification tools fail the test during elections in India” on May 29, 2024
An article published by Rest of World (RoW) titled “Fake news verification tools fail the test during elections in India” on May 24, 2024, raises concerns about the effectiveness of the WhatsApp tiplines run by Newschecker and other Indian fact-checking organisations.
Unfortunately, the article contains several inaccuracies and misinterpretations of our work. Despite multiple requests sent to the editors to set the record straight, the team at Rest of World remains unresponsive, claiming the responsible individuals have been “in transit” for over four days straight. As Managing Editor of Newschecker, I aim to clarify these discrepancies and provide accurate information.
Incorrect Timeline and Context
The article states, “In February, a consortium of fact-checking and media organizations launched a tipline on WhatsApp with seed funding from Meta.” The consortium referred to here is the Misinformation Collaboration Alliance (MCA), which Newschecker is a founding member of. The MCA has a fairly new initiative called the Deepfakes Analysis Unit (DAU). While it is true that the DAU’s tipline was started before the ongoing elections, Newschecker’s tipline has been operational since 2020. The article inaccurately combines the timelines and operations of different tiplines, misleading readers.
It further states: “The following month, San Francisco-based tech nonprofit Meedan made its chatbot software available to several partners who are monitoring viral misinformation shared on closed messaging apps ahead of the 2024 Lok Sabha parliamentary elections in India.” However, Newschecker has been using Meedan’s Check platform since November 2022. Between 2020 and 2022, Newschecker used the services of a different service provider to run the tipline.
Query and Response Discrepancies
The authors claim that 15 queries were sent to Newschecker’s tipline, out of which only three responses were received. Our records show that 12 queries were sent by three individuals from Rest of World, and we responded to 10 of these. One query was unclear, and we requested more context twice without receiving a response. Only one query went unanswered.
Six more queries from individuals linked to Rest Of World were not recognised as legitimate queries on the tipline because the users exhibited spam-like behaviour and failed to adhere to the step-by-step procedure outlined in the tipline. This brings the total number of queries sent by RoW to Newschecker to 18, which differs from the number reported by RoW. This indicates that they themselves are uncertain about the exact number of queries they sent.
The article further claims, “The tip lines did not provide a follow-up response 85% of the time.” In Newschecker’s case, the response rate is over 90%, contrary to the authors’ claim of 20% (3 out of 15). This is a gross misrepresentation. Despite our efforts to correct this with RoW, they have not made the necessary amendments.
Misunderstanding of Helpline Operations
The article claims, “The WhatsApp tip lines often require human intervention as the chatbots frequently struggle with verifying the obvious.” This highlights the authors’ misconceptions about the functionality of the tiplines. This is intentional and not due to tiplines’ limitations. The tiplines are primarily structured for human responses and are not reliant on bots. The “chatbot” handles initial filtering and procedural guidance. This design ensures accurate and reliable fact-checking, contrary to the article’s implications. Unlike simple verification tools, the WhatsApp tipline is designed to provide an easy way for users to submit potential misinformation for review. The goal is to crowdsource and fact-check viral or public-interest content. While some automation is used, it was never intended for users to receive automated and immediate replies for all queries, a point emphasised by the experts in the article but ignored by the RoW team.
How Do These Tiplines Work?
Newschecker’s tipline has a well-defined conversation flow. The chatbot’s role is limited to setting language preferences and guiding users on how to submit a claim.
Other chatbot features allow users to subscribe to a daily newsletter, access the latest fact checks in their preferred language, and more.
When a query is submitted, the chatbot first searches our database for relevant fact checks. If none are found, the query is escalated to a human for review. The assertion that human intervention is required due to technological shortcomings is inaccurate. The system is designed to filter out spam and ensure only serious queries reach our team, protecting our resources from being overwhelmed.
Bear in mind, that fact-checking organisations handle extremely high volumes of misinformation while facing multiple constraints to combat the overwhelming tide of misinformation, especially during elections. In such circumstances, this type of automation becomes a vital tool to be able to filter through non-serious queries and spam and only focus on responding to genuine users and their genuine queries.
Alleged Inconsistencies in Responses
RoW raises concerns about inconsistencies in responses from the same tipline, attributing them to chatbot errors. To illustrate this statement, the Rest of World team included three screenshots from Newschecker’s tip line. Unfortunately, they have not been transparent about this and show different stages of a conversation as “inconsistencies”.
The tipline is designed to sift through the database for available fact-checks and information immediately upon receiving a query, subsequently forwarding it to a human for evaluation. This process mirrors what occurred in the example cited by RoW to highlight “inconsistencies.”
The varied responses they refer to as “inconsistencies” (in the other two images of Newschecker’s tip line) actually reflect human fact-checkers addressing queries with different wording, but with consistent and accurate conclusions. The claim did not require a written fact check, however, when it was sent twice, two different reviewers responded, and neither response was incorrect.
During the critical election period, we have successfully responded to thousands of user inquiries via the tipline in March and April alone. We strongly disagree with the portrayal of Newschecker in the article and find it both incorrect and uninformed. The premise of the article, that the tiplines solely rely on chatbots for real-time verification, is far from accurate. At Newschecker, we remain committed to transparency and accuracy in our fact-checking efforts. We hope that Rest of World will address these discrepancies to uphold journalistic integrity.
Authors
Ruby leads editorial, operations and initiatives at Newschecker. In her former avatar at New Delhi Television (NDTV), India’s leading national news network, she was a news anchor, supervising producer and senior output editor. Her over a decade-long career encompasses ground-breaking reportage from conflict zones and reporting on terror incidents, election campaigns, and gender issues. Ruby is an Emmy-nominated producer and has handled both local and international assignments, including the coverage of Arab Spring in 2011, the US Presidential elections in 2016, and ground reportage on the Kashmir issue since 2009.
