Meta/Facebook defends error-prone chat control CSAM scanner algorithms in court [Updated 16.05.2024]
In May 2022, Member of the European Parliament, civil liberties activist and digital freedom fighter Patrick Breyer (German Pirate Party) filed a lawsuit against Facebook’s parent company Meta Platforms Ireland Limited and applied for an injunction against the automated searching of private messages for suspicious content at Kiel District Court. The corporation, however, is now defending its controversial and error-prone “incrimination machines”, arguing that they help to “more efficiently achieve detection, removal and referral of CSAM to law enforcement and relevant non-governmental organisations (…).”
As a user of the “Facebook Messenger” service, Breyer is suing against the suspicionless automated searches of his private chats. However, in its statement of defence, Meta now takes the position that “even indiscriminate data processing can be proportionate in the fight against serious crime”.
Reportedly Facebook took action against 340,000 EU accounts during a period of two months for sending or receiving alleged child pornography to or from EU users. 4,900 users complained about this (extrapolating to almost 30,000 users per year). In 207 cases, Facebook upheld the complaint (extrapolated to 1,200 users per year).
Plaintiff Patrick Breyer comments:
“It makes me furious that private US corporations want to self-police our personal messages. This presumption lacks any respect for the digital privacy of our correspondence. Flooding our already overburdened investigators with often false machine denunciations has nothing to do with efficiency. The result is above all the criminalisation of young people because of carelessly sent or received chat messages – thus the opposite of child protection is achieved.
With this incrimination machine in place, even the most intimate nude photos and sex chats can suddenly end up with dubious and underpaid company personnel or the police. Those who destroy the digital secrecy of correspondence destroy trust. We all depend on the security and confidentiality of private communication: People in need, victims of abuse, children, the economy and also state authorities.
Organised child porn rings do not use e-mails or messenger services, but rather self-operated forums. The fact that law enforcement officials refuse to have known child sexual expoitation material removed is a scandal. What we need is removal, not mass surveillance!”
The background to the lawsuit is the EU Commission’s plan to make indiscriminate messaging and chat control mandatory for all providers of email, messenger and chat services. With reference to legal opinions by a former ECJ judge and the Scientific Service of the Bundestag, Breyer wants the Kiel court to rule the scanning illegal. According to the Swiss Federal Police, 80% of the US NCMEC reports are irrelevant in terms of criminal law.
Update 16.05.2024: The District Court refused to rule on the legality of Meta’s voluntary chat control, which is why I am now appealing to the Higher Regional Court.
