Elon Musk’s xAI faces a federal lawsuit alleging the company used child sexual abuse material (CSAM) to train its Grok models, marking the first time a tech startup has been accused of this specific practice. The complaint, filed on Wednesday on behalf of a plaintiff identified only as Jane Doe, alleges that xAI utilized a dataset containing Doe’s childhood abuse images to build the company’s image and video generation capabilities. Doe’s legal team claims the material was part of a CSAM Hash List maintained by the National Center for Missing and Exploited Children (NCMEC), which organizations use to identify and remove illicit content online. This is the first lawsuit to directly accuse xAI of training on CSAM, although previous reports have documented researchers finding the prohibited material in other AI training sets.
Doe’s images have been hashed and flagged by safety groups since the early 2000s to prevent their circulation. Despite this, the Canadian Centre for Child Protection (CCCP) recently notified Doe that AI-generated CSAM depicting her was found on xAI. The complaint alleges that this discovery led to messages between offenders discussing the creation of new AI-generated CSAM based on Doe’s likeness. Doe’s legal team suggests that because xAI treats public posts and its own outputs as training data by default, posting an image exposes it not just to viewers but also feeds it directly into the model’s training pipeline. This process allegedly allows Grok to generate new violative content based on inputs that have haunted Doe for more than two decades.
Related: Court blocks FCC rule on election ads
xAI’s terms of service for Grok do not explicitly list CSAM or non-consensual intimate imagery as excluded categories for training data. The complaint argues that while the company filters violent content from outputs, it has not publicly claimed to have removed the influence of CSAM that was already ingested during training. Because fully removing the influence of a training example from an already-trained model is technically difficult, the lawsuit claims the model’s outputs were likely shaped by the prohibited images even after they were removed from public view. The complaint notes that Grok’s training method means any sexualized outputs, including images of survivors, could potentially be used to create further abuse material.
xAI’s approach to training data creates a cycle where the company’s own products may contribute to the abuse material it seeks to avoid. The lawsuit alleges that because the model treats its own outputs as training data, any CSAM it generates can be fed back into the system to create more of the same. This dynamic effectively uses the generated abuse material to train future iterations of the model, amplifying the harm to victims whose original images were used as a starting point. Doe’s legal team argues this ongoing process exacerbates the trauma of survivors whose images are already circulating among predators online.
Related: Smart travel gadgets to pack for your next trip
The proposed class action lawsuit seeks to represent every victim whose childhood images have been used to generate Grok CSAM. Doe’s legal team is asking the court to order xAI to destroy all CSAM stored on its servers and to block the model from generating such content in the future. The complaint suggests this would require blocking Grok from producing any sexualized outputs, including non-consensual intimate imagery and other NSFW content. Doe’s lawyers also allege that xAI violated federal child pornography laws and Masha’s Law, which provides survivors with the right to sue for the production, possession, and distribution of CSAM. Margaret E. Mabie, one of Doe’s attorneys, stated in a press release that xAI “did all three” of these crimes. xAI has not responded to Ars Technica’s request for comment.