Wikipedia is making its data more accessible to AI developers by releasing a dataset optimized for machine learning, in an effort to reduce scraping and strain on its servers caused by automated AI bots.
The Wikimedia Foundation has partnered with Kaggle, a Google-owned data science community platform, to publish a beta dataset of structured Wikipedia content in English and French. This dataset is “designed with machine learning workflows in mind,” making it easier for developers to access machine-readable article data for various AI applications, including modeling, fine-tuning, benchmarking, alignment, and analysis.
The dataset includes a variety of content such as research summaries, short descriptions, image links, infobox data, and article sections. However, it excludes references and non-written elements like audio files. As of April 15th, the data is presented in “well-structured JSON representations,” which should be more appealing to developers than scraping or parsing raw article text. This move is expected to alleviate the strain on Wikipedia’s servers, which are currently being heavily consumed by automated AI bot activity.
The Wikimedia Foundation already has content-sharing agreements in place with Google and the Internet Archive. However, this partnership with Kaggle is aimed at making the data more accessible to smaller companies and independent data scientists. By hosting the dataset, Kaggle is playing a crucial role in keeping the data accessible, available, and useful for the machine learning community.
“As the place the machine learning community comes for tools and tests, Kaggle is extremely excited to be the host for the Wikimedia Foundation’s data,” said Brenda Flynn, Kaggle partnerships lead. “Kaggle is excited to play a role in keeping this data accessible, available, and useful.”
The release of the dataset was announced on April 17, 2025, marking a significant step in Wikipedia’s effort to engage with AI developers and manage the impact of AI-driven traffic on its platform.
