Reddit made a big change this week to protect its content from being taken without permission. They updated something called the robots.txt file, which tells computer programs (like those used by search engines) whether they can look at a website. This file has been important for letting search engines show you websites when you search for something.
But now, with the rise of artificial intelligence, some companies are taking content from websites like Reddit to train their AI systems without asking or giving credit to the original creators. This has become a big issue because it doesn’t respect the people who made the content or the websites that host it.
What is Reddit doing?
Reddit’s update to the robots.txt file is aimed at controlling how its content is used. They’re also going to limit and block unknown computer programs and bots from accessing their site if they don’t follow Reddit’s rules or have permission to use the content.
Reddit says these changes won’t affect most people or good organizations like researchers or groups that save internet history (like the Internet Archive). Instead, they’re trying to stop AI companies from using Reddit’s content without permission. However, these AI programs could still ignore Reddit’s rules.
Recent investigation and responses
The announcement follows a report from Wired, which found that an AI-powered search company called Perplexity was taking content from websites, even though it was told not to in the robots.txt file. Perplexity’s CEO argued that these rules are not legal requirements, sparking a debate about how websites can protect their content.
The Reddit data belongs to Google, for now
Reddit’s new rules won’t affect companies that already have agreements with them. For example, Reddit has a $60 million deal with Google, allowing Google to use Reddit’s data for its AI projects. This shows that Reddit is careful about who can use its data and wants to make sure they’re trusted partners.
“Everyone who uses Reddit’s content must follow our rules to protect Reddit users,” Reddit said in a blog post. “We choose carefully who we work with and trust with access to Reddit content.”
Looking ahead
This change by Reddit is part of their effort to control how their data is used, especially by companies for commercial reasons. It shows a growing trend among websites to protect their content in the age of AI and big data.
Reddit’s move sends a clear message: while AI has great potential, respecting where data comes from and getting permission is really important. As the internet changes, Reddit’s actions might influence how other websites protect their content and users’ rights.
All images are generated by Eray Eliaçık/Bing
