Cloudflare is launching a marketplace that will let websites charge AI companies for scraping their content, aiming to give smaller publishers more control over how AI models use their data. Large AI models scrape thousands of websites to train their systems, often without compensating the content creators, which could threaten the business models of many smaller websites. The marketplace, launching next year, will allow website owners to negotiate deals with AI model providers, charging them based on how often they scrape the site or by setting their terms.

Cloudflare’s launch of AI Audit is a big step for website owners to gain better control over AI bot activity on their sites. Providing detailed analytics on which AI bots access their content empowers site owners to make informed decisions about managing bot traffic. The ability to block specific bots while allowing others can help mitigate issues related to unwanted scraping, which can negatively impact performance and increase operational costs. This tool could be handy for businesses and content creators who rely on their online presence and want to safeguard their resources.

Cloudflare’s CEO, Matthew Prince, believes this marketplace will create a more sustainable system for publishers and AI companies. While some AI firms may resist paying for currently free content, Prince argues that compensating creators is crucial for ensuring the continued production of quality content. The initiative could help balance the relationship between AI companies and content creators, allowing even small publishers to profit from their data in the AI age.