Major publishers – including Reddit, Yahoo, Medium, and Quora – today announced support of a new licensing standard, Really Simple Licensing (RSL). In theory, RSL gives websites leverage to demand compensation when AI companies scrape and train on their data.
Why we care. Until now, publishers could only block or allow bots via robots.txt. RSL upgrades that model, letting publishers embed licensing and royalty terms directly into robots.txt files, online books, videos, and datasets.
How it works. Publishers can set subscription, pay-per-crawl, or pay-per-inference fees (compensation when an AI uses their content in responses).
- Bots for non-training purposes (like search indexing or archiving) can crawl as usual.
- RSL relies on AI companies’ cooperation. Without buy-in, enforcement will be a challenge.
The big picture. The RSL Collective, led by RSS co-creator Eckart Walther and former Ask.com CEO Doug Leeds, is pushing the standard.
- They see it as a scalable business model for the web, modeled after rights groups like ASCAP in music.
- Fastly is working with RSL to build “gatekeeper” tech that can admit or block bots based on licensing compliance.
Supporters. Brands that are supporting RSL include:
- Quora
- Yahoo
- Medium
- Adweek
- Fastly
- Internet Brands
- The MIT Press
- O’Reilly
- People Inc.
- wikiHow
- Ziff Davis
Between the lines. Some big media players (e.g., The New York Times, News Corp, Vox Media) have cut direct licensing deals with AI firms. But RSL aims to simplify the process for any publisher, big or small.
The catch. AI model builders have a history of ignoring robots.txt. Without enforcement teeth, RSL’s success hinges on whether major AI players adopt the standard.
Not the first. Cloudflare, which powers about 20% of the internet, is also cracking down on AI bots. It blocks AI crawlers by default and is testing a Pay Per Crawl system that charges AI companies for access – a parallel effort to RSL that could reshape how AI firms get training data.
Really Simple Licensing. You can dig deeper on the RSL website.
Search Engine Land is owned by Semrush. We remain committed to providing high-quality coverage of marketing topics. Unless otherwise noted, this page’s content was written by either an employee or a paid contractor of Semrush Inc.
Source link
Disclaimer
We strive to uphold the highest ethical standards in all of our reporting and coverage. We blogs.grocliq.com want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It’s possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.
Website Upgradation is going on for any glitch kindly connect at [email protected]
