AI companies are reportedly still scraping websites despite protocols meant to block them

Perplexity, a company that describes its product as "a free AI search engine," has been under fire over the past few days. Shortly after Forbes accused it of stealing its story and republishing it across multiple platforms, Wired reported that Perplexity has been ignoring the Robots Exclusion Protocol, or robots.txt, and has been scraping its website and other Condé Nast publications. Technology website The Shortcut also accused the company of scraping its articles. Now, Reuters has reported that Perplexity isn't …

ai companies ai search ai search engine block companies fire forbes free multiple perplexity platforms product robots scraping search search engine stealing story technology & electronics them websites wired