OpenAI just admitted it has a bot that crawls the web to collect AI training data. If you don't block GPTbot, that's self-sabotage.

AI scraping bots are threatening the web's grand bargain. OpenAI's GPTbot crawler is being blocked by online content creators.

ai ai training ai training data artificial intelligence bot bots chatgpt copyright crawler creators data generative-ai google gpt-4 gptbot large language models llms online content online publishing openai scraping search engine tech insider training training data web websites