Hi, I’m building a personal website and I don’t want it to be used to train AI. In my robots.txt file I blocked:

  • ChatGPT-User
  • GPTBot
  • Google-Extended
  • FacebookBot

What bots should I also add? Are there any other ways to block AI bots?

IMPORTANT: I don’t want to block search engine crawlers, only bots that are used to train AI.

  • ⚡⚡⚡@feddit.de
    41·
    1 year ago

    If you want this to work reliably for future bots BUT also want to allow search engines, you’ll loose this game.

    BTW: What makes you sure, that the search engine bot of Google does not crawl your website, store it in a cloud and AI is then used to later allow the search engine users to ask questions about your website and get AI generated answers. I think, that’s the goal of the search engines to improve results with AI…