LLM scrapers are taking down FOSS projects’ infrastructure, and it’s getting worse.

  • refalo@programming.dev
    link
    fedilink
    arrow-up
    0
    ·
    2 months ago

    What you’re doing is filtering out bots that can’t be bothered to execute JavaScript. You don’t need to do a computational heavy PoW task to do that.

    Most bots and scrapers from what I’ve seen already are using (headless) full browsers, and hence are executing javascript, so I think anything that slows them down or increases their cost can reduce the traffic they bring.

    Canvas fingerprinting filters out bots better than PoW

    Source? I strongly disagree, and it’s not hard to change your browser characteristics to get a new canvas fingerprint every time, some browsers like firefox even have built-in options for it.

    • SkotchY@ieji.de
      link
      fedilink
      arrow-up
      0
      ·
      2 months ago

      @refalo @sudo If Proof of Work gets widely adopted I foresee a future where bot running data-centers can out-compute humans to visit sites, while old devices of users in poorer countries struggle to compute the required task for hours … Or is that fear misguided?

      • sudo@programming.dev
        link
        fedilink
        arrow-up
        0
        ·
        2 months ago

        Admins will always turn down the bot management when it starts blocking end users. At that point you cough up the money for the extra bandwidth and investigate different solutions.