LLM scrapers are taking down FOSS projects’ infrastructure, and it’s getting worse.

  • refalo@programming.dev
    link
    fedilink
    arrow-up
    0
    ·
    2 days ago

    What you’re doing is filtering out bots that can’t be bothered to execute JavaScript. You don’t need to do a computational heavy PoW task to do that.

    Most bots and scrapers from what I’ve seen already are using (headless) full browsers, and hence are executing javascript, so I think anything that slows them down or increases their cost can reduce the traffic they bring.

    Canvas fingerprinting filters out bots better than PoW

    Source? I strongly disagree, and it’s not hard to change your browser characteristics to get a new canvas fingerprint every time, some browsers like firefox even have built-in options for it.