All the posts about Reddit blocking everyone except Google and Brave got me thinking: What if SearNGX was federated? I.E. when data is retrieved via a providers API, that data is then federated to all other instances.
It would spread the API load out amongst instances, removing the API bottlenecks that come from search providers.
It would allow for more anonymous search, since users could cycle between instances and get the same results.
Geographic bias would be a thing of the past.
Other than ActivityPub overhead and storage, which could be reduced by federating text-only content, I fail to see any downside.
Thoughts?
You must log in or register to comment.
I think you are not a computer programmer. Trying to build an index of the web by querying other search engines is not an efficient or sensible way to do things. Using ActivityPub for it is insane. Sharing query results in the obvious way might help a little during events where everyone searches for the same thing all at once, but in a relatively small pool of relatively sophisticated Internet users I don’t think that happens often enough to justify the enormous amount of work and complexity.
On the other hand a distributed web crawler that puts its results in a free and decentralized database (one appropriate to the task; not blockchain) might be interesting. If the load on each node could be made light enough and the software simple enough that millions of people could run it at home, maybe it could be one way to build a new search engine. If that needs doing and someone has several hundred hours of free time to get it started.
