

If the same user downvotes everything I’ve ever said,
Right. How would you know what “the same user” is? Let’s say that your posts get downvoted at random intervals by 5-10 users in the first 45-120 minutes. They all have different user names. What are you going to do? Create a report against any particular user and hope that the mods look into it?
Yes, but that’s kind of my point?
if downvotes are public, the admin of your instance can see who is downvoting you and then they can take action. If the downvotes are coming from an instance that hides the real user for every vote, you and the admin are SOL.