While I think this is a bit overblown in sensationalism, any company that allows user generative AI, especially as open as using LoRas and any amount of checkpoints, needs to have very good protection against synthetic CSAM like this. To the best of my knowledge, only the AI Horde has taken this sufficiently seriously until now.
So far im the only commentator who is fine with this. the problem with csam to me is children being molested. if its art, or stories, or basically made up and not reality. then im pretty much fine with anything. I mean I may not want to consume it myself but I don’t see a problem with it.
I agree with you
With fictional content their is no child involved and their certainly isn’t anything living involved
I’d just wish governments in countries that have made this class of fictional images illegal would go after real child molesters instead of makers / consumers of fictional images where no living being is involved
People who consume and make fictional content won’t harm anyone
I’d agree with the caveat that a model that may be trained on actual CSAM is a problem. Anything that is an actual product of, or cannot exist without, child abuse should be absolutely prohibited and avoided.
I’m not sure whether there is such a model out there, but now that I imagine it, I assume it’s inevitable there will be one. Apart from being disturbing to think about, that introduces another problem - which is, once that happens how will anyone know the model was trained on such images?
Oh I can totally agree with the training part but that can’t be fought at the ai level it needs to be stopped at the csam level.
I don’t think this is solvable… Unless general images of children are excluded from the training data, which is probably a good idea.
Pretty much what SDXL did to “fix” this. They excluded all lewd images from training.
New Lemmy Post: a16z Funded AI Platform Generated Images That “Could Be Categorized as Child Pornography,” Leaked Documents Show (https://lemmy.dbzer0.com/post/9782750)
Tagging: #StableDiffusion(Replying in the OP of this thread (NOT THIS BOT!) will appear as a comment in the lemmy discussion.)
I am a FOSS bot. Check my README: https://github.com/db0/lemmy-tagginator/blob/main/README.md
I suspect that Marc Andressen is fine with AI-generated kiddie porn. He probably tells himself that no children were actually harmed. Never mind that the AI had to get its training data somewhere…
To be fair, the AI doesn’t have to see CSAM to be able to generate CSAM. It just has to understand the concept of child and the various lewd concepts, and it can then mix them together.
Which only further supports my opinion that no programmer should be considered employable until they’ve read and understood Mary Shelley’s Frankenstein, and understood that they are in Frankenstein’s position.
The AI is able to merge images, you can definitely merge child pics with “normal” porn. So technically you can make “CSAM” without having actual CSAM in the training data. The first versions of some of the image AIs have been able to do that, and there was definitely no CSAM in the training data.
Its not good to have that around, but definitely better than actual CSAM.
