This seems like a potential actual good use of AI. Can't have been much fun to train it though.
And is there any risk of people turning these kinds of models around and using them to generate images?
this post was submitted on 21 Nov 2024
153 points (97.5% liked)
Technology
59588 readers
3051 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
I think image generators in general work by iteratively changing random noise and checking it with a classifier, until the resulting image has a stronger and stronger finding of “cat” or “best quality” or “realistic”.
If this classifier provides fine grained descriptive attributes, that’s a nightmare. If it just detects yes or no, that’s probably fine.
Nobody would have been looking directly at the source data. The FBI or whoever provides the dataset to approved groups, but after that you just say "use all the images in this folder" and it goes. But I don't even know if they actually provide real full-resolution images, or just perceptual hashes, or downsampled images.
And while it's possible to use the dataset to generate new images assuming the training data had full-res images, like I said, I know they investigate the people making the request before allowing access. And access is probably supervised and audited.
load more comments
(9 replies)
This is a great development, albeit with a lot of soul crushing development behind it I assume. People who have to look at CSAM or whatever the acronym is have a miserable job, so I'm very supportive of trying to automate that away from people.
Yeah, I’m happy for AI to take this particular horrifying job from us. Chances are it will be overtuned (too strict), but if there’s a reasonable appeals process I could see it saving a lot of people the trauma of having to regularly view the worst humanity has to offer without major drawbacks.
me
no
rikey
... robo chocolate?
I think all CSAM should be destroyed out of respect for the victims, not proliferated. I don't care who is hanging onto this material or for what purpose.
This ain't about the victims... It never was, otherwise churches would NOT exist in current form.
This is about police and corpo state gaining power.
load more comments
(3 replies)
At this point how does it differ w/ generating AI powered CP? morons
Uh, well this one tells you if an image looks like it or not. It doesn’t generate images
If it knows if an image looks like it it can generate something like it, one step further
Correct, this kind of software is trained on CP data. So such models can be easily used to generate CP instead of recognizing it, which makes them very dangerous indeed.
Same idea as the current models that are trained to recognized cars, these models can also be used to generate a car from noise as a starting poiint.
In pretttty sure you can’t just run it in reverse like that. There’s a whole different training and operation methodology you have to use to support generating images rather than simple text classification
load more comments
(1 replies)
It differs in basically being something completely different. This is a classification model, doesn't have generative capabilities. Even if you were to get the model and it's weights, and you tried to reverse engineer an "input" that it would classify as CP, it would most likely look like pure noise to you.
Moron
Generate porn, classificate output, result very young looking models.
Moron
So you need to have a model that generates CP to begin with. Flawless reasoning there.
Look, it's clear you have no clue what you're talking about. Stop demonstrating it, moron.
Alright, I found the name of what I was thinking of that sounds similar to what they're suggesting: generative adversarial network (GAN).
The core idea of a GAN is based on the "indirect" training through the discriminator, another neural network that can tell how "realistic" the input seems, which itself is also being updated dynamically. This means that the generator is not trained to minimize the distance to a specific image, but rather to fool the discriminator. This enables the model to learn in an unsupervised manner.
load more comments
(2 replies)
Not CP, but normal porn and select on CP traits, moron
https://en.m.wikipedia.org/wiki/False_positives_and_false_negatives
Not that I think you will understand. I'm posting this mostly for those moronic enough to read your comments and think "that seems reasonable"
Thanks
The model I use (I forget the name) popped out something pretty sus once. I wouldn't describe it as CP, but it was definitely weird enough to really make me uncomfortable. It's the only thing it ever made that I immediately deleted and removed from the recycling bin too lol.
The point I'm making is that this isn't as far fetched as you believe.
Plus, you can merge models. Get a general purpose model that knows what children look like, a general purpose pornographic model, merge them, then start generating and selecting images based on Thorn's classifier.
You can't merge a generative model and a classification model. You can run then in series to get a bunch of false positives/hallucinations, but you can't make it generate something from the other model.
When I said a "general purpose model that knows what children look like" I didn't mean the classification model from the article. I meant a normal, general purpose image generation model. When I said "that knows what children look like" I mean part of its training set is on children, because it's sort of trained a little on everything. When I said "pornographic model" I mean a model trained exclusively on NSFW content (and not including any CSAM, but that may be generous depending on how much care was out into the model's creation).