21
Generators Making CP........Just NO! (lemmy.world)
submitted 6 months ago by to c/perchance@lemmy.world
25 comments fedilink hide all child comments
 

I'm so angry I'm shaking while typing this. Many others and I have reported this one alone, and it's still here! It looks like it's gotten more tabs to try to hide.

Please. Ban this filth. This is NOT okay.

There is no valid reason for children to be made on a generator that allows NSFW. And some of the tabs are filthy when I think of them in the context of CP.

you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 5 months ago (5 children)

I think people seem to forget:

  • This is a public website. Users can create and post their own generators. Often, these users are in control of the censorship, moderation, and safety of their generators, not the dev of the website. Its not solely his responsibility. Perchance is not a large corporate website; its a individual-made website driven by the community and users. It was never designed with universal safety like that to begin with.

  • Its ran and maintained by one person, who's already burning his money to keep the site up and running it while paying for these AI models. He has limited resources. Implementing stronger safety measures takes time, money, and resources. And this website is completely free! He makes money through ads for the AI image models (or model, I think he only uses one).

  • It's going to be nearly impossible for him to add a universal safety system for all these user-made generators, not without more resources and money. He's making very little from the website with ads. He can't do it at the moment give him some grace. Its hard to combat inappropriate images with limited resources. Same principle with Discord: they have a report system so they can handle problems but ultimately its their responsibility to the discord mods from user-made servers to manage rules, moderation, safety regulation, and reports on their server. Same with Reddit, you can create communities based off of an interest or topic. The mods are the ones in charge of all the rules and moderation. Reddit only handle reports submitted by people.

Perchance, like Discord and even Reddit, is a community/user-driven platform. The users are in charge of the things they create, not the developers of the platform. The developers can't oversee and deal with individual user created stuff. Do you think it will be easy for the creator to add some sort of rule, moderation, safety, or censorship issue to all these generators? No. It will take a very long time, especially with limited resources. As far as the AI image model generating illegal things such as the CSAM/CP, he would need to find a way to add censorship for the model. He can't inject it into the model; he has to code and program the censorship feature. But he'll have to import it into the ai image model plugin but he can't without building the censorship feature first. Platforms like Grok or Microsoft's image generator already implemented censorship. It'll blur or show a message the user cannot generate the content because its against the rules/inappropriate. He has to brainstorm how to design it, code it, program it, implement it, then start develop it from scratch, figuring out how to censor the images, words or prompts, figure out how to code and program it into the AI image model, texts or generators, on top of that—little funds, making some money only but running ads for the AI image models, a public community/user-driven website, thousands of individual-made generators,

ONE person paying and maintaining the servers, the domain, AI image models and AI generated text, functions, coding, and programming of the website and its features, burning all that money for those resources all by himself without making little to no money...it's not going to be as simple as you think. Censorship takes a lot of development: it isn't just coding few words into the AI and call it a day. No, you have to build and develop it, then implement it into the system. And he might already have plans or working on the features, improvements, bug fixes, etc. He just completed a migration. He will need time to work on this especially if the website was never designed or implemented such complex features like this. Good lord.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 2 points 5 months ago* (last edited 5 months ago) (4 children)

    Talk to other projects. There's ready-made solutions on Github, which you can just deploy and hook into your pipeline. No need to re-invent any wheel. I know db0 wrote a filter for the Fediverse. The AI Horde is also successfully filtering that kind of stuff. And they've being doing it for a long time now.

    It's done after generation an before the result gets sent to the user.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 1 point 5 months ago* (3 children)

    Yeah, I think something like that would be very useful and helpful. Its easier to use pre-made tools or features and integrating in your project or service. Consulting other projects is a smart thing to do because they can give you some insight about filters and censorship, stuff like. Filter for the Fediverse, like this one: https://github.com/db0/fedi-safety? Seems interesting, I also remember AI Horde, I think I used it a couple of times. It's great their filtering works as intended. I think the dev should take this advice, and maybe even hire or consult somebody to help him with this. I am not programmer or coder (I can't do this shiii without Microsoft's Copilot, it was the one helping with all the coding and troubleshooting lol. ) but maybe I can make a "filter" example, albeit a simple one, just as an example. I'd rather have filtered out words than Grok's horrible use of censorship, blurring images that's not even NSFW...

    I kinda wanna aim for:

    (child) ≠ [forbidden word]

    or

    (little\young\small) | girl/boy/child ≠ [forbidden word] Note: No specific order. Just the words contained in ( ) with (like girl/boy/child) to let the filter know to filter those forbidden words out those in combination of the other words.

    (of course not the only one)

    Filter system shouldn't censor innocent prompts, but rather specific ones based off the instructions its given. Child ≠ forbidden word example I've provided is a good way to add censorship without blurring images that uses the prompt: Two adults, Sally and Tom lays in bed together, wearing underwear. Sally wears bras and panties and Tom is shirtless in boxers. A prompt like this shouldn't be censored because Sally and Tom are adults, yet why is being censored anyways? If it allows NSFW as long as the adults are characters, then it shouldn't be an issue. This was Grok's mistake: allowing NSFW but had to enforce stricter censorship because the dumbahh developers allowed Twitter users to edited pictures in posts and creeps and pedos made sexualized images of women and children. Now Grok users can't generate any NSFW content involving consensual, legal adult characters only, after being marketed as "unrestricted/uncensored" type AI image generator. This is the type of issue we want to avoid. Perchance allows NSFW and not have this "family friendly" or "all ages" image. So allow NSFW content for any prompt containing adult-aged characters only. Shouldn't be too hard...

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 1 point 5 months ago (2 children)

    I'm thinking like (for programming filter only):

    contains/includes/with = (words go here) exclude/omit = (words go here)

    Now, you can't include everything at first, just keep adding and refining the filters over time or from reports from users. Very minor idea:

    Words (adjective, noun): little, young, small, tiny Containing/including with (nouns): boy, girl, child, kid

    (can even add suffix/prefix for like pre-teen, pre-adolescent, etc)

    Exclude/omit from (adjective, verb, adverb, preposition): [you get the idea]

    if there's a word with (-s, -es, -ed) would also include that too...

    This tells the filter to what to scan or analyze for. Whether the words are categorized or not (doesn't matter) the core concept remains the same. I think there may be similar pre-made solutions or tools on github like my idea, I would also look into those. Its not as hard as you think, just a bit time consuming to configure...

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 5 months ago* (last edited 5 months ago) (1 child)

    Yes. https://github.com/db0/fedi-safety is the Fedi code. And it leverages this project:

    https://github.com/Haidra-Org/horde-safety/

    Mind you're currently in the process of fleshing out your own interpretation of the wheel. My recommendation was not to do that. And your approach has the usual downsides: People can use other words, synonyms. Describe it. Or leverage wrong spellings. Jam an adjective in-between. Do an exact request, but in Chinese language... All that will circumvent filtering by simple word filters. And that's usually what happens 5 minutes after you roll out such a system. It's a) pretty much not effective at all. And b) it's complicated and you'll end up in a cat and mouse game.

    Fedi-/Horde-Safety use an image interrogator, to query what's actually in the image. That's immune to that. They also have all these heuristics. (You can have a look in their code.) It'll come with false-positives, though. It's a trade-off.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 5 months ago

    Ah, okay. I see horde-safety kinda like integrates the Fedi code in their filters.

    Fedi-/Horde-Safety use an image interrogator, to query what’s actually in the image. That’s immune to that. They also have all these heuristics. (You can have a look in their code.) It’ll come with false-positives, though. It’s a trade-off.

    Huh, now that's pretty interesting. I will defiantly take a look into their code, I want to see how they programmed it.

    Mind you’re currently in the process of fleshing out your own interpretation of the wheel. My recommendation was not to do that. And your approach has the usual downsides: People can use other words, synonyms. Describe it. Or leverage wrong spellings. Jam an adjective in-between. Do an exact request, but in Chinese language… All that will circumvent filtering by simple word filters. And that’s usually what happens 5 minutes after you roll out such a system. It’s a) pretty much not effective at all. And b) it’s complicated and you’ll end up in a cat and mouse game.

    Yeeaaaahhhh...I just realized that that now...emphasis on the process of fleshing out my own interpretation. I wouldn't recommend as something to actually make and integrate as censorship. I think I was literally doing that, trying to showcase how I thought how the censorship/filtering worked, like the basics of it...I knew it wouldn't be as accurate or effective because not all censorship/filters actually work like that...I guess in my head it was more of a "demonstration" rather than an actual tool/solution. I think I meant to demonstrate how the censorship/filter on what words and associations to focus on instead of banning innocent prompts because of too much censorship (Unfortunatly, that what Grok does. Because they effed up knowing releasing the ability someone's post without consent and now it all backfired, cranking up the censorship and making it practically unusable. And the fact the government is enforcing it also makes it worse) ...But you are 100% right: its not effective since people can use ways to circumvent the filter like you said, use different words, synonyms, mispellings, sentences, etc. (Man, I wouldn't be too surprised if it was less than 5 minutes...its insane...). Very ineffective and complicated...I just thought about its trying to keep adding words when people can bypass is hard as hell. And the cat and mouse game? God, couldn't be more true. Trying to add more words to the filter and people STILL find ways to bypass would be exhausting.

  • permalink
  • fedilink
  • source
  • parent