5
Content Warning (lemmy.world)
submitted 2 weeks ago by to c/perchance@lemmy.world
25 comments fedilink hide all child comments
 

My generator, "Bestaigen," now displays the following warning—as shown in the attached image: "Warning: The moderation system flagged this page because it may log or publicly show user inputs that one might expect to be private by default. Please do not input any private information on this page. If you are the creator of this page, you can ask the AI ​​Helper to fix this issue." Furthermore, it no longer appears in the "Recently Updated Generators" list at "https://perchance.org/generators." My generator has been live for about two months, and this issue did not exist previously. The sequence of events is as follows: We have a feature called "Prompt Gallery." A user shared the following message in the chat:

"9V3A: URGENT: Illegal content found in Best AI Gen prompt history, On October 20th, 2026, while using the Best AI Gen image generator (perchance.org/bestaigen#generation), I opened the prompt dropdown menu and found user-generated prompts publicly visible in the prompt history depicting graphic sexual abuse of minors, which included characters described as young as 7 years old. Multiple such prompts were present and could be viewed and copied by anyone. I understand the site's Pedo Shield makes users responsible for generated content, but these prompts are publicly accessible and appear unmoderated. I have already reported this to Cybertipline. I'm requesting the immediate removal of these prompts from the prompt history, and a review of the generator's prompt history for further illegal content. Thank you for your time."

Although "Prompt History" is intended to be a private space where each user sees only their own prompts, I realized the user was actually referring to the "Prompt Gallery." Consequently, I expanded the "Pedo Shield" AI-moderation tool—already in place for our generator—to also cover the "Prompt Gallery." I subsequently informed the individual who raised the concern, as well as the generator's user base in general, via the chat. Following this, another user quoted my message and posted the following in the chat:

"KUV3: Probably best to allow the prompts to be filtered or removed entirely. Also allowing individuals to choose whether or not their prompts even get publicly shared without their consent regardless of anonymity. Some prompt styles are formulas for work they do, while most are petty indulgence. People who choose to share it can show off their prompt abilities. While obviously some are just. Horrible."

Agreeing with the user's concerns and thoughts, I added a "Share My Prompts" toggle—set to OFF by default—to the "Generate" page. This toggle controls whether prompts entered into the "prompt description" field are sent to the "Prompt Gallery" when the "Generate" button is clicked. After implementing this change and saving the code, I refreshed the page (F5) and encountered this warning: "Warning The moderation system flagged this page because it may log or publicly show user inputs that one might expect to be private by default. Please do not input any private information on this page. If you are the creator of this page, you can ask the AI ​​Helper to fix this issue.

🔙 Leave I understand "

I am not sure why this happened:

  • Was it due to the AI ​​code itself? (My generator had been public for two months without any issues prior to these changes.)
  • Did someone report it? Could it be related to the "Prompt Gallery" or a "Cybertipline" issue?
  • Could the automated bots on Perchance have monitored the chat or analyzed the saved generator code, resulting in a "false flag"? I don't know.
  • Regardless, the goals that both I and my community wish to achieve—and the values ​​we aim to protect—are clear.
  • As a safeguard for minors: I have implemented several safeguards: prompt filtering, a review process for images intended for upload to the public gallery, and strict constraints for the chatbot—including an 18+ age requirement for roleplay (RP) and fictional storytelling, as well as a complete prohibition on the inclusion of minor characters.
  • The "Content Mode" setting in the generator is configured to the default "G" (General) level.
  • A "Pedo Shield" notification and warning appear in the welcome panel for first-time visitors; this same warning is also permanently displayed at the very top of the generator's settings page.
  • Regarding the optional user login system I previously considered: I decided against implementing it after users expressed a strong preference for remaining anonymous.

Following the appearance of this warning window, I had an AI review the code, removed potentially ambiguous phrasing, and reverted all recent changes—yet the generator still displays the same warning window. Could you please assist me? What, specifically, is triggering this warning? The only user-provided content that could potentially be made public—prompts—is now set to "private" by default, subject to user preference. Dear Perchance moderators, if there are any other issues or content you deem inappropriate or problematic, please let me know; I am fully prepared to make the necessary adjustments.

you are viewing a single comment's thread
view the rest of the comments
[–] [S] 1 point 2 weeks ago (19 children)

Hi @Mack2 im sorry to hear that. But weird part, your generator totaly closed to enter? Also i saw this; https://lemmy.world/post/28283458/16506859 From @Perchance : "Yeah imo images have been pretty bad and out of date for like 1.5+ years now :') Everyone please leave perchance - it’s embarrassing for me at this point.

Still working on getting the new model cheap enough to be free+unlimited, which I already knew would be the hard part, but it’s still harder than I expected. The model is 10x larger, so need literally an order of magnitude of optimizations. If everything goes right (which it won’t), the upgrade is a few weeks away™. Then text model upgrade.

Also, apologies @RudBo@lemmy.world and anyone else who had/has their gen incorrectly marked with “Content Warning” / age verification checkbox. Still lots of false positives due to ai based detection and bugs. For now, you might be able to move lists/things that are triggering false positives to another generator and import them. This stuff is needed due to ad network requirements. Mo website traffic, mo problems."

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 19 child comments
    •
  • [–] 1 point 2 weeks ago (18 children)

    That's old...I can get into mine and anyone who didn't close out their browser session can still use it. My normal active users is about 80-120...right now it's 25 🫤

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 18 child comments
    •
  • [–] 4 points 2 weeks ago* (last edited 2 weeks ago) (17 children)

    @Mack2@lemmy.world sorry about this! I'm rolling out a new moderation system - can you check if your generator has anything that might trigger the "may publicly show user inputs that most users would expect to be private by default" message? And also features that might be interpretted as specifically designed to create nsfw deepfake images of real people. You can ask the AI Helper to do a search for anything that might be triggering it. If you've checked and made any relevant changes and waited an hour or so and it's still not fixed, please let me know. If you can't find anything and think it might be a false positive, please also let me know and I'll dig deeper. It's very possible this new system needs some tuning.

    @Bestaigen@lemmy.world I believe the message "may log or publicly show user inputs that one might expect to be private by default" is gone now. The reason it didn't go away immediately after you fixed the issue is because the moderation system was overloaded. Sorry about that, and thank you for letting me know. I'm working on making it faster so this doesn't happen again, though it'll likely always take at least 20 minutes, so I should make that clear in the UI somewhere.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 17 child comments
    •
  • [–] [S] 3 points 2 weeks ago* (last edited 2 weeks ago) (1 child)

    First of all, thank you very much for your quick response. Yes, the previous warning—"Warning: The moderation system flagged this page because it may log or publicly show user inputs that one might expect to be private by default"—has disappeared, but now I’m getting a "Mature Content" warning. While I understand why this is happening—given that my generator’s design included an "NSFW" option under the "Content Mode" settings—I’d like to remove it. It’s quite embarrassing for me when someone visits my generator and is immediately confronted with an age-verification (18+) prompt. I don’t want to be perceived as someone who creates "pornographic content generators"—especially since people I know in real life, including my spouse, visit my site. I did add an age-verification step yesterday for when the "Content Mode" switch is toggled to "+18," but it seems your moderation AI didn’t consider that sufficient; the age-verification prompt still pops up the moment someone lands on the page. This can also make the user feel uncomfortable. I’ve decided to strip my generator of all NSFW or sexually suggestive content, as it seems to be the only way to remove this warning. That said, I still believe that if content is "gated" within the code by the generator’s creator, it shouldn’t trigger a warning—after all, a "typical user" wouldn’t even encounter that content unless they manually changed the "Content Mode" themselves (and I had already implemented an age-verification prompt for that specific scenario).

    Since I have you here, I have a few other questions and requests; I would be very happy and grateful if you could look into them and provide an answer:

    • What steps are required to become a "pinned" (featured) generator owner at https://perchance.org/generators? I haven’t been able to get a clear or sufficient answer—neither from AI nor from my own research. I’ve noticed that some generators with only 14 views get pinned, while others that were created just a few hours ago also get pinned. It feels a bit unfair.
    • Some features in my generator rely on "Durable State" (DS), which is subject to a 50ms traffic limit. This limit is far too low; even a very simple feature using DS can easily trigger a "quarantine" status due to "traffic overflow." This affects other features as well, creating a snowball effect. Is it possible to increase this 50ms traffic limit?
    • Because of the DS traffic constraints, I’m unable to build a proper, functional chat panel. I’ve been forced to use the "Default Comment Plugin," but it is extremely limited. Increasing both the DS storage space and the traffic limit would provide us, as generator owners, with more creative possibilities. Furthermore, regarding the "Default Comment Plugin," would it be possible to add features such as badges, a list of online users, a private messaging system, a "moderator" role (with admin-level permissions, since we aren't always online), and reaction emojis for chat messages (e.g., thumbs-up, heart, sad face, etc.)?
    • Currently, as generator owners, our permissions are extremely restricted; we cannot even delete images from our own generator's gallery. I am currently able to hide or "ban" problematic content—such as images depicting child abuse or other objectionable material—by using a custom "Image Card" system that conceals the default gallery. However, these images still remain visible in the "Default Perchance" gallery associated with the generator. For instance, the "Block this user" button found on default images is contained within an iframe, which prevents it from functioning with my custom card system. Could you make these accessible to generator owners? This would allow the buttons we integrate into our own card systems to function in sync with the standard "Block this user/report" features.
    • I have personally created a password-protected, private page for my generator (https://perchance.org/bestaigen#closecodeadmin), which is the only place where I can report or hide images that I identify as problematic. I believe that generator owners who are logged in should have access to an admin panel—perhaps located under "Perchance default/Account - View your generators"—that grants them the authority to ban users, delete images from their generator's gallery, adjust ratings, and assign moderators. Currently, every image generated places a burden on you; by enabling generator owners to manage this, the workload could be shared. Wouldn't this be a significant step toward a cleaner Perchance?

    Thank you very much in advance for your response; I look forward to hearing from you. Perchance is a fantastic platform—please continue to show it your care and support. Perchance = Real Free Internet. @perchance@lemmy.world

    @perchance@lemmy.world

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 2 weeks ago

    the default comment plugin and image-to-text gallery should open some more api/options :(

    owner either user server and suffer from 50MiB cap, or hybrid/geek and hack here and there currently

    some build a custom gallery by storage image url in comment and render them. wow, so creative! live will always find a way out right?

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 2 weeks ago (3 children)

    The message changed appearance but still won't allow incognito users through. Also now has mature warning and privacy warning. Mature is correct, privacy shouldn't be.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 3 points 2 weeks ago (2 children)

    Please check this (it may be wrong - from ai helper):

    The "Save as proven" button in Proof Lab still sends the user's phrase, with no opt-in, to a server vault that any visitor can read. Meanwhile the page tells users it was "Saved as proven on this device".

    Also check for anything that could be reasonably interpreted as a feature designed for creating celebrity/real-people nudes/deepfakes, since I think it's flagged for that too. If there's nothing like that, let me know - it could be a false positive.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 weeks ago

    Good catch! Fixed the "Save as proven" button. Now 2 buttons, one for save to device, and another for share to public. Deleted entries in my own AI Prompt Helper that would provide a blended Celebrity likeness, and my Enhance feature and Bess (AI Assistant) refuse requests for celebrity likenesses. Hopefully that will be the end of this. All saved. I'll keep checking over the next hour or so.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 2 weeks ago (9 children)

    Thanks for getting back to us. I had it run a thorough scan...it said it's fixed everything. The flag is still blocking users. VPN or incognito only get the "Leave" option.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 9 child comments
    •
  • [–] 2 points 2 weeks ago (8 children)

    I’m one of your regulars (if we’re talking about “Mack’s Place”), and I can’t get in. I don’t use VPN. I have no idea if I’m an “incognito” user. I only get the “leave” option.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 8 child comments
    •
  • [–] 1 point 2 weeks ago (6 children)

    Working on this. I'm not sure why there is only the "Leave" option. I made an assumption that it was mainly due to browser security settings, VPN, or incognito mode. It may be that you have to sign in to bypass it. Whatever it is I'm hoping my last edits fixed the concerns that the moderation system flagged. It may take a while to clear the flag. I'll post here when it is resolved.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 2 points 2 weeks ago (5 children)

    Even when signed in to perchance, I only get the leave option.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 2 points 2 weeks ago (4 children)

    This right here. I have been a regular at Mack's for months. I actually created this account just to add my experience since yesterday. I have turned off all VPN, AdBlock, anything my browser (Opera) might get in the way. I confirmed mature content on another page and refreshed it.

    Current situation is still a "Mature Content" warning that only let's me leave.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 1 point 2 weeks ago (3 children)

    I'm working on it. The moderation system is a bit crazy at the moment. I've done everything I can do. The page is clearly marked 18+ NSFW, it has a built in consent gate, I've deleted anything that might cause age related issues. I've messaged @perchance@lemmy.world again to ask what I can do to get this working. Thanks for your patience.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 2 points 2 weeks ago (2 children)

    Always. In the meantime, i got to tape together my own little project. No self-advertisement.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 weeks ago

    If this keeps up I may start a new project and scrap Mack's Place. It seems like anything NSFW is going to be an issue. Which is sad for a place that's advertised to be unlimited and uncensored. I could delete every preset and AI tool I have and a user could still create risky content with just the text box. It seems a little heavy-handed, but I understand the desire to protect what you've built. I'm of the opinion that anything that the dev doesn't want being produced should be moderated at the actual plugin and not rely on individual page creators to moderate it. That way if something changes it doesn't break the whole page, it just restricts the output.

  • permalink
  • fedilink
  • source
  • parent