126
Admins: Instnace randomly running extremely slowly? Check for this (lemmy.world)
submitted 1 year ago by to c/fediverse@lemmy.world
36 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 6 points 1 year ago (8 children)

This is for Lemmy I presume (or also for Piefed or Mbin)? You've modified yours heavily though, I thought, which could complicate matters. I wonder if you are having those bot scraping issues that semi-recently (a month or so ago?) started increasing in frequency. So many instances now have a human detector before letting you in whereas before it was not necessary.

  • permalink
  • fedilink
  • source
  • hideshow 8 child comments
    •
  • [–] 4 points 1 year ago (5 children)

    PieFed has a similar API endpoint. It used to be scoped, but was changed at the request of app developers. It's how people browse sites by 'New Comments', and - for a GET request - it's not really possible to document and validate that an endpoint needs to have at least one of something (i.e. that none of 'post_id' or 'user_id' or 'community_id' or 'user_id' are individually required, but there needs to be one of them).

    It's unlikely that these crawlers will discover PieFed's API, but I guess it's no surprise that they've moved on from basic HTML crawling to probing APIs. In the meantime, I've added some basic protection to the back-end for anonymous, unscoped requests to PieFed's endpoint.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 1 point 1 year ago (3 children)

    Could you elaborate on this:

    it's not really possible to document and validate that an endpoint needs to have at least one of something

    In what sense it is not possible, as I can easily see it done in the code?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 1 point 1 year ago (2 children)

    It's straight-forward enough to do in back-end code, to just reject a query if parameters are missing, but I don't think there's a way to define a schema that then gets used to auto-generate the documentation and validate the requests. If the request isn't validated, then the back-end never sees it.

    For something like https://freamon.github.io/piefed-api/#/Misc/get_api_alpha_search, the docs show that 'q' and 'type_' are required, and everything else is optional. The schema definition looks like:

    /api/alpha/search:
    get:
      parameters:
        - in: query
          name: q
          schema:
            type: string
          required: true
        - in: query
          name: type_
          schema:
            type: string
            enum:
              - Communities
              - Posts
              - Users
              - Url
          required: true
        - in: query
          name: limit
          schema:
            type: integer
          required: false

    required is a simple boolean for each individual field - you can say every field is required, or no fields are required, but I haven't come across a way to say that at least one field is required.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •