Zagorath comments on Piefed now has a built-in alternative to Lemmyverse to discover and search for active communities, with the "Active people" filter on the communities page (weekly basis)

[–] Zagorath@aussie.zone 1 point 1 year ago (4 children)

Eh, I don’t think it’s that surprising. Getting a list of comments on a post vs getting them from a search term are very similar operations, so it doesn’t make too much sense for these to have different queries in the backend

Sure, but one would have thought that the ordering in a search is fundamentally different from the ordering in other places. Because you want something that contains the words you've searched for near each other to appear ahead of a post that has those words scattered at random because it's a 500 word essay. You want exact word matches prioritised ahead of entirely unrelated words that include the same characters. Like "enum" should turn up your comment, but rank a comment that contains the text "renumbers" much more lowly. A particularly smart search page might keep "enumerate" high while rejecting "renumbers", though.

Of course, it's true that at least in the current latest release, Lemmy fails at all of this. I hope 1.0 is at least fixing some of it?

permalink

fedilink

source

parent

hideshow 4 child comments

[–] flamingos@feddit.uk 2 points 1 year ago* (last edited 1 year ago) (3 children)

This doesn't have anything to do with sort ordering though, which is based on time and votes. Text search is just a filter on top of sorting.

You want exact word matches prioritised ahead of entirely unrelated words that include the same characters. Like “enum” should turn up your comment, but rank a comment that contains the text “renumbers” much more lowly. A particularly smart search page might keep “enumerate” high while rejecting “renumbers”, though.

Of course, it’s true that at least in the current latest release, Lemmy fails at all of this. I hope 1.0 is at least fixing some of it?

How Lemmy does text search is via pg_trgm which works by breaking down both the content text and search text into trigram* and if the content contains enough of the search trigrams, it's considered to match the search term.

* A trigram is just a 3 character 'words', for example the trigram of 'enum' is {" e"," en",enu,num,"um "}.

What you're describing is closer to a tsvector, so you could open up an issues on Lemmy's GitHub to move from trigram to tsvector. One advantage trigrams have though is that they're language agnostic while tsvectoss need both a dictionary and to know the language (thankfully, Lemmy already has this info via the language setting, though the way it's stored will need to be changed to accommodate this). But tsvectors does provide much more intuitive language matching, like what you outlined.

permalink

fedilink

source

parent

hideshow 3 child comments

[–] Zagorath@aussie.zone 2 points 1 year ago (2 children)

This doesn’t have anything to do with sort ordering though, which is based on time and votes. Text search is just a filter on top of sorting.

That doesn't feel like how search should work. It should be ranking results that fit the search query better higher than ones that fit it less. Regardless of how the search is done, that should remain true. So if you're using trigram matching, instead of a binary "does the comment contain 80% of the trigrams in the search query", it should be "if it contains 100% of the trigrams from the search query, rank it higher than something with 90% match, which is higher than 80%." Or maybe not that precisely, but something so that more relevant results appear above less relevant ones.

Without doing something like that, it's just...not very useful. Which is the observed behaviour of search on Lemmy right now which started this whole conversation.

permalink

fedilink

source

parent

hideshow 2 child comments

[–] flamingos@feddit.uk 2 points 1 year ago (1 child)

That doesn’t feel like how search should work. It should be ranking results that fit the search query better higher than ones that fit it less.

But the existing filters already prescribe an order outside of how closely the search term matches, you brought up top month and I don't see how you'd want that to work other than a binary filter sorted by votes.

What you're describing would be a new sort order, analogous to Reddit's 'Relevance' sort. It's certainly doable with postgres' builtin distancing operators, though it be slower.

permalink

fedilink

source

parent

hideshow 1 child comment

[–] Zagorath@aussie.zone 1 point 1 year ago

you brought up top month and I don’t see how you’d want that to work

The truth is I don't want "top month". What I really want is "best result, filtered by this month". But unfortunately that doesn't exist, and in the absence of that, I use "top month".

permalink

fedilink

source

parent