top 28 comments

sorted by: hot top controversial new old
[–] 25 points 1 year ago (9 children)

It should mention that Qwant (France) and Ecosia (Germany), announced last year a joint project for an independent European search index, although only for French and German according to this publication. https://betterweb.qwant.com/en/2024/11/08/ecosia-and-qwant-join-forces-to-develop-european-search-index/

  • source
  • hideshow 9 child comments
  • [–] 7 points 1 year ago (8 children)

    On doit apprendre le français (ou l'allemand) pour rechercher le web sans indices Américains? Défi accepté!

  • source
  • parent
  • hideshow 8 child comments
  • [–] 8 points 1 year ago* (7 children)

    On doit apprendre le français (ou l’allemand) pour rechercher le web sans indices Américains? Défi accepté!

    grue@lemmy.world 🔗 English 🤨

  • source
  • parent
  • hideshow 7 child comments
  • [–] 10 points 1 year ago (6 children)

    Right, I speak English natively but have been learning French.

  • source
  • parent
  • hideshow 6 child comments
  • [–] 2 points 1 year ago* (last edited 1 year ago) (4 children)
  • [–] 1 point 1 year ago (2 children)
  • [–] 5 points 1 year ago (7 children)

    Looks like for the hardware requirements for self hosting some of the open source options, I'll be saving up quite a bit for SSDs.

  • source
  • hideshow 7 child comments
  • [–] 4 points 1 year ago (2 children)

    FWIW, I gave YaCy a try a while back, and I agree with the article on that one. Shit tier results that make ancient AltaVista look good. Might be fine for intranet search. I like the idea of its distributed hosting, but pass on this one.

    Other poster mentioned SearXNG, and while I haven't delved into that too much, it's probably worth a check. Pass on YaCy.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 2 points 1 year ago (3 children)

    Consider self-hosting SearXNG, which can aggregate results and filter.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 3 points 1 year ago (2 children)

    Already am, but it still pulls results from the companies I want to separate myself from. I'd rather see what it takes/how well it performs to have my own indexer.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 2 points 1 year ago (1 child)

    Good luck with that one. It takes a lot of resources from my limited and aged experience, and I'm sure it's more now. Might be worth focusing the indexer on a topic area to start, just to get a feel for sizing (if your chosen solution supports that).

  • source
  • parent
  • hideshow 1 child comment
  • [–] 4 points 1 year ago

    That was a good read, well done! I'd be interested if you ever reconsider Startpage. I've had good success with that on my work computer.

  • source
  • [–] 2 points 1 year ago (5 children)

    Great article, appreciate that I'm not the only one concerned around some of the ethical choices Kagi has been making.

  • source
  • hideshow 5 child comments
  • [+] 3 points 1 year ago (2 children)
  • [–] 4 points 1 year ago (1 child)

    Last I saw they still paid Yandex for access to that index (weigh how important that is yourself), they also pushed back on suicide warnings if you ask Kagi how to kill yourself, and I learned from this article that they may be using additional data sources that contain higher levels of homophobic sentiment.

    Basically, the company's tagline is "Humanize the Web", but I don't think their actions thus far show we agree on what Humanize means.

  • source
  • parent
  • hideshow 1 child comment
  • load more comments (1 reply)
  • [–] 2 points 1 year ago (1 child)

    Same. I use Kagi because search is an essential function of my job and I can’t extract decent results from Google anymore, but if there were another engine with equally good results and a better ethical track record I’d switch.

    (There isn’t. I’ve tried Qwant, Ecosia, DuckDuckGo and a handful of others. Was not impressed.)

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 1 year ago*

    I share the sentiment. For me the results are acceptable, and being able to custom rank sites in results is very useful, but the killer feature is not having ads or forcing AI down my throat.

  • source
  • parent
  • [–] 2 points 1 year ago*

    Great list! I've been wanting to start using a smaller search engine with its own index, just for the sake of making sure there's an alternative to GBY. (Also, there's a new and useful acronym, haha.) Mojeek was the only one I was aware of before today.

    BTW, exalead.com doesn't seem to be a search engine anymore. I recognized that one because I remember discovering it a while ago...in 2007, maybe? But it looks like it's not available as a regular search engine now.

    edit: Also, this is the first really useful page I've read via the Gemini protocol, so thanks for that!

  • source
  • [–] 0 points 1 year ago (2 children)

    The plural of index is indices.

  • source
  • hideshow 2 child comments
  • 17 comments in c/privacy@lemmy.dbzer0.com view that post
    [–] 8 points 1 year ago (1 child)

    Some caveats here.

    DDG is a wrapper for Bing results, but the DDG business model is to use only the search term to serve ads mixed in to the Bing results. Startpage is similar, with Google results instead. So you have to dodge SEO pages occasionally.

    SearX isn't bad, but you have to trust the instance owner a bit.

    In short, any user should alternate and use multiple search engines. Spread your risk out.

  • source
  • hideshow 1 child comment
  • [–] [S] 3 points 1 year ago

    This is one part of what I wanted to convey, but for example - bing has some results - they uprank sites they are paid by - then send to ddg - ddg again adds ads - sends to you. /you are effectively using bing with better privacy (I don't mind ads, I can recognise/avoid them). And other problem with ddg is the thing that I brought up - bing is planning/already underway (depending on source of information) to scratch its search api, and mostly replace with a "ai powered api". Most users of bing api would have to find alternatives.

    Same with searx, you are more private (assuming you use a public instance), but it is not great for open web. the up/downranking done at source by bing/google can not be avoided (it can be done, afaik there are some scripts/extensions to hide results from certain sites, but you still can not uprank (can not uprank something that was not even sent to you)), and it hurts smaller websites, like articles or forums that are not big platforms or doing heavy seo.

    One should most definitely use more than 1 search engine. You can even find interest specific engines.

  • source
  • parent
  • [–] 3 points 1 year ago (3 children)

    Thank you for the post and insights. I’m a very happy paying Kagi user, for me it’s so worth it to never see ads and enjoy better privacy. The results are always great as far as I know (I’m not a pro at comparing them with Google but they seem just as good or better)

    I especially like the AI summaries (even though I’m a skeptic about over using AI generally) because many sites block my IP because of being in China and using weird VPNs and proxies, so the summaries let me know the info without having to open the page up.

  • source
  • hideshow 3 child comments
  • [–] [S] 1 point 1 year ago (2 children)

    In this particular case, have you tried checking out uncensored models, maybe trained primarily for west locally, maybe some mistral model. I am not saying western models would give you correct or better or unbiased answers, but if you use multiple sources, you can probably form some results yourself. you can possibly keep a combo of smaller quant of small models (8b should run reasonably if you have 16 gib memory, even 12-14 B models can work, and you can download 2-3 models from different providers)

    Side questions: Are you allowed to use huggingface in china? similarly, is all your lemmy life through vpns, or is lemmy small enough not large enough to yet be censored. Also, if you do not mind answering, how is the infamous firewall implemented? is it a blacklist banning certain stuff, or whitelist allowing only selected stuff, or some combinations (like for example, (and I am not accusing china of censoring wiki here, I just do not know enough, and just trying to form a hypothetical example) ban whole wikipedia, but whitelist only certain articles, or whitelist all of wiki, and ban only certain articles, or something like ban english wiki, then whitelist some stuff, but keep all chineese wiki whitelisted and then possibly block some? Also, how is vpn usage treated by government? is it discouraged, or encouraged, or if you use it a lot, and possibly mix some irl activity and vpn activity, is it pursued in some ways?

    Sorry for asking so many questions, but I was just curious. In any case, if you feel uncomfortable, do not answer any question. Thank you

  • source
  • parent
  • hideshow 2 child comments
  • [–] 3 points 1 year ago (1 child)

    It’s a blacklist censoring stuff, so if the domain you want to access is obscure and not mainstream you might get it. If it’s at all political in nature though, or an open discussion forum, or a news site it’s most probably blocked.

    Anyone who needs the global internet needs a VPN. The govt is so powerful with their filtering and knowledge they can block any vpn at any time, basically. You might be able to guarantee a way through if you have a very dialed in and obscure configuration, otherwise you’re just getting lucky on borrowed time.

    Only a small handful of commercial VPNs from the west work, and they are only ones that are very purposefully made with stealth and obfuscation modes. Those modes blend traffic and go to great lengths to hide the fact that it’s a VPN tunnel occurring. To punctuate that, off the shelf Wireguard or OpenVPN are immediately detected and blocked with no chance to get through at all.

    Also it varies by region, if you’re at a fancy hotel in Shanghai you might have an easier time if you’re in the countryside. Also you have an easier time on mobile networks.

    About Lemmy, it’s obscure enough I think you don’t need a vpn to browse it a little depending on the instance. But a lot of stuff in the view coming from global URLs dues get blocked, so the general experience is broken so need a VPN on.

    Generally, I need so much global internet that’s blocked all the time for work and stuff that I need multiple VPN solutions and provider accounts active at any time in case any goes down (this is very much the case recently, my usual ones not working and have to use backup ones) and am always testing out new ways. On average I have to spend 10% of my working time every just trying to get a good connection online.

  • source
  • parent
  • hideshow 1 child comment
  • [–] [S] 1 point 1 year ago

    Thanks for sharing. I have heard pretty crazy stuff, and this seems more believable.

    off the shelf Wireguard or OpenVPN are immediately detected and blocked with no chance to get through at all

    Is it possible for you guys to buy vps outside and somehow pay them (that would probably be the hard part) or maybe find some friends or diaspora outside, and have them pay for vps to set your personal vpns. You can make it somewhat obfuscated, and mostly hide it as some work related stuff.

    Do you guys use tor or i2p? I would presume you are "allowed" but still have to hide it.

    On average I have to spend 10% of my working time every just trying to get a good connection online.

    that is really sad, sorry to hear about that.

  • source
  • parent
  • [–] 4 points 1 year ago* (last edited 1 year ago) (5 children)

    First off. I hate ads. I do everything in my power to avoid seeing them.

    But for my search engine, I tolerate them. Why? Because 80% of this ad revenue profits is used to plant trees to combat global warming.

    I've been using Ecosia for quite a while now and as it uses Bing for most the algorithmic stuff, you won't notice a huge difference. But there is some custom indexing in there.

    Ecosia passes all kinds of integrity tests and certifications, so I do not question whether they are legit or not. I just feel my search efforts are best spent on Ecosia, who find a way to actually do some good in the world with something as heinous as ad revenue.

  • source
  • hideshow 5 child comments
  • [–] [S] 2 points 1 year ago (1 child)

    I am not gaoing to say anything on ecosia here, since i do not know much about them. All I can say is, most of the ad revenue is going towards bing api pricing, then their servers and other costs of doing a business (for example, rent, electricity, regulations, etc), then their employees, then if they have some amoun of oney remaining, they would possibly use it to plant it. Most people doing business do not immediately want to use their profits, and generally invests parts of profits back to build some runway (ad revenue is not consistent), invest in employees (insurances or incentives), or payback any loans. Even if they have some vc funding or recieving grants, they would have to profitable by themselves, and then only can they plant tress. And keeping this in mind, my guess would be that they are not planting that much

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 1 year ago

    I should've mentioned that of course, they cover costs first and then invest their profits in good causes. From an article by Utopia.org:

    "Ecosia donates 100% of its profits to climate action projects. Of this, a minimum of 80% goes to tree-planting projects. This focus on planting trees is for the benefit of the planet, animals and people in general."

    They surpassed 100 million in 2020 and according to their website they are well past 200 million at the moment. The 80% figure also comes from themselves and they have been certified after intensive scrutiny.

    Source

  • source
  • parent
  • [–] 2 points 1 year ago
    [–] 2 points 1 year ago* (last edited 1 year ago) (3 children)

    Boy, that was a great piece of work that I will refer to again in the future.

    But it did YaCy kinda dirty.
    If you're really in this search engine game, you will run your own proxy like whoogle or searxng.
    And if you're really_really into it, you will back it with YaCy, or a similar tool.

    I haven't gotten quite this far. Started with whoogle, switched to searxng, been at it less than a year.
    I think there's probably quite a bit I could still do with searxng, but its worked pretty darn well without tweaking.
    I might point it at some of the engines here on your post. Thanks!

    YaCy is on my project plan. I will do it. Check it out for yourself folks.
    https://github.com/yacy

  • source
  • hideshow 3 child comments
  • [–] [S] 2 points 1 year ago (2 children)

    I also did use yacy for some time, but problem is - index is too small. I could within weeks visit each and every page indexed by yacy, and that is not a good thing

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 1 year ago (1 child)

    Are you running your own crawlers?

  • source
  • parent
  • hideshow 1 child comment
  • [–] [S] 2 points 1 year ago

    no (ish), what I meant there was i can manually click through all links in yacy's index (a bit of a hyperbole). But as I had hinted, I have offline backup for a lot of pages (so it is not a index really, but it is searchable and i can just open pages). I have nearly 0.5 million articles. 90+% is wiki, then arch and programming books/docs, and hopefully more to come. And all of this requires less than 60 GiBs, and searching is really fast (my typing speed is the limiting factor)

  • source
  • parent
  • 14 comments in c/youshouldknow@lemmy.world view that post
    [–] 11 points 1 year ago

    I am currently using Gibiru (a Searxng frontend by Murena; also known for their /e/OS android operating system), but have been toying with the idea of running my own Searxng on my home server

  • source
  • [–] 9 points 1 year ago* (1 child)

    Very much a hit or miss, but I’ve enjoyed the marginalia search index.

    Otherwise I just use duckduckgo for simplicity’s sake. But when I want a non-commercial result marginalia is my go-to.

  • source
  • hideshow 1 child comment
  • [–] 5 points 1 year ago (1 child)

    You are still bringing the big providers traffic

    gonna be honest, this falls under the "no ethical consumption" bit for me. someone's gonna get my traffic and ad revenue, i might as well get the best results for it. And to be honest, the best results i've had have been with paid, industry specific search engines. It removes some of the emotional barrier i've got toward trying some of the paid services like Kagi, just i'm on a pretty tight budget. $5/mo for 300 searches (do your unused searches roll over to the next month?) means i have to cut back somewhere, and we have been doing that month after month after month. A search engine just is not worth eating less. (note i did not say eating out less).

    search engines seem to be going the way of streaming sites. if you know where-ish to look, you can put the search in there. but there's no good general, has everything except specialized knowledge search engine that i've found.

  • source
  • hideshow 1 child comment
  • [–] [S] 3 points 1 year ago

    fair point, you currently have to compromise on either "ethics" (a bit stronger word than I would like to use) or result quality. My only try is to promote independent engines in hope to improve them such that market gets more competitive

  • source
  • parent
  • [–] 4 points 1 year ago (1 child)
  • [–] 1 point 1 year ago (2 children)

    Ddg has terrible results. I switched from google to ff because of ubo. But I’m always open to something better.

  • source
  • hideshow 2 child comments
  • [–] 1 point 1 year ago (3 children)
  • [–] [S] 1 point 1 year ago (2 children)