It is becoming near impossible to find relevant information from search engines. Duckduckgo, SearXNG, Bing, Google, and so many more mainstream engines have a significantly high noise to signal ratio, and it is getting worse.
Here are a collection of the best search engines I know, please add more to the list.
- Forum Search Engine: https://crowdview.ai/
- Non-commercial Search: https://search.marginalia.nu/
- Libre Meta Search Engine: https://librey.devol.it/
- Golden Age Search Engine: https://www.wiby.org/
- Yandex: https://yandex.com/
If no more high quality search engines exist, would it be possible to host your own?
EDIT: Some new discoveries. The addon uBlacklist and filters can block super SEO sites from appearing in search.
deleted by creator
I like the french one “qwant”.
deleted by creator
deleted by creator
Why pay? I think the answer is pretty easy. If one doesn’t want to self host. Running any kind of web based service costs the person running it money. Google obviously makes money off of a user doing searches via adds and data collection. I would actually have no issue paying for certain Google services if it meant that as a paying customer they would not double dip and try and profit off the data they are collecting on me.
This is all coming from a person who has a server rack in my basement and multiple PCs scattered throughout my home, so I am no stranger to self hosting.
I would actually have no issue paying for certain Google services
Excuse me sir, do you have a moment to talk about our Lord and Savior, Richard Stallman?
People have spare pc’s or laptops they can sacrifice as a SearxNG instance. I too have a homelab & i have not paid a single penny to have a hobby SearxNG instance on my old hackbook pro that I share with friends.
I feel like, the internet is more than just paying your way for services. It’s about creating and sharing services and decentralization. If Kagi takes off then we have another evil Google that wants to profit at the cost of users, but its name is Kagi. It might be an impractical answer, and it does take some work to create a better internet. End of philosophical rant about the true spirit of the internet
I find all the kagi mentions to be very suspicious
I’ve mentioned it a few times because it’s actually good.
If I had stock/investments in a search engine, you better fucking believe Id also have a bunch of bots crawling for the terms “What is the best search engine” and immediately hijack the convo with bots upvoting my search engine.
I cannot explain how easy it is to do this.
Your comment belongs higher. When given the opportunity to make money by social media advertising sometimes in the thousands or millions, companies, share holders, and conflict of interest groups take it. Cablemod’s burning adapters, cryptocurrency scams, payed positive youtube reviews are some great examples. In general there is no honor system and its best to assume anything that can be abused will.
Also manipulating votes is incredibly effective towards swaying public opinion due to the bandwagon effect. Spend a days worth of effort making fake accounts and downvoting any opinion you see as undesirable and most people will follow suit. This is especially bad in echo chambers like on twitter, reddit, etc.
I wish a broader audience could be aware of this. The best I can do is try to spread the word.
Sources:
- The Problem with Linus Tech Tips: Accuracy, Ethics, & Responsibility: https://www.youtube.com/watch?v=FGW3TPytTjc
- Cablemod Recall: https://old.reddit.com/r/cablemod/comments/18o7bnv/planned_voluntary_safety_recall_of_cablemod/
- FTX Scandal: https://en.wikipedia.org/wiki/Bankruptcy_of_FTX
- Bandwagon Effect: https://en.wikipedia.org/wiki/Bandwagon_effect
- Echo Chamber: https://en.wikipedia.org/wiki/Echo_chamber_(media)
I always fear it comes across that way when I recommend it to people here. I’m just a very happy user and want to see them succeed.
I find it expensive for what it is (given that I still get a limited number of searches) and I’m not comfortable with some of their ways (I don’t want anything to do with AI, and I the idea they have of being nonpolitical seems dangerously naive to me). I also don’t like supporting non-FOSS projects all that much.
Still, it’s the best search I’ve found, and I’m paying every month until I find something better. It’s worth it.
Why? It’s a great search engine that a lot of people find extremely useful.
It’s not, really, I switched from Google some years ago and had accepted my faith with DuckDuckGo, but then tried out Kagi. I use search so much daily for work, the relief of getting quality results again is immense and probably saves me hours per week. I get much better results from Kagi than I got at the end from Google, and I can tune them to my liking:
- block Pinterest results when I search for images,
- downprioritize shopping results,
- rewrite all Reddit links to go to old.reddit.com,
- unamp google AMP links
- summarize long texts / documents
- quick answer from the top 5 results
…and so on and so on. It’s just so effective.
You can redirect old reddit with the tampermonkey extension
Yeah there are some good solutions to achieve this, I’m a big fan of the libredirect project, but currently I’m just setting up the redirects I want directly in Kagi so any URLs in the search results are already rewritten to my liking.
… suspicious
same. specially considering how privacy invasive kagi is.
How is it privacy invasive? For example, compared to competition like Google?
Not op, nor i have any experience with kagi, but i suspect there is no way to do an Anonymous search with kagi.
copypasting the other comment I made in this thread:
and am I supposed to believe such a bold claim? the only reason they give is “trust me, bro. I pinky promise I’m not logging anything”.
You have one account, every search query you make is associated with that account. And even if they aren’t selling that ultra sensitive data, I’m sure they are keeping logs to prevent abuse and fix bugs which could be used when a third party gains access to their servers (malicious actors, law enforcement, etc).
And that’s assuming that Kagi is not mining and or selling any data themselves, which is a bold assumption given how little we know about their proprietary product. If at least they published the source code, but no. I’m supposed to trust a proprietary black box which could potentially be linking every search query back to me.
I don’t have any skin in this game. I just wanted to point out that you went from “given how privacy invasive this particular entity is”
To
“… assuming… how little we know… could potentially”
That’s a pretty big leap from a bold and confident assertion that an entity is doing something all the way to saying that entity maybe could be doing something but we don’t know. It’s just a weird logical leap to me, and I felt compelled to mention it.
How is it privacy invasive?
I know nothing about Kagi, except that it requires an account and is paid. So I assume all searches are tied to that account and there is no way to do an anonymous search.Disregard I’m a fool
We care about data protection: We will be good stewards of any personal information you share with us. We do not log or associate searches with an account. More at our privacy policy.
Literally the first paragraph on their website
and am I supposed to believe such a bold claim? the only reason they give is “trust me, bro. I pinky promise I’m not logging anything”.
You have one account, every search query you make is associated with that account. And even if they aren’t selling that ultra sensitive data, I’m sure they are keeping logs to prevent abuse and fix bugs which could be used when a third party gains access to their servers (malicious actors, law enforcement, etc).
And that’s assuming that Kagi is not mining and or selling any data themselves, which is a bold assumption given how little we know about their proprietary product. If at least they published the source code, but no. I’m supposed to trust a proprietary black box which could potentially be linking every search query back to me.
I don’t trust or ever plan on getting Kagi, but in their defense, the “trust me bro” is a large portion of privacy services. I use Mullvad VPN and think they have a great reputation that have proved themselves. I have no however, personally checked the servers to verify myself what’s running, so I am trusting then. Even when running open source software, I know none of us here have actually looked into every line of code of our browsers or our phones to see what’s all running. It’s simply unfeasible, so trust and reputation is still required at the end of the day.
That’s absolutely true. The problem is that, to make use of VPN services, it’s required to have an account or other identifier.
But that’s no true for search engines. If I wanted to, I could make completely anonymous searches using SearXNG or DDG from different IPs and they would not have any way to correlate the search queries.
That’s not true with Kagi and it’s a completely unnecessary privacy risk you’re taking when using it.
I don’t know if I believe that. It’s a paid service, so the only way to enforce that unpaid users cannot search is to take a search request and check if it is coming from your account. Same with basic things like rate limiting requests. You literally need to associate your requests to an account to make basic functionality like this work.
If they do this but just don’t log it, then that means there is no way for their devs to ever debug issues users have or to monitor their services. I’m highly skeptical.
Also, “trust us” is something I’ve heard too many times.
They don’t need to tie the searches to an account. They log them anonymously. From their privacy policy:
Absent from our logs are any identifying information about your client. As such, any query or traffic logging that we do cannot be tied back to your account, ensuring that Kagi developers are the only people that the logs will ever be useful to.
It’s a paid service, so the only way to enforce that unpaid users cannot search is to take a search request and check if it is coming from your account
That’s not the same as logging.
You literally need to associate your requests to an account to make basic functionality like this work.
They just need to check the session of the user on the fly during the search operation. Once the search is done they don’t need to persist any record linking the search and the user.
Thanks, edited the comment!
The Patriot Act and Snowden’s leaks have shown companies will go against their privacy policy to appease governments. Search engines especially are targeted by five eyes with the PRISM program where copies of all your data, linked to your payment, are sent to Five Eyes and stored. Gag orders and legal threats prevent disclosure, as has been done with prior tech companies who have tried to push back against this.
Be wary of trusting corporations with your data as monetization is a powerful incentive.
Lemmy is different though right?… right?
With lemmy you are trusting whatever instance your account is on, and really any federated instances since they could choose to hold onto your posts and comments
I’m glad I’m finally not the only one feeling this way. I’ve been seeing them aggressively pushed seemingly out of the blue for months now. Especially for what is such an awful deal and zero evidence to their claims, just citing the marketing page at face value.
Those AI ones like Perplexity.ai, Kagi FastGPT and even Copilot (Bing Chat) give good results not just in the responses, but also in the links they return.
I’ve been slowly pivoting toward Perplexity.AI as the search engine. It basically does what I do, search + find the resources and summarize it but it is automatic with Perplexity.AI.
I rather pay them 20$ because that is saving me time a lot (and time is money in my case). Kagi’s search is okay but I can get nearly the same by using ublocklist on Bing or DDG for my use case.
This site collects quite a few web search engines: https://www.searchenginemap.com/
Brave Search works really well for me, and they have AI responses at the top of the page now that are really good.
Thanks! Never given brave a try before
Not sure why you got downvoted like crazy, but Brave legitimately has a great search engine. It also uses its own crawlers unlike DDG, Startpage or Kagi who are really just meta search engines piggybacking off other companies’ results.
Good thing I wasn’t replying to the down voters. OP got my response and that is what I wanted. 🙂
deleted by creator
Ddg cannot filter out results. If you don’t want pages containing the term, term you add -term to your search and those results should not be included.
Ddg doesn’t do this. I did a brief test of many search engines, and only google and mojeek filtered results correctly.
Edit: yadex seems to have working filtering.
Google doesn’t do that properly anymore, either, much like the “literal search” (quotes) - used to work, now it’s a crap shoot.
DDG is ok for most searches, but they have definitely hit a plateau. Programming search results are quite poor, for instance.
I’ve started paying for kagi. Their results are just way better at this point.
It’s not as bad as Google yet, but I find myself getting terrible or no results quite a few times.
Ex: if I’m looking for a niche blog post from example.com, just entering the keywords doesn’t return the right result, if anything at all. I have to add “site:example.com” and the right link shows up on top.
It’s kinda amusing when this happens, but I keep using ddg anyway because bing and Google had the same issue for the same keywords when I ran into the issue.
It is as useless. After all, it’s just Bing. But if the results are good enough for you, then why bother finding something else.
Some results come from Bing.
most are. just do a side by side comparison. for most queries it’s literally the same results.
Hmm, I did notice a sudden severe drop in quality recently. Perhaps they are A/B testing something.
ddg has been going downhill for awhile now. they changed something significant a couple years back that just made results, especially after the first half-page, absolute shit.
I think it was because they dropped Yandex results.
I don’t live in the US and the regional results for my country are worse on DuckDuckGo than on Google.
But still, I noticed Google’s quality drastically falling down in recent years.
Here are some options I use in my rotation.
Brave Search (skip the browser)
Mojeek
Qwant (French)
Yandex (Russian)
Mullvad Leta (Mullvad VPN subscription required)
MetaGer (German meta search)
Startpage (Private Google results)
DuckDuckGo (Private Bing results)
SearXNG and similar self hosted options are awesome, but I’ve found them unreliable.
Be skeptical of Kagi… It’s promoted pretty heavily around here for something that’s not FOSS.
THANK YOU for mentioning startpage! I have not heard of them but I haven’t liked using DDG very much so I’ve begrudgingly using google in private mode (and then I always forget to switch back and forth)
Took wayyy to long to find something not recommending Kagi.
I’m convinced at this point that Kagi is astroturfing on Lemmy, it’s unnatural. All they do is just cite the marketing page when questioned about the quality, or as to why they supposedly think it’s worth paying for compared to much more mature as-driven services that do respect your privacy.
I think you meant to write ass-driven
I’ve been using Kagi for 3 months now I think? And I’ve found it to be extremely useful for research…to a point, my only real issue is that after the first or so series of results it either doesn’t offer anything further or just no longer is showing relevant results to my search. Honestly though I’ve been very happy with Kagi and continue to pay the 10/month but if I do find something even better I’d happily switch. I’m not married to any particular search engine (well …least not to the extent of using Google services).
If you’re skeptical just give it a try I think they have a trial system in place.
Is “super SEO sites” a catch all term for those 99% filler websites that have a tomato soup recipe (in theory) but actually start out with, “Historical evidence seems to suggest that the tomato was first cultivated in the territory that would eventually become Guam back in 1464…”
I’ve wondered if we had a common reference term for those? I wish it didn’t have a positive connotation though…
Those plus sites that are lists of buzz words that have nothing to do with the actual site.
Technically the best one was Altavista.
But they are long gone because they came from the old academic & idealistic internet and they never learned to survive in that internet where money rules.
From a very old memory, Google blew AltaVista out of the water no?
I mean we all switched for a reason and it wasn’t the cute logo.
I believe yahoo killed altvista before google came and finished it off
Altavista was ahead of their time. The modern internet desperately needs a technical search engine.
Yandex was way better for searches in russian sources, but it came to shit just like google and also excludes whatever russian government don’t like at that point. I searched for some software in it multiple timea and the first link was some noname, probably malware site. It also promotes it’s own malware like browser with questionable russian security sertificates and their own Alexa. I’d honestly not include it in any list.
I like DDG and don’t switch from it that much. I’ve also heard Kagi as paid search engine is good, but I’ve never tried it.
For everyone who uses searxng, is it great for day to day browsing? Do I require to host my own instance or the setup is as easy as requiring to add “searxng” option on my browser app?
I’m interested to move away from google as it becomes shitty everyday and loses its effectiveness for advanced query (based on my own result compared during 2013 up to pre covid). Bing have weird result on my region so cannot use it, ddg only for occasional use.
Thanks!
pick a host from searx.space
Might be a good idea to bookmark this too because sometimes some hosts go down.
Appreciate it, will read more in documentations 👍🏽
Kagi is the highest quality for sure. There isn’t a better one, I have tried all of them except perplexity, and that’s more like chat gpt rather than search.
I mostly use bing now. I like that it can answer complex queries and provides sources.
Don’t have much trouble not finding things.
Its nice when you are deep in Microsoft already from your company and get BingChat Enterprise included anyways.
Its slower than OpenAI GPT4 at times and its alot more restricted, but it gets the job done mostly.
You need to hack the UI to make it nice, unlock longer inputs, disable search tool at-will, disable synthetic streaming responses (consistent token speed, but takes longer overall)Simple query via DDG, complex stuff and ChatBot stuff via BingChat Enterprise.
I thought the AI assistant was free? I just downloaded the bang app and I just use the AI assistant whenever I search for anything now.
I don’t use any sort of enterprise, or full disclosure know what any of the other things you said are, any of those options or parameters.
But bing’s really impressed me lately, haha, so I still use it.
Actually on my desktop, I just use the regular Bing search, which I do now find superior to Google for just finding quick accurate answers for basic stuff.
It’s still crazy to me a year later after switching all my default search engines to Bing after using Google for so long, but Google search is just such trash now, it’s insane.
It is free, but the Enterprise version doesnt store anything (not even usage statitistics) and runs on separate systems (allegedly) aswell as having no limits.
Oh, I see, yeah that does sound way better
