What are people actually using these hosted search providers for? For personal use I've found using searxng-ai-kit [1] (disclaimer: I'm the author) or similar tools to be far more cost effective (free!). oh-my-pi uses a similar method and will scrape search results for you out of the box, no api key or search provider needed.

[1]: https://github.com/nikvdp/searxng-ai-kit

Since SEO is big business, and agent driving search is probably the future, I suspect we might see the price of search API's going negative in the future.

Ie. Providers pay you to use their API over competitors, since then they get control of who has influence on the internet. They can probably get sites to pay to be up ranked for specific queries etc

@DeepSeek teams, why not train your next series of LLM models with source-aware training?

https://arxiv.org/abs/2404.01019

LLM's would be able to answer questions, delineating exactly which facts/assumptions/reasonings are prior work and from where (this URL, that book, etc.), and which parts are it's own deductions, or the user's.

These services only exist because Google refuses to provide a paid search API.

For comparison that’s $5 with Brave Search API and $12 with Kagi API

For comparison, Kagi charges $12 per 1000 searches (only talking about their API pricing, like what you’d use with an LLM; their $10/mo plan has unlimited searches but it’s a TOS violation to use those credentials with an LLM).

Meant as a comparison given the discrepant pricing points:

I pay the Exa premium for semantic retrieval with far higher SNR, plus the academic index [1].

A field like objective can be up to 4096 characters [2] versus ori_question which is limited to 128 here.

[1] https://exa.ai/blog/publications-search

[2] https://exa.ai/docs/reference/search

Artificial analysis' compilation of search api costs per 1000 queries, for comparison:

https://artificialanalysis.ai/agents/search-api?search-api-c...

How well does this (and similar services) deal with anti-bot protection? What happens if I attempt to retrieve a page protected by the cloudflare checkbox or by Anubis? An error, no useful content, or the actual content I want?

About a year ago I built my own search and fetch MCP tools for agents based on third-party scraping providers. It is about 15-20x more cost-effective than what OpenAI and Anthropic is offering with their integrated web search and fetch tools. Very impressive that DeepSeek has such a competitive offering today.

Ran a few quick tests on my web-fetch bench. As much as I like DeepSeek, its search and fetch aren't very good (is quite cheap tho). You'd probably be better off using free services like Jina directly, or if you're going paid, Parallel, Firecrawl, Tavily and Exa are all pretty good now.

DeepSeek Flash really came at a bad time before the Anthropic IPO. I use this thing daily and it’s neck and neck with SOTA models. The search is one of the things setting it apart.

Does anybody use https://www.anysearch.com/ ?

I wonder if providers do this so they can 'scrape' for 'free' because the customers pay for it? They all seem to have an abstraction around search rather than suggesting the use of native search apis.

they had this for a while now under the Anthropic API

I "fetched" a website with hyperlinks and it only returned the text, which makes it a bit irrelevant?

How does the quality compare to SerpApi?

I started to think differently about web search for agents, instead of traditional index, I would prefer to have an agentic WALKER that literally walks through pages and finds out whats needed, even if seeded with some initial web search.

Can someone explain why fetch costs money. I thought it was your locally-running harness doing it. How can they charge you for that?

This is nice. Does search include their LLM data, or does it do real-time search like Perplexity?

We are building our own in-house search API/MCP too, with anti-bot and CAPTCHA workarounds, and we plan to launch Monday morning.

Is there any way to test this without an API key?

I am curious if my large website is in their search results. My company charges $1 per page crawled of our website, and the only traffic that worms through our firewall is using hacked residential ISPs. I really need to know who to send the (large!) bill to.

I'm having trouble finding what this actually is

They're entering the search engine scene since they already have "all of the internet" stored due to the expansive LLM crawling -> training?

I use to manage Google search appliances. This feels a lot like Google old search API product. Nice to see some new players here.

As reference point exa charges 4 bucks

Deepseeks browser platform has replaced my typical LLM search routines over the last few months.

Its extraordinary good. They arent trying to sell you things and I cab often find obscure sources/papers.

Its also great for finding research papers in chinese on a given topic.(also its free in the browser and doesn't slow my comptuer to a crawl like most webUIs by llm companies). I have it open all day nowadays.

Edit: How does this get downvoted immediately. Does Anthropic have bots just downvoting any positive sentiment about Chinese AI..

Is AI search where the mysterious relentless crawler traffic is coming from lately?

I wonder if this is censored. If not then users in China mainland can use the api to workaround gfw (though there are much better ways)

Which ones of the search providers listed in this thread use ethical indexing and not outright DDoS?

as expected, the typical "politically sensitive" search queries return 0 results

egress on fetch looks to be huawei cloud in hong kong

fetch tool uses a linux firefox 149 user agent with lang set to china. it runs js. could be using something like camoufox.

The rate-limit given there seems really low. 5 requests per second?

I wouldn’t be able to use that API in any app with any significant number of concurrent users.

Or am I reading it wrong?

DeepSeek models are amazing and this search pricing is fantastic. All these companies like Parallel, Exa and others are gonna have hard time justifying their hefty valuations. Add Perplexity and many others into the mix. The demand for what they are building is there, but not the margins for the amount of money they raised.

I was really excited to use this (well, I still am), but then I realized Ollama Cloud offer this too, though they don't really mention pricing. I think it's somehow rolled into their subscription.

[flagged]

[flagged]

[dead]

[flagged]

[flagged]

[flagged]

[dead]

[dead]

These prices will skyrocket eventually. AI companies need more money.

RAM prices will also probably stay very high as long as AI companis act in such a parasitic manner.

Will DeepSeek pay the content creators whom pages were fetched?