Findxbot – a real search engine bot

We received many questions in our last AMA on reddit and from webmasters about Findx, about how we build the Findx search engine index, and how our crawler, Findxbot, works. We hope this post can answer some of those questions!

Read more

Why are some sites missing from findx?

Because we are an independent search engine and not a metasearch engine, our spider is continuously crawling the web, adding pages to the findx index. And because we are a European search engine, European and English-language sites are a priority for findx as it is growing and improving its index. But not all websites allow every … Continue reading Why are some sites missing from findx?

Read more

How does internet search work?

Everyone uses a search engine and expects good quality and relevant search results to match the terms they were looking for. But very few people know how a search engine finds what you are looking for. When you do a search, you aren’t actually searching the internet – you are looking for results in the search engine’s … Continue reading How does internet search work?

Read more

robots.txt – subtle challenges

robots.txt controls which content web crawlers may reach, but the lack of proper specifications has lead to some interesting corner-cases.

Read more