XJoin for Solr, part 1: filtering using price discount data

In this blog post I want to introduce you to a new Apache Solr plugin component called XJoin. I'll show how we can use this to solve a common problem in e-commerce - how to use price discount data, provided by an external web API, to either filter the results of a product search or boost scores. A further post will show another example, using click-through data to influence the score of subsequent searches.

What is XJoin?

...Continue reading

Elasticsearch vs. Solr: performance improvements

I had been planning not to continue with these posts, but after Matt Weber pointed out the github pull requests (which to my embarrassment I'd not even noticed) he'd made to address some methodological flaws, another attempt was the least I could do. For Solr there was a slight reduction in mean search time, from 39ms (for my original, suboptimal query structure) to 34ms and median search time from 27ms to 25ms - see figure 1. Elasticsearch, on the ...Continue reading

Elasticsearch vs. Solr performance: round 2

About a year ago we carried out some performance comparison tests of Solr (version 4.10) and Elasticsearch (version 1.2) and presented our results at search meetups. Our conclusion was that there was not a great deal of difference. Both search engines had more than adequate performance for the vast majority of applications, although Solr performed rather better with complex filter quer...Continue reading

Luwak 1.3.0 released

The latest version of Luwak, our open-source streaming query engine, has been released on the Sonatype Nexus repository and will be making its way to Maven Central in the next few hours.  Here's a summary of the new features and improvements we've made: Batch processing I...Continue reading

Elasticsearch Percolator & Luwak: a performance comparison of streamed search implementations

Most search applications work by indexing a relatively stable collection of documents and then allowing users to perform ad-hoc searches to retrieve relevant documents. However, in some cases it is useful to turn this model on its head, and match individual documents against a collection of saved queries. I shall refer to this model as "streamed search". One example of streamed search is in media monitoring. The monitoring agency's ...Continue reading

Free file filters, search & taxonomy tools from our old Googlecode repository

Google's GoogleCode service is closing down, in case you hadn't heard, and I've just started the process of moving everything over to our Github account. This prompted me to take a look at what's there and there's a surprising amount of open source code I'd forgotten about. So, here's a quick rundown of the useful tools, examples...Continue reading

Comparing Solr and Elasticsearch – here's the code we used

A couple of weeks ago we presented the initial results of a performance study between Apache Solr and Elasticsearch, carried out by my colleague Tom Mortimer. Over the last few years we've tested both engines for client projects and noticed some significant performance differences, which we thought deserved fuller investigation. ...Continue reading