London Lucene Solr Meetup – Enterprising attitudes to open source search & query completion strategies

Last night the London Lucene Solr Meetup was hosted by Elsevier in their Finsbury Square offices. Our first speaker was Martin White, expert consultant, author of many books about enterprise search and intranets and visiting professor at the University of Sheffield (oh, and Flax partner). Martin showed us some scary numbers about the terribly low level of satisfaction with enterprise search, drawing on research from AIIM and Findwise (I highly recommend you contribute to their ongoing survey if you can, it’s a great resource). An example is that around 55% of people in enterprises find it ‘very difficult’ to find information which can have a huge effect on productivity. Martin suggested that there is a huge opportunity for open source search in the enterprise market, but that we need a way of communicating the benefits to non-technical staff – as these people are generally the ones in charge of budgets. He ended with a suggestion that a trade association for smaller, independent search companies could be formed, an idea I’m going to further explore.

After a short break we continued with Tomasz Sobczak of Findwise (who had travelled from Poland especially to speak) on query completion strategies – you’ll have seen this feature where a search system suggests endings for the query you’ve begun to type. He described the various applications of this (including completing place names in map searches and available products in e-commerce) and described the many ways it can be implemented in Solr: facet.prefix, facet.contains, using N-grams, Shingles, the Suggester component, queries using synonyms and the Terms component. He noted the various pros and cons of each approach including how they may affect performance and suggested how a separate Solr index might be used purely for query completion. Data for query completion should also be clean and secure (you don’t want to show something the user isn’t allowed to know exists via query completion!). He finished with an example from Findwise’s work for Ericsson.

After the talks we had a brief discussion around how some of the less exciting features of Solr might be improved (we’ve blogged about our search for sponsorship for fixing some of these issues) and the suggestion arose that we might run some more Solr hackdays, in London or even the U.S.A. We’ll be looking into this possibility.

Thanks to our hosts, speakers and indeed everyone who came – see you next time!

Leave a Reply

Your email address will not be published. Required fields are marked *