We’re often asked to review how our clients use search technology, both open and closed source (although we specialise in the former, we’ve also encountered most of the commercial search engines over the last 16 years). One common mistake is to assume that search is purely a technical problem, and that all issues can be resolved by writing software and changing configurations, or worse by throwing away the old engine and replacing it with a new one at huge expense. More
I've recently been asked to join the Industrial Advisory Board for the School of Computer Science and Electronic Engineering at the University of Essex and will be talking to students there on Monday 5th February, repeating a similar talk I did last year. The subject is 'Working in Open Source Search' and I'll describe how we founded Flax back in 2001, how we've built, tuned and implemented open source search engines and some of the client projects we've worked on. It's been a fascinating journe...Continue reading
The news today that Elastic (the company behind the open source Elasticsearch software) has acquired Swiftype will have surprised a few people, even though Elastic has already acquired a good number of other companies. Swiftype have a couple of products that deliver cloud-based site and enterprise search and under the hood, all of this is built on Elasticsearch. Swiftype are part of a new ...Continue reading
A few weeks ago I sat in on a workshop in London at the Taxonomy Boot Camp conference, run by Jeff Fried of BA Insight. I've known Jeff for many years from various events and we share some views on how search systems should be built and managed - using best-of-breed technology and effective management processes.
He was kind enough to ask me to join a recent podcast. During the podcast, we had a great conversation about open source search, enterprise...Continue reading
We've known about Apache Kafka for several years now - we first encountered it when we developed a prototype streaming Boolean search engine for media monitoring with our own library Luwak. Kafka is a distributed streaming platform with some simple but powerful concepts - everything it deals with is a stream ...Continue reading
We've spent a lot of time working with clients who recognise that their search engine isn't delivering relevant results to users. Often this is seen as solely a technical problem, which can be resolved simply by changing query parameters or the search engine configuration - but technical teams need clear direction on why a result should or should not appear at a certain position, not just request for general relevance improvements.
It's thus important to consider relevance as...Continue reading
For the last year or so I've been working with Professor Udo Kruschwitz of the University of Essex on a long-form journal article on enterprise search - although at 156 pages this is more of a book than a journal. Released as part of the Foundations and Trends® in Information Retrieval series by Now Publishing, the b...Continue reading
Last night the estimable Martin White, intranet and enterprise search expert and author of many books on the subject, flagged up two surprising articles from Forrester who have declared that Cognitive Search (we'll define this using their own terms in...Continue reading
We're sometimes asked by clients to examine not just their technical implementation of search, but also the wider picture: how search functionality is exposed to users, how it compares to competitors' websites and best practice. This process usually takes us ten days to two weeks and results in a highly detailed report with clear recommendations for improvement.
This process is slightly different each time, but usually includes the following steps, shown with some examples of questions we mig...Continue reading
Here at Flax, we're often called in to take a look at existing Apache Solr or Elasticsearch search applications, to suggest improvements, tune-ups or enhancements. It's impossible for us to know ahead of time what we might find - out-of-date versions of the software, slow performance on either (or both) the indexing or search side of the application ...Continue reading