We’ve been working recently with a couple of clients who sell significant volumes of products via their websites: in both cases the current search solution is underperforming. In this situation it’s far easier to justify an investment in better search: if customers can’t find products they simply won’t be buying them. Simple changes to the search algorithms can have huge impacts, costing or saving the company millions in revenue. However, installing a new search engine is just the beginning – it’s vital to consider a solid test strategy. To start with, the new engine should be at least as good as the old one in terms of relevance. The search logs will show what terms your users have typed into the search box, and these can be used to construct a set of queries that can be tested against both engines, with the results being scored by content experts, beta testers and even small groups of real customers. The results of this scoring can be used to inform how the new engine can be tuned – and of course, this should be an ongoing process, as search should never be built as a ‘fire and forget’ project.

There’s sadly little available on the subject of real-world relevance testing, although there’s a forthcoming book by Doug Turnbull and and John Berryman. Based on our work with the clients above, we hope later this year to be able to talk further about relevance testing and tuning – and how to do it right for e-commerce, avoiding significant financial risk.

UPDATE: Doug has been kind enough to give our readers a discount code for his book – “39turnbull” – not sure how long this will last but it gives you 39% off the price so worth a try!

The post When bad search hurts: finding that elusive ROI appeared first on Flax.