How to get started on improving Site Search Relevancy

A series of blogs by Karen Renshaw on improving site search:

  1. How to get started on improving Site Search Relevancy
  2. A suggested approach to running a Site Search Tuning Workshop
  3. Auditing your site search performance
  4. Developing ongoing search tuning processes
  5. Measuring search relevance scores


You know your search experience isn’t working – your customers, your colleagues, your bosses are telling you – you know you need to fix it, fix something but where do you start?

Understanding and improving search relevancy can often feel like a never ending journey and it’s true – tuning search is not a one-off hit – it’s an iterative ongoing process that needs investment. But the resources, companies and tools needed to support you are available.

Here, I’ll take a quick look at how to get started on your search tuning journey. I’ll be following up in subsequent blog posts with more details of each step.

Getting Started

Like any project, to be successful you need to understand what you want to achieve. The best way is to kick off the process with a multi-functional Search Workshop.

Typically ran over 2 days, this workshop is designed to identify what to focus on and how. It becomes the key to developing ongoing search tuning processes and driving collaborative working across teams.

Workshop Agenda

Whilst the agenda can be adapted to be specific to your organisation, in the main there are 4 key stages to it:

  1. Audit
  2. Define
  3. Testing Approach
  4. Summary

1. Audit – Where are we are now?

Spend time understanding in depth what the issues are. There are many sources of information you can call on:

  • Web Analytics – How are queries performing today?
  • Customer Feedback – What are the key areas that your customers complain about?
  • Known Areas of Improvement – What’s already on your product backlog?
  • Competitive Review – Very important for eCommerce sites – how are your competitors responding to your customers queries?

2. Define – Where do we want to be?

As a team agree what the objectives for the project are:

  • What are the issues you want to address?
  • Are there specific types of search queries you want to focus on?
  • Is a overhaul of all search queries something you want to achieve?
  • What are the technical opportunities you haven’t yet exploited?

3. Testing Approach – What’s the plan of attack?

This is the time to plan out what changes you will make and what methodology for testing and deployment you are going to use.

  • What order should you make your configuration changes in?
  • Are there any constraints / limitations you need to plan around?
  • What resources do you need to support search configuration testing?
  • How are you going to measure and track your changes so you know they are successful?
  • Do you need to build in a communication plan for stakeholders?

4. Summary

Ensure that all actions are captured in a project plan with clear owners and timescales.

Workshop Attendees

Within an organisation multiple teams have responsibility for making search better, so at a minimum a subject matter expert from each team should attend.

Key attendees:

  • Business Owner
  • Search Developer
  • Content Owner
  • Web Analyst

Benefits of the workshop

There are practical and cultural benefits to approaching search in this way:

  • Collaborative working practices across the different disciplines are improved
  • Shared objectives and issues leads to better engagement and understanding of the approach
  • A test and learn approach can be developed with the time between testing iterations reduced
  • The workshop itself is an indicator to the wider business that search is now a key strategic priority and that it is getting the love and attention it needs

In my next blog I’ll cover how to run the workshop in more detail.

Karen Renshaw is an independent On Site Search consultant and an associate of Flax. Karen was previously Head of On Site Search at RS Components, the world’s largest electronic component distributor.

Flax can offer a range of consulting, training and support, provide tools for test-driven relevancy tuning and we also run Search Workshops. If you need advice or help please get in touch.

Leave a Reply

Your email address will not be published. Required fields are marked *