The Open Source Search Specialists
My colleague Richard Boulton will be presenting at Europython in Birmingham, U.K. next week, specifically at 15.30 on Tuesday 30th June – an abstract is available. He’ll be talking about Xapian, Xappy and Flax, and showing examples of these in action including one using a Django integration layer.
Update: you can now download the slides for Richard’s talk in OpenOffice format.
The Flax team are pleased to announce the alpha release of Flax Search Service (FSS). FSS combines powerful, high-level indexing and search features with a well-designed Web Services interface. FSS is Open Source software (under the MIT licence) and is available as a free download from Google Code.
Web Services and Service Oriented Architectures (SOA) have become increasingly popular in recent years due to their many advantages. FSS provides a RESTful interface in which databases, documents, and searches are represented as resources identified by URLs. For example, to add a document to a database,the document data is POSTed to the database resource. To search for a word or phrase,the client sends the query as a GET request to the database, which responds with a list of matching documents. Indexing transactions may be handled automatically or explicitly by the client.
For convenience, client libraries are being developed in several languages, including PHP, Python, Java and JavaScript. It would be a simple matter to interface to FSS in any language with support for Web protocols. The FSS distribution also includes example code to get you started, and basic documentation.
FSS alpha supports enough indexing and search functionality to implement basic but useful information retrieval systems. Over the next few months we will be adding support for advanced features like facets and tags, geolocation and image search. It will run on any system with support for Xapian and Python (Windows, Linux and Mac amongst others).
Microsoft have announced a roadmap for their enterprise search products: none of this is very surprising. How successful they’ll be at integrating the FAST technology (which comes from a Linux background) with Sharepoint, .NET etc. remains to be seen. More coverage here.
They’ve also released an ‘Express’ (i.e., free but feature limited) version of Microsoft Search Server. We’re going to take a deeper look at this soon.
I’ve spent some time recently trying to find where people gather and discuss different search engine technologies and approaches. There is a Yahoo group which seems friendly and full of useful content, and a group on LinkedIn, a business networking site. Stephen Arnold’s blog is also a mine of information, with profiles of vendors and some very interesting comments on particular technologies. I’ve also found some more blogs which I’ve added to the blogroll on the right.
As we continue to develop Flax, it’s very interesting to hear about customers and developers’ experience with other engines. If you know of any other places to look please let me know!
In concert with our new Flax website, we’ve decided to start blogging about development of Xapian and Flax, search technology in general, interesting open source projects and indeed anything else we can think of.
In this first post, I’ll try to explain a little about the motivation behind the Flax project. Here at Lemur Consulting we’ve worked with search engines for decades, starting with Muscat, then building a half-billion-page search for the Webtop project, to working with technologies such as Autonomy IDOL, Ultraseek and Lucene. We know a lot about the features customers need from search tools and how to build them. However, we’re also committed open source enthusiasts – and very few enterprise search engines are open source.
So, we feel the time is right for a complete open source enterprise search product. We’ve called this product Flax, and based it on the Xapian core (because we’re also heavily involved in Xapian, having helped develop it to support the aforementioned Webtop project). Flax is a combination of Xapian, various other programs we’ve developed over the years from spiders to indexers to content extractors, other complementary programs and our combined years of experience in the sector.
If you want to try Flax, right now, you can download Flax Basic, a free search tool for Windows. You could also see Flax in action searching millions of interior design items at mydeco or searching tens of millions of UK newspaper stories at NLA Clipsearch. If you want to know more about Flax and how it could help you build a powerful search tool, contact us.
We intend to develop Flax to rival or even surpass commercially available closed source search engines. Even at this early stage, the examples above prove this is a solid, scalable platform with a great future. It’s an exciting project and we’re glad to be able to share our story with you.