Last week we heard from various sources that Microsoft had announced they would only be continuing to develop its recently acquired FAST Search technology on Windows. This had long been feared by some in the sector, and it must be worrying for existing customers.
Platform choice can be a key issue for those looking to implement advanced search, as there may be significant existing in-house expertise and investment in a particular platform. Our Flax solution works just as well on Windows, Linux or Solaris. It’s sad to see such a powerful technology as FAST become so narrow in focus, but it’s not particularly surprising after the Microsoft acquisition.
UPDATE: more coverage on this from The Register
A new year, and a chance to think about what might happen in the world of enterprise search over the next twelve months. I’ll make a stab at some predictions:
- Price cuts – possibly driven by even harsher competition between Google and Microsoft FAST, I can see prices coming down for packaged enterprise search. Autonomy will probably raise theirs
- Real time search matures – not just Twitter or Facebook, but real time data from many sources being part of enterprise search results
- More geolocation-aware search – in the U.K. at least, we’re seeing signs that the source data is finally being freed up, which should make it a lot simpler and cheaper to build location-aware solutions
- A few less second-tier players in the market – it’s still difficult out there, I’m afraid not every company will survive the next year.
You’re welcome to take any of these with a generous pinch of salt!
Microsoft have been asking open source companies not to compete on cost, but rather on value, according to ZDNet. Unfortunately the response to this hasn’t exactly been positive, as CNET reports. I doubt many open source vendors will be taking much notice of what Microsoft would like them to do, and suspect they will happily continue to make the point that if customers are looking at buying software & services, taking the cost of software completely out of the equation is almost certain to save them money.
Some more resources for those looking for open source search components: Search Components Online has some great lists, including an exhaustive list of filters based on an article by New Idea Engineering called Where Have All The Filters Gone? Filters in this case are defined as components that extract plain text and metadata from file formats, i.e. Adobe PDF or Microsoft Word.
Another more general list of search engines and technologies can be found at SearchTools.com, although parts of this are a little out of date.
One of the challenges we often come up against is indexing data held in other proprietary or open source systems, such as databases or content management systems. Talend is an open source data integration platform that lets you connect to a huge variety of these systems, from Salesforce to Oracle to SugarCRM. Talend is an offshoot of the Eclipse open source community. We’ll be following the development of Talend with interest.
There’s also the related problem of translating file formats before indexing them. Luckily there are lots of open source converters (as used by Omega, part of Xapian), or if you run on a Microsoft platform there’s IFilters – the latter aren’t open source, but you can easily connect to them from another program using COM. In our experience, the IFilters are better at extracting content from Microsoft-specific formats .
UPDATE: I’ve also recently discovered the Tika project, under the Apache umbrella. Not a lot of formats supported so far, but it’s a start.
Microsoft have announced a roadmap for their enterprise search products: none of this is very surprising. How successful they’ll be at integrating the FAST technology (which comes from a Linux background) with Sharepoint, .NET etc. remains to be seen. More coverage here.
They’ve also released an ‘Express’ (i.e., free but feature limited) version of Microsoft Search Server. We’re going to take a deeper look at this soon.
