Search In 2007

Today, everyone knows how to use Google or a search capability. It is very intuitive and even my son who is in first grade has figured out how to use the search feature in online dictionaries like Merriam-Webster. What surprises me recenlty is the how the enterprise search market is becoming a very competitive space.

A few years ago, Google and Yahoo! were the two enterprise level products on the market with an low cost to entry. This is a critical point in the modern IT environment. Today, there are numerous open source and proprietary solutions all claiming some type of advantage in their implementation of search technology. I can name quite a few search technologies today that I have reviewed Google Search Appliance, Yahoo SDK, Amazon A9 OpenSearch, IBM OmniFind, Corpora Find!, Microsoft Sharepoint 2007 Enterprise Search, Nutch, Lucene, Autonomy/Verity, and whole lot of embedded search engines. In the Java world, Lucene appears to be the best of breed search framework. As a matter of fact it is the basis for the IBM OmniFind Yahoo! Edition search technology.

I really like the Google appliance mentality. You just install it and use it. It just works kind of like your refrigerator. When was the last time you had to do any type of maintenance, patches, reconfiguration, optimization, of your refrigerator. You do not! That is the point. You just use it, it works and you keep it clean and it looks good. That in summation is what I consider the baseline needs and wants for a search solution. No one else has anything like it on the market yet.

I have designed and built solutions using Lucene, Verity, Google API and Yahoo! SDK. Generally speaking, they all work in a similar fashion. Personally, I prefer the appliance approach that Google provides. It just works. It uses the same Google API that you can already use and implements a dedicated hardware based solution that requires very little systems administration overhead.

As for the state of enterprise search today, I would say the the IBM OmniFind Yahoo! Edition is a powerful open source entry into this highly competitive space. With it's low-cost to entry and open source foundation, it looks like it will mature into a viable competitor to the Google Search Appliance in the next few years. Comparing the Google API to the Yahoo! REST based search SDK, I think the Yahoo approach using REST is much easier to use from a developer perspective.

I would not count out the open source product Nutch either. It is based on Apache Lucene which is the defacto best of breed Java indexing framework. Nutch has been brought into the Apache Software Foundation which is a powerful force in the open source universe.

As for the state of search, I think the leaders remain the same and the fresh new open source competition is breathing life into this area that was once completely dominated by a few products.