Wednesday, May 27, 2009

Enterprise Search in BI & SOA


Let me start this post by exciting you through what one could achieve through enterprise search in Business Intelligence. Think of following scenarios:-

  • You can do a Google equivalent search on all published reports and analytics lying in your data publishing servers and BI engines.
  • You can do a search on all the meta-data including hierarchies, data models, report structures, views etc.
  • You can find out on what all reports and views, a given data is lying (best way to find inconsistencies).
  • What if you can have a common interface of search, by which you can search the unstructured data and Business Intelligence data. For example, you may like to find out the technical specs of the products which are least selling in last 3 months. Etc..

Here are the natural benefits of Enterprise search in Business Intelligence:

  • People can home into the information they are seeking without specifying the specific report.
  • Enterprise search will definitely reduce the query load on the Business Intelligence systems, as people will find many of their queries being answered from existing reports, instead of firing a new query.
  • Less training- People don't have to learn to make a query or understand the structure of reports repository. They simply need to understand on how to place a good search string.
  • Combining Business Intelligence data with unstructured data completes the picture for the users.

Now let's see why the Enterprise Search in its nascent stage

  • Market still has to wake-up to its potential, and Vendors have yet not offered effective, widely acceptable and convenient offerings.
  • The enterprise search and Business Intelligence players are different specializations, and they still have to actively engage with each other.
  • Frantic consolidation phase, leading to an eye off the innovation, with more focus on integration of the acquisitions.

The barriers to growth of enterprise search:

The enterprise search is more of a business and modeling challenge than a technology challenge. For an enterprise search to be effective, we need to have the following enablers, which will take time to achieve value add.

  • A robust and well-defined meta-data model, and the user search needs to be guided by the meta-data model. Just imagine, if there are 100 different reports having hundred different terms of revenue. A user will see "Net Revenue" figures as being different across multiple reports, because inherent definition of "Net Revenue" may be different in these reports. Building a robust meta-data repository could be a big issue, as most of the organization struggle with that.
  • Guided discovery of search: The Business Intelligence search will become effective, if there is a guided search mechanism in the search capabilities. This will mean that a person when he gets 100 reports as the search results, he should be able to provide intelligence clues to home onto the right report which will provide the data. This is different from adding some additional keywords in your search bar.
  • Published reports will most-probably need to have "search tags" to ensure a faster and effective search. This will need technical changes in the products and also a more informed super-users.

Lets explore different ways including of using SOA to address the above barriers in my next post.