Service Oreinted Data Warehousing - SODW

When I sat down to write this blog on service oriented data warehousing (SODW), I had to reflect on the work and struggles I have had in data warehousing and data integration, the people I had learned from, and some of the failures trying to push these concepts before they were technically feasible. From those reflections I came to realize that

1) a data warehouse is a proxy for good enterprise data architecture, and

2) the process for which these things are being built makes them very difficult to change as the needs of the organization changes. Not to mention that most organizations have separate groups that do operational integration and data warehouse integration (ETL).

Data warehouses, to some extent, have become part of the problem they intended to correct: another system that demands resources for its care and feeding, yet doesn’t really integrate well with the other systems.

How can we build solutions that allow us to integrate data for both operational and analytic purposes?

If we can integrate data more easily, would that not facilitate operational analytics and active data warehousing?

How do we integrate and tap into all that semi-structured and unstructured data?

To satisfy these questions, to really close the loop and make the warehouse part of the organizations daily activity, is to rethink how we integrate data and expose that data to the organization.

I don’t want to state the obvious here but many data warehousing and business intelligence projects are not as successful as they could be because they lack a sufficient enterprise foundation. There are many things that must be in place such as data that is standardized and of a high quality, consistent definitions of information and function, and ownership of data and function.There is an important point I need to make here: the need for a solid foundation doesn’t change just because we are throwing around the “service oriented” tag. Let’s not confuse the need for flexibility and agility with quick and dirty. SODW still requires good enterprise data architecture practices that are aligned with how the data is used. Perhaps even more discipline and rigor are required than we have seen in the past. In fact, a successful SODW program will help drive improvement by focusing on doing things well once and reusing them, instead of doing the same things many times quickly. In the long run, reusability is more cost effective and more agile.

So what do we need to for successful SODW? Really, we need the same things across the enterprise to make things truly interoperable and reusable. These things are not only keys for successful SODW but, in my opinion, successful enterprise applications. I will explain them in my next post