SODW Success is Enterprise Success

Mantra for Successful SODW

1. A focus on the nouns and the verbs of your organization to provide guidance in decision making when contemplating new projects. Does the proposed project fit within and/or adhere to our organizational definitions and requirements of nouns (data) and verbs (function)? This new system must integrate with the rest of the enterprise.

2. Data governance programs and policies to define clear ownership, purpose, and use of data in the organization.

3. Function governance programs and policies to define clear ownership, purpose and use of shared functions and services in the organization. As data management professionals, we all know the importance of data governance, so we are all prepared for making sure our functional architecture has the same discipline and rigor.

4. A Master Data Management (MDM) strategy that supports the integration of and use of the shared data in the organization. MDM, in my mind, is one of the key components to any effective service oriented architecture (SOA), and often gets overlooked. You cannot effectively share services if you do not effectively share the data that is the backbone of your organization.

5. Data integration and data access services are the core services that allow us to integrate data into our MDM solution and warehouse, as well as provide common access to that data. This is where we make data integration lighter and provide the ability to access analytic data more dynamically with other, non-analytic processing. This is also where we can integrate the rest of our organization’s data, the semi and un-structured data.

6. Data quality and standardization services to make sure that the shared data being integrated into the MDM solutions are all applying the same rules. In fact, it would be best to have these services pushed as close as possible to where the data enters the organization, whether it is typed in on a web page or comes in as an XML transaction from a 3rd party. Many data warehouse solutions do the data quality and standardization; but if I can leverage these as services and put them at the point of entry then we will have better data across the organization, making integration easier and more cost effective.

7. A technology platform that facilitates the deployment and orchestration of data services as part of every day business processes. Something like “closing the loop” in data warehousing. This is where it becomes possible to leverage data and functionality that was exclusive to the data warehouse and business intelligence applications and make them part and parcel of everyday business processing.

To be successful in deploying a SODW, your organization must realize the increased interdependency with the rest of the applications in your organization, which is a good thing. Previously we moaned about the hassles with poor data governance upstream from the data warehouse. Well, now we also have to worry about having a consistent, interoperable function architecture as well. Again, in my mind this is a good thing. It is an opportunity, initially borne out of the need to integrate data, and now function, that will push the data warehouse to the center of the organization.

To conclude the data warehouse is no longer a proxy for the enterprise data architecture, but a driver for better overall enterprise architecture which is sharing, reusing, standardizing, increasing efficiency… Sounds an awful lot like saving money to me.

Service Oreinted Data Warehousing - SODW

When I sat down to write this blog on service oriented data warehousing (SODW), I had to reflect on the work and struggles I have had in data warehousing and data integration, the people I had learned from, and some of the failures trying to push these concepts before they were technically feasible. From those reflections I came to realize that

1) a data warehouse is a proxy for good enterprise data architecture, and

2) the process for which these things are being built makes them very difficult to change as the needs of the organization changes. Not to mention that most organizations have separate groups that do operational integration and data warehouse integration (ETL).

Data warehouses, to some extent, have become part of the problem they intended to correct: another system that demands resources for its care and feeding, yet doesn’t really integrate well with the other systems.

How can we build solutions that allow us to integrate data for both operational and analytic purposes?

If we can integrate data more easily, would that not facilitate operational analytics and active data warehousing?

How do we integrate and tap into all that semi-structured and unstructured data?

To satisfy these questions, to really close the loop and make the warehouse part of the organizations daily activity, is to rethink how we integrate data and expose that data to the organization.

I don’t want to state the obvious here but many data warehousing and business intelligence projects are not as successful as they could be because they lack a sufficient enterprise foundation. There are many things that must be in place such as data that is standardized and of a high quality, consistent definitions of information and function, and ownership of data and function.There is an important point I need to make here: the need for a solid foundation doesn’t change just because we are throwing around the “service oriented” tag. Let’s not confuse the need for flexibility and agility with quick and dirty. SODW still requires good enterprise data architecture practices that are aligned with how the data is used. Perhaps even more discipline and rigor are required than we have seen in the past. In fact, a successful SODW program will help drive improvement by focusing on doing things well once and reusing them, instead of doing the same things many times quickly. In the long run, reusability is more cost effective and more agile.

So what do we need to for successful SODW? Really, we need the same things across the enterprise to make things truly interoperable and reusable. These things are not only keys for successful SODW but, in my opinion, successful enterprise applications. I will explain them in my next post