Data Governance Litmus Test: Quality, Reliability and Timeliness of the Data

This is a seventh blog entry in a blog series highlighting the criticality and the importance of executive sponsorship for data governance initiatives. In last entry we identified the need for capturing basic contextual information around systems, business processes, which has been automated by those systems and data being used by those systems to automate business processes. This information will help you in base lining current level of policies, procedures and their implementation as it is relates with systems, business processes and data. Armed with this information, you should have clear visibility into who (from the teams prospective), which (from a systems perspective), and what (from data perspective) will be involved in the prescribed scope (by KPIs) for your data governance initiative

At this stage it becomes necessary to profile the data, analyze the data for the accuracy, cleanliness and completeness. Reliability of the data from the perspective of business process which uses this data as well as the KPIs in which this data is used must also be evaluated. Profiling and analyzing data will also help you in capturing any policies and procedures which have not been explicitly defined, documented in the system of record, but yet are being implemented at data level (for example, after profiling data you might realize that only two payment terms are present in the data, but there is no explicit mention of this as a part of the policy definition)

In one of my consulting engagements with high-tech companies, I was methodically going through this process. CFO had identified reduction in DSO as one of the key goals for the year. Following these steps, when we mapped the actual systems being used to capture and report DSO, we realized that instead of being one financial application, this organization had three financial systems one which was supposed to be a corporate standard and two which had not been fully migrated as a result of past acquisitions. This was a big eye opener not only for me but all the people who have been crunching numbers to report DSO to CFO. To make the matters even complicate further, data at the summary level was being transferred from other two systems to the organizational standard financial application once in every 15 days. This meant that CFO was not getting accurate data for DSO on a week to week basis. Also, when we did some data analysis/reconciliation between these three systems, we found many data discrepancies (mainly associated with dates on transactions etc.) because one system was capturing summary level data versus other systems had detailed data.

Doing this detailed analysis of various systems and processes involved helped us (i.e. data governance team) in establishing the quality, usability, and accuracy of the data coming from three systems. It also helped us quickly identify the need to migrate two non-standard financial systems still being used to the corporate standard. The benefits of system migrations were very huge for the organization (standardization and optimization of collection processes, standard payment term usage and enforcement, being able to use appropriate cleansed addresses for billing thus reducing any delays because of wrong billing addresses, being able to leverage the data enrichments done to customer accounts in the main system, Being able to generate more accurate DSO metrics for reporting purposes etc.)

Documentation of this information in the metadata repositories and modeling tools really help application team responsible for maintenance of the financial systems in estimating and kick starting initiatives around system migrations. Also knowledge of specific data issues and possible benefits of fixing those data issues, standardizing policies (credit, collection, billing, etc. ) across the three areas of the organization helped us in demonstrating the benefits which can be had in improving DSO KPI.

By far, I would say that critical understanding of data issues, interdependency between different systems, teams (people) and business processes, and a clear understanding of lack of (which) policies or consistencies between the policies is extremely vital for the process of identifying impact of data governance initiatives on organizational strategic direction.

Previous Relevant Posts:

Litmus Test for Data Governance Initiatives: What do you need to do to garner executive sponsorship?

Data Governance Litmus Test: Know thy KPIs

Data Governance Litmus Test: Know goals behind KPIs

Data Governance Litmus Test: How and who is putting together metrics/KPIs for executives?

Data Governance Litmus Test: Do You Have Access to the Artifacts Used by Executives?

Data Governance Litmus Test: Systems, Processes and Data Behind the KPIs and Goals

Advertisement

Read Full Post »

Who is responsible for data quality? – Survey Results

After my recent post about “Who is responsible for data quality?” I received several comments from people. I posted this question to Business Intelligence Professionals in LinkedIn to get an understanding of what is the experience of BI professionals when it comes to responsibility for enterprise data quality… following are some of the findings based on responses to this survey/discussion.

In this survey I asked several questions

1. As to whom do you think is responsible for data quality?
2. How did you handle situations where data quality was an issue in your BI project or deployment?
3. How do you monitor data quality on a ongoing basis? How do you measure it?
4. How often do you assess data/information quality?

Following is summary of what I learned from the responses I received…

1. Unanimously, it was expressed and felt that the data quality responsibility lies with all parties who deal with data across organization (creation, maintenance, usage, management etc…).
2. It was suggested that IT should play advisory and enabler role where IT is not only providing tools and technologies to find out quality problems but also making recommendations and providing capabilities to correct the data quality issues business faces. Extreme collaboration between business stake holders and IT was suggested and expected in fixing data/Information quality issues.
3. There was consensus around validating data quality periodically. Response as to how often it should be validated varied from person to person and was based on their specific experience from the past and current engagements. (It varied from daily to monthly to quarterly to every once in two years)
4. While data warehouse or BI solutions can correct data quality within the warehouse domain, there was preference to fixing data quality at the source where ever possible (off course with the help from business stakeholders or business stakeholders taking responsibility to do this with enablement from IT)
5. Other measures/factors which could be used to make data quality a continuous process was…
   1. Building business rules/processes in the System of records applications
   2. Training users who are responsible for generating data as well as people responsible for using/consuming data for decision making
   3. Use and aid of various technologies including but not limited to data profiling, business rules repository, metadata repository to capture metadata/rules etc… was suggested.

In summary, I was really happy to see the unanimous response about IT-Business stakeholder collaboration to fixing data quality. This really puts IT in much more strategic and valu additive role in the organization when it comes to valuing the data assets of the organization.

Read Full Post »