Who is responsible for data quality?

Recently I have been reading a lot about data quality. My curiosity in data quality field was re-ignited by my experience with SMB customers in the recent past.

Data quality in many ways is manifestation of business process issues (lack of the process or lack of enforcement of business process). I believe that data quality is a leading indicator of the possible underlying business process issues.

It is really important to look at data quality holistically, it is not only IT’s responsibility but it is a business issue and should be addressed by all stakeholders. Business is responsible for instituting right amount of processes and enforcing of those processes on an ongoing basis to ensure availability of high data quality. Availability of high quality data can only be ensured through partnership between various stakeholders within an organization. These stakeholders are all of the parties who are either:

a)      Using data (decisions makers),

b)      Generating data (users using systems or people in different roles running the business),

c)       Maintaining data (owners of the business units) or

d)      Providing a mechanism to capture data (IT)

In his recent post, Jim Harris (The Data information Continuum)  talks about how data quality and information quality is interrelated.  He also talks about how information quality standards are subjective to specific initiatives within the business units (rather than being enterprise wide). Extending Jim’s thoughts further, I believe that any time business makes a decision to use data for the purpose of either making strategic or tactical decisions, IT and business should look at any and all underlying assumptions about the data in light of how it is going to be used. In nutshell, every time data is used in new ways or new patterns by business, a mini data quality project should be spawned to ensure that there is no data/process quality issues which will make decisions/information gleaned from the data inappropriate or ineffective.

Let us take one real life example to illustrate my point. Let us assume that IT has built a BI system to provide analytics/reporting on pipeline data. Sales now want to decide on allocating more resources in the area where sales cycles are shorter and market is growing. To execute on this strategy, Sales operations team decides to look at the current and past pipeline by Industry verticals and success rates, and cycle times by each industry vertical. While looking at data by industry verticals, Sales Operations realizes that almost 80% of the accounts don’t have industry vertical information captured or the information is not accurate. In this situation, lack of Industry Vertical information can be construed as data quality, but in reality, Sales needs to identify this as a business need and implement a process by which when a new account is created, lookup on D&B data for industry vertical and entering that industry vertical information on account record as a mandatory process.

I truly believe that data quality and business process issues go hand in hand. It is not solely IT’s job to fix data quality issues as most of the time data quality issues lead into business process implementation/enforcement issues.  Data quality monitoring is not a onetime project, there needs to be awareness across all stakeholders that any time they are going to use data for making decisions in ways different than they have used in the past, they need to start a data/process quality assessment project.  More on practical ways to implement this ongoing data/process quality improvement process in next blog entry….