Archive for July 17th, 2009

How many times in the careers of data quality professionals, they are called upon when data quality has become pervasive issue in the organization? I always wonder as to why it has to be this way? Why can’t data quality be considered, injected into any business initiatives before it becomes such a big issue?

I personally think there is a hope and there is a way around it. We (data quality, integration, applications, IT professionals) just have to make sure that data quality becomes one of the most critical initiatives on the radars of executive management and it needs to be championed by CIOs. IT can act as enablers, evangelist in the whole process.

In a recent interview of Jim Harrisconducted by Ajay Ohri,Jim gives an example where in a financial services company had a critical data source in the form of applications received through mail or over the phone. All the data entry personnel were compensated on how many applications they enter in a given time frame, whereas most critical information for the financial institution was correct social security number (there lays the issue). As Jim explains in his findings, when ever social security number was not present or was not legible, data entry personnel entered their own social security number to proceed with data entry operations. This clearly creates for a very low value data base for this financial services company.

Had the IT/CIO data quality evangelist participated in the process of capturing this information and if they had asked some critical/right questions about the intent and usage of capturing this information (e.g. what is critical and most important information in this application form? What happens if the information is not accurate? Are there ways to validate the information being entered), one could have easily

  1. Put some checks around social security number to be valid (cross ref with existing customers etc….)
  2. Encouraged reward model to the team based on accurate information/valid data quality of number of applicants rather than just number of applicants. (Aligning goals of organization through correct data quality and right rewards model)
  3. Create a process where in doubt transactions/records can be triaged through and corrected before accepted and fully complete records etc….

Again my intent is not to pick on one instance or industry. Point I am making here is that Data Quality/Information Quality should always be injected in any new/existing initiatives at the front end rather than back end of the business process. Data quality and how it will be ensured becomes one of the input/drivers in implementing any new business initiatives (it would go side by side with the business objectives). Also consideration of data quality is not a just technology issue but it is a business issue (and hence in above example, I suggest that questions/suggestions from Data quality evangelist could influence how team is compensated)

I know that this discussion would automatically lead into data/information governance and management but there are really small steps any IT organization (CIOs) could take to fix this issue incrementally. Create a role whose (one of the many) responsibility is to make sure that

  1. Any business process/initiative which captures/modifies data has a set of requirements around intended use of the information and assumptions around what that information will be
  2. Outline what data quality/validity checks be performed in a mandatory fashion to achieve clean data for the purpose/intention of the business use
  3. Create a monitoring system to ensure that outlines created for validating data quality are being implemented and are being worked on.
  4. Evangelize importance of correct data and correlation between data quality and information quality.

I would love to get your thoughts on this topic… I think that prevention is better than cure (while we cannot always prevent, we can try) and time has come (given the explosion of data and emphasize on objective decisions based on data rather than gut feel) for all of us to start pushing for data quality at the front end of the process.


Read Full Post »