In his recent blog titled Data and Process Transparency Jim Harris makes the case that “a more proactive approach to data quality begins with data and process transparency”. This is very true of any organization striving towards availability of highest quality data for its decision-making as well as efficiency of business processes.
In order to embed transparency in every situation across organization, organizations need to be data driven. After all, what does it mean to be data driven organization, you may ask? There is great deal of literature around this topic in print as well as on the web. I will try to simplify this discussion and say that, to me, when culture of decision making is purely based on the factual data (KPIs/Metrics etc…) within an organization (and not on gut feel, emotions and subjectivity of decision making individuals), organization can be said to have become a data driven organization. Of course, this is a very simplistic definition (just for the purpose of this blog).
Many a times, depending upon organizational maturity you may have organizations which are completely data driven versus organizations which are more mature in one area (vis-a -vis data driven decision-making) versus other areas of the organization. For example, in some cases finance side of the organization might be much more data driven than either marketing or sales site etc.
Using data to make decisions drives both data and process transparency across organization. It discourages use of anecdotal information (and gut feel) and forces people to think in terms of realistic data and evidence presented by data. Also using specific KPI’s/metrics allows organizations to clearly define issues associated with underlying data or business processes more readily.
For example, if the sales operation team is discussing order return rates, they cannot simply say that we have a very low order return rate because of poor addresses in a “data driven organization”. They will say that they have 1% order return rate for (on average) 125,000 orders they ship every month because of the poor shipping addresses. This way of expressing performance not only helps everyone involved in understanding the importance of good data quality but also helps organization with creating sensitivity around capturing good data to begin with. Also expressing performance this way helps with ready-made business case for supporting underlying data management initiatives.
Transforming organizations to a data driven organization is a gargantuan change management task. It requires significant cultural/thinking change up and down the organizational hierarchies. During such transformations, organizational operational DNA is completely changed. Obviously, the benefits and rewards of being the data driven organization are immense and worth the efforts of transformation.
On the other side, during the data driven organizational transformation if organizations find that data is not of reliable quality, this finding will force data management discussion across organization and help kick start initiatives to fix the data as more and more in the organization start using data for decision making.
In end, I would encourage everyone to be as data driven as possible in their decision making and influence areas within your organization to be data driven. As data professionals, this will allow us to be more proactive in addressing data management challenges for the organization.
Business Intelligence: Process, People and Products 
Great post, Vish.
I definitely agree that data and process transparency are essential for organizations to be truly data driven.
However, sometimes transparency can initially have the opposite effect, especially if it reveals poor data quality and poorly designed processes.
This is the downside of transparency, since it can reveal how bad things are, but without this awareness, improvement is not possible.
But, as you mentioned, transparency can ignite the data-driven organizational transformation.
Best Regards,
Jim
Jim,
Thanks for stopping by and your comment.
Agreed, finding the problem and acknowledging that data quality problem exist!, are first few critical steps to get organization to commit to addressing DQ problem. More data driven culture you have in the organization, faster you will get to agreement/acknowledgment part.
Vish Agashe
I totally agree with Vish. Data and the process transparency are key critical factors for the success of any organizations. There are two areas of the interest here.
1. Source system :
• Source systems (OLTP) generates the revenue or helps in support and services. Most of the processes are defined in the source systems from the organizations prospective. They captures the new data or change the existing data as per defined processes at the source system
2. Decision support systems :
• Decision support systems pulls the data from source system and builds the KPI’/metrics as per business requirements.
Now the questions comes the quality of data. The data is owned by the business and BI team is not suppose to change it but report as is rather than changing the facts. The business teams reports the issue about data quality. Now the issue becomes and own by the BI team as you are reporting. Source systems owner disagree to change the process as they put together the point that the systems booking the revenue as expected and no issues so we cannot change the process and do changes the way source should capture or modify the data.
The transparency to data and process are key for the success of the any decision support systems which helps business to make key decision based on the facts but not on the gut feel as Vish pointed out. There is a need for the awareness within the organizations that the business owns the process and data and if any issues reported with the quality of the data first place to see is the processes which either capture or change the data. There should be transparency within the organizations for the processes and the data so that if there is an issue reported with the data will get owned and fixed with the help of business. This will creates the positive energy and confidence in data to make critical decisions based on the facts and will help for the success of many decision support systems.