Three City Speaking Tour

I am really excited to be setting out on a three-city speaking tour of Australia next Sunday, 27 July.

Three city talking tour for John Owens July & August 2014

MDM Summit

My first stop is at the MDM and Data Governance Summit in Sydney, which starts on 29 July.
Ibll be giving a talk on the 30 of July and then running a half-day workshop on the 31 July. My subject on both days will be my special Master Data Management technique of Multi-Dimensional MDM.
My workshop on 31 July will be especially powerful with over 3 hours of fantastic insights and techniques designed to empower your enterprise in the each of the areas of the four Master Entities, whichB I reveal in my talk on 30 July.
If youbre going to this event, please feel free to make contact with me so that we can have a coffee and catch up.
You can still register for the Summit, click MDM & DG Summit.

PPDM Symposium

When I finish in Sydney, I then fly to the PPDM Symposium in Perth, Western Australia.
Here I will be talking to representatives of the Oil and Gas Industry on 6 August on my very latest development, which I call the DNA of Data Quality.
If you are involved with the Oil and Gas Industry or are eager to know about powerful new approach to Data and Information Management, then you can still register for this event b B clickB PPDM Symposium.
If you are attending, then please come and say bHellob. I would love to connect with you.

ITEE, Brisbane

After Perth I fly to Brisbane to give a talk at the School of Information Technology and Electrical Engineering at the University of Queensland to research students and members of the Data Quality community in and around Brisbane.
This will be on 12 August from 2pm to 4pm in Room 49-502.
If youbre in Brisbane on that day, please come along, as I would love to meet up with you.

New Client

While Ibm in Brisbane I will also be shooting down to the Gold Coast to do a Rapid Strategic Assessment for a new client there, who are about to do very exciting new things with both their business and their systems in order to take themselves to a whole new level of success.

Getting Business Back on Course!

The main thrusts of my talks in Perth and Brisbane will be about moving information back to the heart of the business, which was where it resided for all successful enterprises prior to the advent of commercial computerisation.
Enterprises have now lost the power of information and are struggling with its poor cousin, data, which is not up to the job!
Data is what you need to do analytics. Information is what you need to do business!

Working Together

If you are an analysis & modeling practitioner and passionate about taking your knowledge and skills to a whole new level, there are now two main ways in which you can work with me.
The first of these is one-to-one mentoring, where we work closely together throughout the year and make great things happen really quickly.
The second is a practitioners master class where, with just 11 other highly motivated practitioners, we work through your challenges over six months, both online and at two separate workshop retreats at a really nice location, over a period of 6 months.
To find out more, drop me an email at john@jo-international.com. I look forward to hearing from you and working together.

The DNA of Data Quality for Oil & Gas, E&P

DNA-for-EPI am honoured to have been invited to be a keynote speaker at theB PPDM SymposiumB inB Perth, AustraliaB inB AugustB this year.
It will be nice to be back again in front of an audience from this industry, with whom I have a long standing connection, to unveil my latest Integrated Modeling Method (IMM) innovation, theB DNA of Data Quality.
My association with Oil and Gas E&P started nearly 30 years ago at Nam in Assen in the Netherlands. It was there that a colleague, Nicholas Hann, and myself were the first people to produce a bcradle to graveb business model for the E&P Industry when we modeled all of the operations of NAM. These ground breaking models were later adopted by Shell and became an industry standard.
My work at NAM and later the lecturing work I did for several years at Oracle, laid the foundation for IMM, the core power and uniqueness of which comes from its unique and complete integration.
In the DNA of Data Quality I take this integration to a yet another level and show how, when they ignore intrinsic data structure rules of the enterprise, Data Quality initiatives are almost certainly doomed to fail.
If you know of anyone working in E&P in Australia, New Zealand or anywhere in the world who is passionate about taking their data quality knowledge and practice to a whole new level, then get them to register for theB PPDM Symposium in Perth. I would love to see you and them there.

Working Together

If you are an analysis & modeling practitioner and passionate about taking your knowledge and skills to a whole new level, there are now two main ways in which you can work with me.
The first of these is one-to-one mentoring, where we work closely together throughout the year and make great things happen really quickly.
The second is a practitioners master class where, with just 11 other highly motivated practitioners, we work through your challenges over six months, both online and at two separate workshop retreats at a really nice location, over a period of 6 months.
To find out more, drop me an email at john@jo-international.com. I look forward to hearing from you and working together.

Waging War on Bad Data? Give Up. Youbll lose!

War on want. War on drugs. War on terror. Have these wars eliminated want, drugs and terror? No. Some would argue, quite the opposite.I-want-you-war-on-data-v02

There are two main reasons for this, both allied to the human condition. The first is based on a phenomenon first observed by Carl Jung, that made him coin the term, bwhat you resist persistsb. He arrived at this conclusion after years of observing that, when people put a lot of energy into suppressing or resisting a negative behavior or trait that, instead of disappearing, it persists and, in many instances, grows worse.

The second reason is a little darker. Imagine that tomorrow all of the agencies that have been set up around the world to fight drugs and terror could, with the wave of a magic wand, bring an end to all drug wars and acts of terror and, as a result of doing this, they could dismantle themselves and free up all of their personnel to do something else with their lives. Imagine this and then tell me, do you think that they would wave the wand and make it happen?

So It Is With the War on Bad Data

The same human traits and conditioning are at play in every enterprise. Because of this, if the means by which you have decided to eliminate bad data in your enterprise is to declare bwarb on it, then be aware that the situation will not significantly improve, that the financial spend will continue to grow and the number of departments and teams set up wage war on bad data will not only become in integral part of the establishment, they will grow and multiply!!!

How many such teams and departments do you have already? Data quality, data governance, data management, master data management, metadata management? More will be created. Big data and data alignment are already on their way.

Whatbs The Alternative?

The rationale that drives the bwarb on bad data is based on the same logic and thinking that has lead to the creation of all of the bad data in the in the enterprise in first place and, as Einstein points out, bno problem can ever be solved using the same logic that created it in the first place.b

So, an entirely new approach is needed. Enterprises need to forget about warring against bad data and concentrate instead on creating only good data.B Letbs face it, if they didnbt create any bad data they will have no need to wage war on it!!!

To some people this idea will be incomprehensible; as incomprehensible as sending a man into space would have seemed only a hundred years Data_warriorago.B However, creating zero data defects is far simpler than mounting a space shot.B It needs no new technology and it actually costs far less than creating defective data and then trying to find and remove it.

All it needs is for enterprises to make it easier and more beneficial for people to get data right first time than it is to get it wrong. All of the resources required for this already exist. It just needs imagination and creative design to apply them properly.

Letting Go

One of the main barriers to bringing this essential mind shift about is that so many people do not want to give up the war. Some, because they find it hard to understand and accept that bwaging warb is a failed approach. Others enjoy the battle too much; they actually find an identity in the battle that gives them meaning b the data warrior b and they will not let go of this, no matter what price the enterprise has to pay.

Working Together

If you would like to work with me and discover ways in which your enterprise can give up its war on bad data and instead achieve the high quality data that will enable it to thrive at far less stress and cost, then contact me now.

Is Data Dialysis Killing Your Enterprises?

One of the major barriers to B improving the maturity of Data Quality in enterprises around the globe is the mistaken belief, by far too many DQ practitioners, that good data creates a good enterprise.

The reality is quite the reverse of this. It is a good enterprise that creates good data. Good data is the output of the effective execution of the Business Functions of the enterprise. B Good data is a major indicator of the health of an enterprise b it is not a driver of it!!!B

Data Dialysis Insanity

Putting healthy data into an unhealthy enterprise in an effort to make it healthy - insanity?

The current approach to Data Quality of finding and correcting data defects in an effort to make an enterprise bhealthierb, is about as effective as trying to turn an unfit and unhealthy body into a fit, healthy one by removing its unhealthy blood, B running it through a dialysis machine and returning it to the body. B In the medical world, this would be seen as an expensive (and insane) waste of B time, blood and money that would in no way improve the fitness of the body and, ultimately, result in the degradation and loss of blood and, not long after that, the death of the body.

Data dialysis, is slowing killing those enterprises that practice it.

Quality Assurance

The world of manufacturing many years ago moved away from the archaic practice of Quality Control. Manufacturers at last realised how insane it was to spend time and money creating defective products and then more time and money trying to find and remove these defects. However, for some bizarre reason, the world of Data Quality has remained locked in this outmoded and totally ineffective Quality Control time warp, doomed to perpetually fail to deliver any real long-term benefits to the enterprises that practice it.

Nothing changes without change. B Until those involved in Data Quality come to realise that the only way to achieve genuine, sustainable Data Quality is to practice Data Quality Assurance, then nothing will change. Data Quality AssuranceB is all about getting it right first time every time. It is all about executing all Business Functions in such a manner that B all data created and transformed by their execution is done so correctly first time, every time.

Zero Is Just a Number

The world of manufacturing was not always as enlightened as it is now. B The concept of zero defects, of manufacturing things in such a way as to get them right first time, every time, seemed impossible to many enterprises. History shows that in some enterprises the conversations had to begin with phrases like, bSuppose we could show you ways to manufacture a car with just B fifteen defects, would you be interested?b When these enterprises realised that fifteen was just a number, they then realised that they could reduce it all the way down to zero.

This realisation has yet to come about in the world of Data Quality. Zero is just a number. At the moment creating 1000 defects a day might be acceptable in an enterprise. That enterprise might well see a reduction of this number of defects to 500 a day as being desirable and achievable, while at the same time seeing a reduction to zero as being impossible. However, when they B begin using Quality Assurance techniques and get to 500 defects a day, they will also begin to see 250 defects a day as both desirable and achievable. They are one step closer to seeing the number zero as being both desirable and achievable.

It can be done. But not until DQ practitioners change from practising Quality Control to B practising Quality Assurance.

Share the Love

If you enjoyed this post, please share it with colleagues and friends B by clicking a social media button below.

To follow me on Twitter or Facebook, click on a floating icon at the bottom right corner of this window so that we can stay connected!

Customer and Supplier are NOT Master Data Entities!

This post was prompted when I received some feedback after I ran a webinar on MDM. B The feedback highlights some of the structures and thinking surrounding some MDM bsolutionsb that, contrary to what they are sold as doing, actually contribute to MDM fragmentation.

Have a read of the feedback and my response and let me know what you think.

bJohn, It was a nice presentation and very different thought process from the traditional thinking of MDM implementation although I am not convinced on some of theSAP-MDM-Logoconcepts around modelling. I have been using SAP MDM for past 8 years and SAP standards data models are restricted by entities like customer, vendor etc. and never created as a single model for party.

Based on my experience it has worked great to have models based on entities as they address needs of different business functions, different set of hierarchies being followed by for each entity and there attributes of entitiy specific to banks, sales, contacts, address linked to the relevant functions they have been used.b

My response to this was as follows.

Thank you for your feedback. It is remarkable how so many ERPs and MDM packages (even mainstream packages) have made the huge mistake of modelling and implementing Customer, Vendor, etc. as separate Master Entities!

The reality is that these are not Master Entities. They are merely Roles played by the true Master Entity ofParty, as are Roles B such as Agent, Guarantor, Referee, Employee, etc, etc. B This fundamental modelling error is one of the main causes of MDM fragmentation globally.B Customer-and-supplier-mdm

This is not just an academic modelling error. It causes huge operational problems and can also put enterprises at serious commercial risk. B This is most apparent in sectors like insurance where one Party can play many roles in a life insurance policy, such as Purchaser, Guarantor, Beneficiary, Agent, Authoriser, etc.

The approach that the packages you mention take would create each of these as a different Master Entity and expose the enterprise to significantB risk. B I have even seen the crazy bData Qualityb solution to this of trying to identify where a Purchaser, Guarantor, Beneficiary, Agent, Authoriser have the same name and try to blinkb them!!! B Why would you be crazy enough to create them as five separate entities and then see if they are the same thing? B All, you have to do is create one occurrence of Party and assign all of the necessary Roles to that Party!

This whole problem is acerbated and perpetuated by the silo mentality that pervades so many enterprises. People in Sales only see Party in the Role of Customer and people in Purchasing only see Party in the Role of Supplier and, with this myopic and blinkered view, many actually come to believe that these are different things. B Others just donbt care.

Identifying Party as the true Master Data Entity is crucial in achieving true MDM and eliminating the pains and losses that this causes enterprises worldwide.

Share the Love

If you liked this post, please share it with colleagues and friends who are committed to improving B Master Data Management in their enterprise by clicking a social media button below.

To follow me on Twitter or Facebook, click on a floating icon at the bottom right corner of this window so that we can stay connected!

Work With Me

If you would to work with me and prevent these MDM Fatal Errors occurring in your enterprise then B email me atB john@jo-international.comB or Skype me onB johnowensnz.B I look forward to connecting with you.

What Are You doing to Protect Your Big Data from 5NF Syndrome?

Big Data x 5NFB = Really Big Data Errors

It is true that Big Data may offer huge opportunities for enterprises to gain hitherto unimagined insights. Alas, it also true that is has the potential to tell enterprises really big lies!

This has nothing to do with the quality of the Big Data. These bliesb can arise from 100% correct data.B They arise due to data structure anomaly.

This huge risk is all down to the fact that the structures of data that you will get from external sources arehighly likely to break Fifth Normal Form (5NF).B In truth, it would be almost a fluke if such data did not! B I call this propensity for merged Big Data sets to lie 5NF Syndrome!

What is Fifth Normal Form?

A good definition for 5NF (that is understandable) is hard to find. The best way to explain it is by an example from real life.

A distribution company in the UK had a data table in its corporate distribution system that looked like Fig1 below.B This showed which manufacturerbs products the enterprise was sanctioned to distribute and to which retailers.

Figure 1 of Big Data and Fifth Normal Form (5NF) Lies

Over time, three regional divisions extracted parts of the data from the corporate distribution system for use in their local standalone systems in order to do some regional analysis. The extracted data looked like this in the three regional systems.

Figure 2 of Big Data and Fifth Normal Form (5NF) Lies

A regional manager, who had responsibility for the three regions, had some creative ideas for expanding his local distribution infrastructure. In order to ensure that his business case was sound, he needed to do some analysis that would enable him to know the overall form and spread of distribution services across these regions.

What easier way to do it than to run a report using the data in the three standalone regional systems? After all, they contained all of the data elements that he required.

All that was required was a simple SQL query joining the three.

New Insights

Before long his analysis was giving him lots of new insights. It suggested that the enterprise was actually distributing many more manufacturersb products to many more retailers than they had previously imagined

Just to make sure that this was not due to an error in the SQL, the code was checked and proved to be working correctly.

It seemed that the data in the three regional systems had actually been concealing valuable sales intelligence. Excited by these new insights, the regional manager built a compelling business case for expanding his distribution facilities and presented it to head office.

Due Dilligence

Doing due diligence, head office staff checked the bexciting new insightsb against their own distribution analysis taken from the corporate distribution system b the system that had been the original source of the data for the three regional databases.

The distribution analyses figures did not match.B The regional system was showing much more activity than the corporate system.B Something was wrong, but what?

Together IT and the business checked the source data against the data that had been originally extracted into the three regional systems and found that it matched.

The SQL query that produced the analysis was checked and double checked. It too was correct.

Many hundreds of man-hours were consumed in trying to solve this huge anomaly.

Itbs All Academic

It was not until an external consultant was brought in, as a last ditch effort, that the mystery was solved.B He listened to the history, looked at the tables and asked, bHave you heard about Fifth Normal Form?b A few had heard of it but no one had any idea what it was, other than some esoteric data rule that might have some relevance in academia, but none in a commercial enterprise.B How wrong they were!

Global Problem

Violations of Fifth Normal Form (5NF) trip up innumerable data projects around the globe every year.B So what exactly is 5NF?B It is a normalization error that occurs when you bover normalizeb data tables and then try to recombine them.

In the distribution company this bover normalizationb occurred when the original three-column table (Fig 1) was split into three separate two-column tables (Fig 2).B Performing queries on the individual tables will not result in any errors as, in standalone mode, there is nothing wrong with their structure.

The violation of 5NF occurs when you try to combine the columns through a query that joins all three tables.B This is not just an academic error.B It is a fatal error that consistently produces false values as demonstrated in Fig 3 below.

Figure 3 of Big Data and Fifth Normal Form (5NF) Lies

The row highlighted by the red arrows did not exist in the original table in Fig 1. Yet every time that you run a query that joins these three separate tables this extra phantom row is created.

This is just one phantom row generated from just four rows in the original table. Imagine how many would be generated if the original table had contained 10,000 or 1,000,000 rows!

Once fragmented in this way, any attempt to recombine the tables will always result in phantom rows being generated b and there is absolutely no way of ascertaining which rows these are.

Whatbs 5NF got to do with Big Data?

The fact is that there are probably billions of Big Data sets out there which, when queried on a standalone basis, will give totally accurate results.B However, these same data sets, if joined together through a query, will always produce spurious extra rows.B This means that the new binsightsb that Big Data is throwing up for your enterprise may well be the biggest lies that you could tell yourself!

When these structures exist, Big Data will always lie to you and you will have no way of telling which elements of data generated by the query are the lies!

The only thing you can do to prevent these errors is to test to see if the data structures violate 5NF b before joining the data!B How to do this is far too much to cover here.B I show one technique for checking this in my book IMM Data Structure Modeling.