Drowning in Data, But Needing Conclusions
=================================

In a world where companies are trying to integrate information, the fascination with technology combined with a lot of bad practices often causes companies to invest in projects that are tactical rather than strategic. The first posting lays out the problem. The next will discuss some of the ways of solving the problem.

The Problem
=========
When you think about information technology strategy, one useful framework is the pyramd.

At the top of the pyramid is the stuff that humans understand and which is not captured in information systems. This kind of knowledge is called wisdom, insights, judgment, values - it's the stuff of which we are made.

The next level down is sometimes called BI or business intelligence software. Typical software in this area is used for decision support, for reporting on the activities of the firm (recently named BAM or Business Activity Monitoring by Gartner Group). Tools in this area are budgeting tools, consolidation tools, activity based costing tools, simulation tools, strategic and financial planning tools, all forms of reporting tools e.g. Balanced Scorecard and other measurement reporting, and OLAP (on line analytical processing or multidimensional spreadsheets).

A layer deeper lies the information warehouse where information from multiple sources is collected in relatively raw form. At this level, customer, product, sales, and operational information is available but not terrible useful as there is so much data it is not comprehensible.

At the lowest level in the organization lie the transaction reporting and data collection systems. These are used for collecting the raw transactions that make up every organization. Some of these systems are integrated into the information warehouse; some are not. For example, a building security system might not be linked in with the HR system in the ERP (enterprise requiremens planning) package.

The Consequences
-----------------------
With all this complexity, five predictions can be made safely.

1. Moving information from each layer requires work and expense and often causes errors and opportunities for fraud.
2. The more ways that you represent your information, the more likely it is that maintenance costs for your system will be high and transitions in the way that you represent your business will be expensive to implement.
3. The more ways you collect information about your business, the harder it wil be to draw conclusions.
4. Cleaning data will be a material activity in firms.
5. Most human beings will find it very difficult to understand the sources of information in the organization.

By the way, the problem is probably getting worse. There are likely three major reasons why the problem is bigger today.

1. Companies have been downsizing.
2. Companies have been spending a high proportion of their budget upon enterprise application integration (40% is one quoted figure).
3. We now have more companies with a huge and wide investment in traditional information management systems, large ERP systems, and e-commerce and web front ends for servicing suppliers, employees and customers.

So, unless you are in a market leader with exceptional profits and an unlimited IT budget, what is the way out of this box?

I'll try to answer that question in the next posting.

Alistair Davidson
www.eclicktick.com