Cover story: Intelligent decision making

Feature

posted 15 Mar 2005 in Volume 1 Issue 8

Intelligent decision making

Business intelligence is storming up the corporate agenda and becoming an important component of IT budgets. But BI projects must be aligned with the strategic objectives of the organisation. Failing this, companies will reap only a fraction of the benefits of their BI deployments. By Tracey Caldwell.

Business intelligence is a rare growth market. Enterprise-wide information that may be easily analysed to inform accurate decision making is a goal that most organisations strive for. And although this goal has been largely out of reach since the inception of enterprise-resource-planning (ERP) and knowledge-management (KM) systems, there are signs that 2005 is the year that companies are looking to invest in BI systems. This trend is being fuelled by the desire of organisations to derive more value from the large investments they have already made in customer-relationship-management (CRM), supply-chain-management and ERP solutions. The maelstrom of incoming legislation governing how businesses handle data is also a key driver.

However, like other enterprise-wide applications intended to improve the bottom line, BI applications do not always deliver the intended results, or live up to vendors’ promises. It’s important that enterprises looking at BI solutions are aware of the potential stumbling blocks so they can avoid falling headlong into the traps that have caught so many before them.

Perhaps the fatal flaw that the technology industry has consistently made with ERP, CRM, knowledge management etc, is that such technologies are always presented as a total solution, with little thought given to exceptions. Organisations need to be aware that information that should inform business intelligence may be unstructured and lie beyond the data warehouse.

As BI technologies mature, the traditional functions of reporting, query and analysis are being enhanced with sophisticated visualisation and analytical tools, and are extending into business-process management. BI and ERP systems are also converging as enterprises that began by integrating BI tools with the ERP repository see the benefit of creating a separate, centralised data warehouse. And ERP and CRM vendors are including BI tools to leverage corporate intelligence, with varying degrees of success.

A Gartner survey of 1,300 CIOs in more than 30 countries found that CIOs plan to increase spending on BI by an average of 6 per cent this year as companies move from cost-cutting initiatives to investing for business growth. Its research found that CIOs believe strategic use of business intelligence, combined with a focus on improving business processes, will be the most significant factors in delivering IT’s contribution to business growth in the next three years.

However, just like any technology investment, BI deployments can and do go wrong. According to Gartner, businesses are still failing to connect IT departments with core business units – the end result is that IT departments are creating data warehouses, the core of any BI implementation, without business involvement. Such a trend has led to a bad press for BI, and many enterprises have abandoned their attempts of implementing an integrated data warehouse.

As Ovum analyst Helena Schwenk writes in The need for business intelligence: “There have been some high-profile data warehousing failures that inevitably resulted in a backlash against their implementation. Many organisations continue to develop line-of-business BI systems in parallel with their data-warehousing projects and/or returned to this paradigm as a reaction to dissatisfaction with the centralised warehouse. In a lot of cases, failures were blamed on ‘political and organisational issues’, but these need to be treated as an intrinsic part of the data-warehousing strategy rather than considering data warehousing as a primarily technical issue and hoping to fix political and organisational issues separately.”

The human issues surrounding the necessary data integration cannot be overlooked. Middle managers protecting their power bases may disagree on how their data should be integrated with that of another business area. Sometimes the process of designing a BI system may uncover areas of management weakness and underperformance, which may be a symptom of poor supporting systems. There is general agreement that many BI projects fail because they are not built on solid foundations. The data quality and integration is simply not good enough to meet the sophisticated needs of the modern-day organisation.

Of course, there seems little point in attempting to graft sophisticated analytical tools on to poor data. “Over the last 15 years there has been an underinvestment in back-end information, such as data being reconciled and cleansed, with more emphasis on front-end tools as these are more visible,” says Richard Neale, UK product marketing manager of Business Objects. “It is human nature to focus on evaluating the tools, but business intelligence is like an iceberg with a lot more going on than is visible at first.”

In the Gartner report The five fatal flaws of business intelligence and corporate performance management, analysts Lee Geishecker and Bill Hostmann argue that middle management is not very interested in ‘one version of the truth’. They recommend that strong force should be applied to achieve this: “Organisations should use the pressure of compliance to achieve greater things, such as cleaning up the many data silos, creating more ownership around performance data and eliminating many of the thousands of spreadsheets.” However, Geishecker and Hostmann are quick to point out that compliance is somewhat of a negative driver. They argue that a more positive driver would be to build a business case to share information with multiple stakeholders.

As a number of recently introduced regulations begin to bite, enterprises can no longer ignore the importance of having consistent data with clear audit trails. Sarbanes-Oxley, affecting US companies and their overseas branches and companies doing significant business in the US, demands rigorous financial reporting and corporate-governance standards. UK CIOs stand to face jail for reckless auditing too, when the Companies Bill receives royal assent later this year.

The finance sector is ahead of the pack in responding to the new raft of regulations, but not all have grasped data quality issues fully. “I have recently been involved with banking events in Europe and the Middle East. Many banks have yet to come to terms with how they utilise their existing data with compliance and business governance issues,” says Rob Graham, product manager at BI vendor Datawatch.

He rejects the view that eliminating unnecessary data is as important as the data quality issue: “That’s all very well, but who decides what is unnecessary data? Prior to Sarbanes-Oxley, for example, certain financial data not deemed important at the time could now be vital.”

Arguably, the real flaw in BI systems is not necessarily the quality of the data held within them, but the much larger amount of unstructured data that is not included.

“BI is not taking into account unstructured information such as word documents and e-mails,” says Ian Black, managing director of Aungate, a division of Autonomy that provides automated-compliance systems. “In corporate governance there is a need for consistent reporting and transparency and the CIO needs to know that what he is signing is concrete. But the majority of information used to get a job done is in e-mails. Robust financial systems may be based on the compliance needs of Sarbanes-Oxley, but the instructions the financial director might give the financial department could be recorded more informally.”

If the definition of BI is extended beyond the numbers in the data warehouse and applied to all data that could affect business decisions and compliance, a whole new set of issues arises. Kevin Miles, head of knowledge management at the Transport Research Laboratory, identifies a new threat to the success of a BI system that includes the records-management system. “The biggest threat to BI management is Google’s desktop-search application, as it makes people question the need to file information in an electronic-document-management system.”

Enterprises have been quick to pass these hot potatoes into the hands of vendors, reckoning a one-stop-shop solution will pass the onus to the supplier to cover all of the issues cost effectively. Lee Geishecker, Gartner’s research VP, warns enterprises not to assume that enterprise-application providers will always address all information requirements. “Although some enterprise-application providers can save you money overall, it isn’t always the case. Always compare your enterprise-application vendor’s solution with that of a market-leading specialist vendor,” she says.

Systems integrators (SIs) may then be brought in to integrate best-of-breed applications, but Gartner analysts warn that SIs could prove the weak link in the chain: “The BI capability of system integrators varies widely. Some can only provide application developers for project-level work. Others have developed specialised competencies delivering a full stack of best-of-breed capabilities from selected vendors with whom they have established strategic relationships. Others bring industry-specific domain expertise.”

Enterprises need to be creative about their partnerships to secure commitment and stability. “When choosing a solution provider, be encouraged by those vendors who will put their money where their mouth is – that is, those who will share the risks as well as the rewards,” says Richard Gregory, global business-intelligence-application architect at global brewers InBev. “But BI initiatives can often be expensive by their nature, so you need to partner with someone who has a high confidence of being successful,” he adds.

It is clear that battle lines are drawn as established BI market leaders looking to tap into the SME market face new competition from the likes of Microsoft with its SQL Server enhancements, as well as enterprise-search vendors who are also adding new analytical and visualisation capabilities to their product suites. SMEs looking to implement a BI strategy perhaps for the first time will face a bewildering range of options. Established market leaders are likely to fight their corner on track record and depth of product development, arguing that medium-size companies should not accept a cut-down version of BI as their needs are just as sophisticated as large corporates.

Andrea McCuish, general manager of marketing company Database Direct, wanted to pay less than £250,000 for a BI system that would help the company launch into a new area of business. She expects a return on investment in two and a half years from the system that was implemented by solution provider Elucid. “The system is real time and at any point we can produce any one of 90 standard reports to simulate different situations,” says McCuish. The system allows Database Direct to make daily decisions about marketing strategy in response to the intelligence provided. The nirvana of enterprise-wide business intelligence seems to be within reach of this size of enterprise as data quality and integration issues are not so wide ranging or prohibitive.

The right choice of vendor will help organisations avoid implementation pitfalls. But it is critical that enterprises retain complete ownership of their approach to BI without making the mistake of adopting a solely tactical approach that builds in limitations for later on. “We have to address the mentality of a tactical problem-seeking approach”, says Neale, who recommends enterprises ignore the softly, softly approach and target the areas where BI will have the largest impact. “This has got to be done to deliver benefit with an eye on the bigger picture. Don’t get too bogged down in process and change control.”

Arguably, the real cost of BI comes after implementation and lies in ongoing maintenance. “It’s a classic IS problem - once the system is implemented, you take your eye off the ball,” says Paul Edge, informatics programme manager, global drug development IS, AstraZeneca. “You need ongoing sponsorship – strategic, management and operational – and you need to make sure the funding is there to keep the thing running.” By definition, BI must be based on timely information updated to reflect the reality of the corporate state of play. “If we don’t fund the evolution of the system it is dead. It loses credibility,” says Edge.

BI deployments should be approached as major, ongoing concerns, reflecting a company’s business processes. Key factors for success include having the support of a top executive to back the project and help overcome cultural resistance. The vision of BI must be all-encompassing, so that the enterprise is working towards a BI implementation that accounts for exclusions and changes, and can support decision making with the best intelligence possible.

The seven fatal flaws of BI

Gartner Group has identified seven fatal flaws in BI implementation:

1. IT departments create data warehouses without business involvement so users don't see the benefit;

2. Management is hiding behind thousands of spreadsheets, many of which could be eliminated;

3. Companies are not addressing the issue of poor data quality;

4. Enterprises wrongly assume a one-stop-shop BI solution is best;

5. BI projects are being designed with inherent limitations, failing to take account of the need for enterprise wide BI to evolve;

6. Enterprises are not preparing to outsource BI even when it is a good strategy for them;

7. Companies are implementing dashboards without a solid BI infrastructure to support them.

Ark Group’s top BI action points

Business buy-in

Executive sponsorship is critical to the success of strategic BI initiatives. You need overt support and real buy-in from the top – someone who is able to knock heads together when necessary.
Conduct regular management updates. You should aim to meet on a one-to-one basis with relevant senior executives. This will ensure that your project is aligned with the strategic requirements of the business.
Train your users and provide ongoing help-desk support – this should avoid overreliance on IT. And don’t forget to train the trainers.
Provide something for everyone – even if you are undertaking a strategic deployment, always try and customise sections in your plan for each department. It is up to you to sell the benefits to each department.
Deliver against business goals regularly, particularly on large, strategic projects. Demonstrating regular wins will help ensure continued support for your project.

Data cleansing and validation

Undertake a data-cleansing and validation exercise before embarking on application development.
If you want to improve data quality you must distinguish between what is ‘good’ (valid) and ‘bad’ (invalid) data. This can only be achieved if you understand the context of data – validity of a data value must be defined within the context in which that data value appears (a business rules approach is recommended).
Understand the importance of consistency, conformity and accuracy with your data – a single version of the truth is critical. Your system is only as good as the information you put in.
All reports should have a similar consistency.

Strategy

Start with a lower-risk tactical BI initiative, before moving on to a bigger strategic deployment. Tactical deployments often produce quick wins with quantifiable results. The lessons learnt from a tactical BI deployment can then be used to build the business case for further initiatives.
Gather business requirements by undertaking a requirements workshop. The workshop should produce a number of business questions that can’t currently be answered or would simply take too long to work out.

BI resources

Licence management – as BI deployments increase it is essential that licence use is tracked to help avoid over allocation.
Avoid tool and report proliferation – establish a strategy to retire old business-intelligence resources. Any cost savings achieved can then be used to reinforce the business case for future BI investments.
Standard reports should not be viewed as just an unnecessary evil. They provide an invaluable benchmark for other ad hoc reports and analysis, and an effective means of validating this information quickly.