What is BI?
“Business Intelligence is a collection of technologies, applications, processes and practices for the transformation of business data into business useful information.”
There are a variety of other definitions, but in essence BI is not a single – product, principle, method or technology, but rather all of them.
Over past few decades, enterprise information levels have grown substantially. According to Forrester (Data, Data Everywhere, July 2007), the world produced 5 exabytes (5 with 18 zeros) of data by 2003. Most of this information is generated by the transactional systems (such as ERP, CRM or Legacy) and these systems are not capable of processing high volumes of data in order to generate useful information. This is where a BI solution becomes important. A BI solution, also referred to as a “stack”, typically, consists of various components and technologies. See Figure 1 below.
A BI solution helps to transform raw data into actionable information which helps support business decision making. As examples, inventory levels could be dissected across products and vendors, thus triggering not only a reorder process but also assisting the managers in identifying the supply and demand trends and the efficiency of vendors. Expenses could be sliced across departments, time periods and drilled down to the level of individuals thus providing a more meaningful analysis for cost cutting. Opportunity reports could be run across various customer segments thus helping in identifying premium customers. In summary, BI is a technologically organic entity which supports business critical decision making capability by leveraging an aggregation of tools, methodologies, people and processes not only to improve the efficiency of the operations, but to improve the effectiveness of business processes and in turn improving the ROI.
In today’s age of a shrinking global economy and increased competitiveness, traditional enterprise systems and solutions which increased the efficiency of the transactional systems are reaching a point of flat growth. This is primarily due to the following reasons:- 1) Matured technology and products making innovation challenging 2) Pre-packaged solutions and business processes across all industries, which standardizes growth and efficiency while diluting the “business edge” factor 3) Generation of extremely large amounts of transactional data, but inefficient and siloed reporting on the same.
Through a successful BI initiative, an organization can benefit in the following ways:
- Extract actionable data from the extensive volumes of business transactions.
- Identify profitable customers, reduce costs, and identify profitable products, services and trends.
- Provide an environment which helps in refining or building business processes, planning strategies by applying results from the BI system/solution which make an organization more competitive.
- Provide users with a platform to run customized reports on-the-fly (ad hoc analysis)
- Eliminate the reporting inertia which results from vast amounts of transactional data and reporting silos resulting from multiple systems within an enterprise.
- Provide a single decision support system for the disparate transaction systems that exist in organizations today. This also helps in standardizing the presentation of information.
- Track internal and operational performance within the organization. Evaluate performance on regional, local, departmental and individual basis.
However, before any organization can realize the benefits of a BI solution, they need to first understand the business questions that they are trying to answer and address the business challenges.
BI applications have evolved from being mere reporting engines of data previously locked away in transactional systems, to providers of information analytics that can be utilized by enterprises to gain competitive advantage.
Although BI applications deliver insight into business processes and help align corporate strategy with individual performance, they are faced with plethora of challenges at various fronts which negatively impact end-user trust and adoption.
Some of the key challenges that hinder the hastened adoption of BI initiatives are enlisted below:
- BI Metrics Management – The lack of common data definitions across different departments and/or geographic regions of the same organization render it impossible to gain a single, consistent view of business. One of the world’s leading direct-selling companies for example has at least 5 different definitions for demand and net sales across marketing, finance and other operational departments and geographies. It is imperative to establish a process to help define and reconcile semantics across the organization with minimal impact on current systems.
- Disparate Data Sources – The one constant rule about enterprise data is that it’s always changing, be it due to internal re-organizations or due to mergers & acquisitions. As a result, IT must continuously be prepared to deal with disparate data that comes from heterogeneous data sources.
- Data Quality and Data Governance – In an increasingly competitive market, BI vendors have made pervasive BI and enhanced predictive analytics a reality. Even so, BI implementations have been continuously plagued with back-end data quality issues culminating in the failure of BI projects.
Most transaction systems are faced with data quality issues which can be attributed to a number of issues like incorrect data entry by field sales representatives, existence of multiple records of the same customer coming from different legacy systems, empty fields (such as missing contact information), and redundant or inconsistent data between two data silos to name a few.
A robust and effective three-fold process, namely data profiling, data cleansing and data validation needs to be set-up to address this concern.
- Data Lineage – “Where did that number come from in the report?” is the most common question asked by business users. The answer to this question can be derived by accessing the BI application and ETL tool metadata together with the source data.
The challenge is that every data source, BI tool, and ETL tool contains its own metadata and they do not talk to one another thereby rendering answers to such questions time consuming and cumbersome. Such queries must often go through as formal BI service requests.
By using a metadata management solution to consolidate and integrate all BI-related metadata into a single location, IT can view, analyze, and explore metadata from disparate systems. A metadata management solution enables IT staff to understand the context of information in their BI environment and the underlying relationships between metadata objects, data structure, end-to-end impact analysis, report-to-source data lineage, and operational statistics.
Once the organization understands the challenges, it needs to define a roadmap in order to approach the BI implementation in order to address these challenges. An important decision that an organization would make at this point is selecting a specific BI solution and vendor.