What are the priorities for cios and ctos these days? what are the technologies to invest in today and what are the ones to keep an eye on for the future? v ramkumar, partner at cedar management consulting, summarises the findings in four key components.
The ability to derive insights from enterprise data is a key determinant of effective decision-making in business. However, over the last two decades businesses have not made sufficient investments to develop this capability. The focus of technology investments has been on functional capabilities developed in silos by specific applications. This has resulted in a fragmented data landscape plagued by the following issues:
- Disparate source systems leading to duplicate, inconsistent data
- Limited and inconsistent data cleansing and harmonisation
- Rudimentary reporting with no data analytics to support decisions
- Multiple captive MIS teams leading to operational inefficiency
- Local data sources and silo reporting is a risk to data integrity and security
With rapid technology improvements in data storage and processing capabilities, data warehouse (DWH) platforms have significantly improved in delivering return on investment. Businesses are examining their current data management landscape and recognising the need for a data warehouse. However, one does not need to look too hard to find projects that have not achieved their desired objectives. The root causes of these failures can often be tracked back to faulty design decisions. This article outlines key data warehouse components and the factors that should drive DWH design decisions
Data warehouse – journey from data to insights
Independent of the nature of business, data is transformed into insights through four stages. Data from multiple sources is extracted, transformed and loaded into a single data repository. Data is sliced and diced across multiple dimensions and business insights are delivered through a reporting tool.
Design Decisions in datawarehouse
For each layer of the data warehouse, the CIO needs to carefully examine the various design options and determine the best approach. The options available to the CIO are given below:
Industry standard data model
Standard data models have significant in-built wisdom around the reporting needs of various user groups and thus can accelerate the implementation of the data warehouse. Industry standard data models are preferable if the reporting needs of businesses are standard, and the application landscape is homogenous.
Data model pre-built by service provider
If the reporting needs of businesses are specific, an industry standard data model will need to be adjusted significantly to meet their needs. This will undermine the case for a standard data model. Such businesses are better off deploying a custom data model or a pre-built data model from a DWH implementation partner.
Custom built data model
If the level of standardisation of requirements is really low, businesses are better off investing in a custom data model. The choice of custom data model is also driven by the need for unique insights that are not available through standard data models.
Standard DB on hardware
The conventional approach is to implement data warehouses in standard database and hardware If real-time analytics are not required by the business and the transaction volumes and querying requirements are relatively low, this approach will help optimise the investment.
In some cases, for instance when real-time analytics and Big Data are involved, performance of DWH queries are limited by the constraints of underlying hardware. The “appliance” model of integrated database and hardware is much more suitable for these requirements. With the increasing adoption of mobile channels, social media and other real-time pricing engines, the appliance model is the preferred option for largescale DWH implementations.
Stand-alone applicationsFor complex and specialised reporting requirements, dedicated analytics engines will be necessary. These engines could either be part of the DWH platform, or be separate stand-alone applications. The case for stand-alone applications is stronger when the nature of reporting requirements vary frequently due to factors such as regulations.
Analytics embedded within DWH
For less complex requirements, the procedures can be embedded within the data warehouse platform. However, this choice is suitable only if the analytics requirements are likely to remain constant over a period of time. Changes in analytical requirements could require re-design of the data warehouse, risking the integrity of the platform.
Finally, the role of the implementation partner is critical for the success of the programme. The implementation partner takes end-to-end responsibilities for the following activities:
- Extraction of data from source systems
- Data quality assessment and cleansing
- History data migration from ODS
- Data warehouse Installation
- Data model customisation
- Database and query performance tuning
- Report development
- New core banking system source integration
- Installation of environment
- Post go-live support
- Project and change management
Given the wide range of responsibilities, the implementation partner should be chosen by careful consideration of track record, solution approach, implementation capabilities and specific team resources who will be deployed on the programme. A data warehouse strategy initiative can help businesses make these decisions and set the right foundation for their DWH programme.