The Business Analytics industry is facing a new level of maturity borne out of a need to improve the ability to complete projects that deliver real value to the business off the back of a global recession.
3-2-1 Data Warehousing (3 million dollars, 2 years and 1 delivery) is now well and truly dead and a new era of smaller more targeted deliveries is becoming the norm. Early indications show this is a much more successful approach. Certainly my recent attendance at The Data Warehouse Institute’s (TDWI) global conference has shown that the idea of providing true business value early in the process is reshaping the industry.
The ways we build DataMarts is changing with a heavier focus on Agile Development principles found in the Application Development world. Companies are successfully delivering huge value using Agile development to build and evolve their Data Warehouses; which has all but nullified the argument that it does not work for this industry. Take a look at Ralph Hughes work in this area and his comparative results, four times faster delivery at a quarter of the cost with improved quality. We must be aware that going agile does have it’s challenges and although we can use it to deliver Data Analytics projects the Agile methodology needs to be altered to make it work.
Some of the trademark principles do apply however including:
- Agile requirements gathering techniques
- Business involvement throughout the process
- Test driven development
- Time boxing the delivery of components
As a consulting company it is hard to sell the idea of Agile to our customers as it takes a leap of faith and knowledge that getting it right will take time.
We have been successfully using an iterative approach for many years as we have never bought into the 3-2-1 approach. The agile approach has a number of real benefits to it and we have recently updated our methodology to include the best parts of Agile into our iterative approach to make it extremely robust. We believe for the NZ market this provides the value and quality while reducing the risk involved in going purely agile.
The Microsoft SQL Server platform fits this methodology nicely with a set of integrated tools that provide a rapid visual platform to deliver from.