Business Intelligence Projects – Lessons Learned

Business Intelligence Projects – Lessons Learned

Business Intelligence Projects – Lessons Learned

“The general who wins the battle makes many calculations in his temple before the battle is fought. The general who loses makes but few calculations beforehand.”

I’ve always enjoyed the lessons of Sun Tzu, at university his book the “Art of War” helped me to complete my thesis applying the ever growing subject of Information Warfare to Business Strategies – endless fun and learnings!  That was a different and exciting time for many reasons!

Of course, I’m not going to go into the ins and outs of my thesis, rather discuss and impart some of the experiences, that I have had within the ever growing subject of Business Intelligence or Big Data or Analytics – dependent on what you and yours call it.

Sun Tzu was alluding to the fact that those who have the ability to plan in advance, are more likely to succeed than those that don’t.  The calculations that are being framed above refer to the combinations of the plan or effort required to overcome the issue – in Sun Tzu’s sphere “war”.  The more experience you have with what has worked and what hasn’t worked, means that you can apply this knowledge and experience into the planning activities in any project – whatever you are attempting to deliver.  The calculations may have happened in many different situations, or for that matter in many different companies, so learning to apply those things that work within the terrain similar to where you have been is important.

That’s all very well you say and what’s that got to do with Business Intelligence, Big Data or Analytics projects?  Well, in my humble opinion, it has a lot to do with how well you plan and the successful outcome you plan for.  Looking back over the years that I have been delivering Business Intelligence projects, there are a number of learnings that I take with me into new customer projects, different industries, different cultures and a diverse workforce.  I never stop learning, there is always a new day where new or similar issues arise within projects, whether they are attributed to people, technology, process or data.  Knowing how to sidestep these issues will save a lot of time and moolah!

This article explores some of the lessons that I have learnt on my journey through Business Intelligence projects.  These are my 5 areas (in no particular order), there are plenty more, but I haven’t the chance to write a tome right now, so I will keep it short and sweet.  You may want to add and share your lessons learned to my list, and as ever I welcome your comments, from both the business and technology angles.

So let’s get started:

Prioritise, Prioritise, Prioritise!!

Did I mention “Prioritise!”  Way too often, requirements are collected, specifications / wireframes for dashboards / reports are generated and then development starts.  Once done, data is sourced and in some cases where I’ve seen it happen, the data isn’t available!  The consequence of this is that you have done all this work to define and make things look nice, however, it’s the data that will make the front end, not the cool amazing UI design!  Most of the clients we work with, tend to work in a more iterative manner, whereby, we understand the requirements that the business would like to achieve with a BI tool, and based on their requirements, start to analyse each and look if there is the data to support.  If not, then a priority is put together which follows this pattern:

  • Priority 1 requirements – the data is available and the requirements can be met – perfect! That means that these are the first requirements that will start to be delivered.
  • Priority 2 requirements – the data will be available but not for some time as perhaps there is an interface that needs to be defined, and more in-depth work needs to be undertaken so these requirements will be fulfilled when the interface is in place. So we can start doing some planning for that and this also forms part of the roadmap. Touché!
  • Priority 3 requirements – the data isn’t available in the business to support specific requirements, so it either has to be externally sourced or will be introduced to the business when a new system is in place. So again, this forms part of the roadmap to get that data in

Doing the above, cuts out the noise of failing at delivering value with BI.  You can only create a dashboard if you have the data, and to boot that is of good quality.  Otherwise, money will be wasted, resources demotivated, and scepticism will be rife as I’ve seen in many companies!  If a roadmap of requirements vs. data availability isn’t created, then you need to ask yourself some tough questions, and how you know that your priority is based on value and benefit to the business.

Plan plenty of time for creating, reviewing and confirming key business rules / metrics

I can’t stress the importance of this activity.  Many companies have complex requirements when it comes to the business rules that are applied to business processes.  Whether you do this in an agile or waterfall method, make sure that you are creating the right business rules with input from the right stakeholders.  That these business rules are backed up by business process maps and sufficient calculations.

Fundamentally, if you are building user stories and going from the agile angle, then make sure you are generating the correct business rules and these are then calculating the right metrics that track what is happening with a denominator (or anchor) that you can hang the calculation off.  I would also go as far as to say that you should start testing the business rule with real data, and not made up data on what you think the target data set will look like.  You can do this asap, agile allows for this approach and you can fail fast.

If using a waterfall approach, then make sure the requirements are set in stone, and once you have collected everything you need from the business and before you start development, check-in with the business to make sure things haven’t changed, and things still remain the same.  There is very little room for failure here, otherwise you have wasted a lot of months working on something that could fundamentally change.

This may sound like an odd one to choose, however, there are companies that we have rescued from making this mistake.  Building business rules and not testing them sufficiently, has caused companies to restart, and this can significantly impact their project, their costs and successful outcome.

Buying Technology for Technologies sake – Understand what you need!

One company we recently supported had a plethora of BI tools that were being used all over the business.  WOW!  It’s as if, someone had their trigger finger depressed on the corporate credit card!  Don’t do it!  Someone has gone out to the market, looked around and said, well we need one of those, one of those, and those, and those, oh and don’t get forget some of those too!  It needn’t be that way!

Please don’t buy technology as it’s the new shiny nice piece of kit that everyone is speaking about.  Different use cases need different technologies.  Not always will you need a dashboard when someone needs a monthly view of data, or if someone wants to predict which customers will buy their products.  Different use cases for different types of roles will need different technologies, in most companies that are fairly large, I have seen a mix of a dashboard technology, some form of predictive capability and some form of static reporting application.  In most cases, these can definitely fulfil most of the user requirements that will emerge.  Don’t go overboard with technology.

By role I mean that the CEO will need to see an executive dashboard that reflects their 5 key metrics to get a pulse on the business, the direct reports to the CEO will need deeper dashboard capabilities for root cause analysis, and those in a tactical position may need dashboards just for their area of the business or perhaps static reports.

This mainly happens, when different departments go for different kinds of technologies, for one reason – there is no strategic view of BI or analytics technologies.  So align with your technology colleagues and vice versa – if this isn’t done then project success is deemed to go down the drain!  This leads me onto my next point.

Business meet Technology – Technology meet Business – now be friends!

This may sound really odd, and you might be thinking well this never happens with our company!  Here is the situation – someone in technology says hey let’s build the BI technology and then deliver it out to the business and see if they will use it.  The business gets their hands on it and says what is this! I never asked for this or that.  Then all hell breaks loose.  I’ve even heard an outside consultancy advise their customers to take this very strategy. Before, I get too many technology people rushing down to the comments section, bear with me it’s not over yet!

There is another scenario which I’ve seen too!  The one where the business ignores technology, goes out and buys their own technology thinking it will be the solution, hires an external consultancy to develop it, said consultancy leaves, then the business expect technology to support the application – and again all hell breaks loose!

Both in my opinion are at fault, and funnily enough, I recently wrote an article about the Chief Data Officer and how this person can bridge the gap.  If technology and business can get along and define what is required from a business point of view to support the business to achieve its goals, then that’s the right way to go.  Equally, the business needs to place trust in technology to come up with the right technology weapons to fulfil the business strategy.

Love the strategy, start small, and deliver quickly.

Why go for a “big bang” approach when what is needed is to create a “proof of concept” with real data, and then build, iterate and learn.  These days, BI projects don’t need to last years, now they can be started fairly quickly and finished in weeks. In most cases, and this goes back to my prioritise view of the world, if you are able to get the business rules / metrics done, calculations written, data sourced and validated, then you are able to build dashboards (etc.) in small iterations in no time at all.  I have seen this done in weeks, where on a Monday the business rules are signed off, on the Tuesday the application is started to be built by the development team and a week later we are already in test.

This isn’t difficult to do, it can happen like that, and we BI professionals shouldn’t be thinking like builders when a customer says how much and how long will that take – we start rubbing our chins and say “ooh well, there’s this and that, ooh and that.”  We can lay down a fairly quick strategy to portray what can be done, prioritise it and start to chip away at the beginnings of a very fruitful BI application that will serve the business.

Remember we don’t need to complicate things (there is enough of that around), we are here to make it easier for the business to get what they want, so taking an iterative or agile process to get there is perfect.  Why?  Because the business sees the results quicker, they get to give their feedback immediately, and things can be fixed even quicker.  It’s almost like instant gratification!  You know sometimes, when you do take this approach, you may not end up with Utopia, you may actually get a realisation from the business users that the data they thought was great is actually really dirty data – in need of fixing and can be done very quickly.  It can also add to changes in behaviour and culture.  One customer noticed this within their sales system, and it made the sales force even more conscious of how they set data up in their systems, that would later be used for analysis.  In the long run this proved beautiful, why, because the sales team essentially became advocates of clean data and its benefits, as well as being data champions for the new data governance policies.

These aren’t the only areas that we have seen out there, however, from a joint business and technology view these are the more common themes that I see where things do tend to go wrong.

Thanks for reading my post and as always please leave your thoughts in the comments section.

If you’re working on a project and you think it’s relevant and valuable to pick my brain, please get in touch by email or call me +44 (0) 845 056 8753.

Samir Sharma – analytics specialist. Helping organisations cut through the complexities of data, and succeed with their data projects.