Merging Business Intelligence with Business Process

In the 1990’s, very soon after the widespread expansion of standalone business intelligence (BI) software implementations, professionals began talking about extending this power to operational applications.  Operational applications were front-office, back-office, horizontal, ERP—they created purchase orders and instructed plant machines to start and stop and automated customer communications.  All of these software applications that automated and made companies run suddenly became the loci for future instantiations of BI.  And the promise was that once BI was integrated into the very business processes operationalized by this software, that the optimization and smarts would happen automatically.

The promise was not immediately realized.  At first, some believed it was because the operational software providers didn’t ‘know BI’.  However, after a time, these business process software modules included a spectrum of BI modalities.  Everything from hard-coded SQL, to high-science algorithms, to OEM’ing top BI platforms such as Business Objects or Cognos.  No one could say these tools had not stumbled on the ‘right’ way of integrating BI into their workflow.  Even today, we have yet to see the promise of possibility latent in the fact that operational software is helping companies act, but without organized data-based intelligence outside of the pre-coded ‘rules’ of the human operators.

Have we finally gotten it right?  Indeed, after all this time, Gartner in 2009 said in discussing their BI Magic Quadrant:

“Areas that have traditionally been under corporate performance management (CPM), such as business planning and forecasting, are increasingly being embedded with BI capabilities. This, together with a trend of embedding analytics into business processes, will drive further investment in BI.”

Gartner went further, with a vision that by 2012:

“…business units will increase spending on packaged analytic applications, including corporate performance management (CPM), online marketing analytics and predictive analytics that optimize processes, not just report on them (emphasis mine).

If we’re still not there after all this time, and the vision has been obvious more than a decade ago, then there has to be one or more significant barriers to this happening.  Clearly the barrier is not simply technical.  My hypothesis is that the largest barriers are:

a)      Managers and analysts like the idea of a black box giving them an answer, but not the idea that the answer will be used—without their intervention—to do their job for them.  Professionals would still rather type in the number of widgets to be ordered or destroyed, the amount to budget for x, y, or z, and the number of spin cycles to run the toothbrush vat.  Workers love the idea of BI laying out the answer for them, but it’s as if we still want to copy the right answer onto the test, rather than letting the robot take the test for us.  Maybe it’s because it’s us—not the robot—who gets stuck with the grade.

b)      Our communication culture at work and a continuing dedication to transparency at low levels of grain means the chance that the professional will have to answer the question ‘Why?’ is very real.  One can imagine being in a meeting and being asked: ‘We always order one truckload of tomatoes.  This week you ordered two.  And we had so many left over.  Why did you do that?’  Imagining our answer as ‘I let the computer decide’ doesn’t have a nice ring to it.

What does work in these situations is having a crib sheet at hand—BI output—that influences the worker to make better decisions.  And this is what some of the more innovative BI applications have migrated toward.  There are beginning to be applications that support business processes.  They don’t give you a BI palette and ask you to envision and create a picture as much as they say: you have to paint this room in your house, and here are the questions you will have or ought to have, and their answers.  Instead of giving you a servant asking ‘What should I do’, they give you a vacuuming robot, a making-coffee robot, and a start-the-fire robot.

An example of a software application with this approach is QuantiSense (fair warning: Quantisense is a partner of Netezza).  QuantiSense invented what they call ‘Playbooks’.  Playbooks are just specific work-flows that any retail merchandiser, planner or allocator is or should be doing anyway.  The Playbooks define the workflow, the process of the workflow and the points within the workflow that could be optimized if a BI process were inserted at just that point.  As a result, the regular analyst can run the ‘plays’ you might see on Monday Night Football rather than something closer to a little kid quarterback saying ‘Go Long…’ as the extent of their vision to the right play.  This sitting of a standalone BI app next to the professional and their operational app, with the BI tool coaching them through a workflow—rather than giving them some output—seems to be the only method that will get today’s workers around the barriers that have plagued knowledge workers for a decade.