A recent book, the Age Curve has received some buzz in pointing to using a life stage approach to understand a macro trend and its impact on the micro-economics of specific business. The author observes that fluctuations in the populations of age ranges may affect total demand. The size of the fluctuations does seem to exceed the rate of inflation or even more compelling factors influencing demand. The size of the Generation X population is less than Baby Boomers and less than Generation Y. One strategy the author touches on includes placing less focus on the Generation X consumer—presumably in that the same investment in marketing at a micro-economic level will show fewer returns if it has the same relative impact, due to a smaller audience.
In a contrarian view, there seem to be some mitigating factors to note. There also seem to be other strategies that can leverage Gen X, even with the fact that Gen X’s population size is 20% less than the mean combined population of the Baby Boomer and Gen Y segments.
First, look at the effects of the current American recession. In theory, the shopping behavioral effects might strike harder at those on a fixed income vs. a discretionary income, on those retired vs. unemployed. Those closest to retirement (Boomers) are ones watching their 401K balances, as they correlate with the actual retirement ages impending for this group. They’ve seen their balances drop precipitously. And while one might think unemployment and financial challenges may have struck equally across age groups, it’s sensible to think that current and recent college graduates and impending retirees would be the most affected at a macro level. The economic phenomenon would be expected to make the newly retired Boomers even more frugal spenders. Net: the impact of the recession is likely felt—across the age spectrum—more starkly for Boomers and the graduating Gen Y population. If this impact dilutes the likelihood of spending by a relative 5% vs. Gen Xers, then one has the first mitigating factor.
Second, remember that the U.S. Boomers in retirement are reaching the age to qualify and use Medicare, and that the fastest growing use of Medicaid is Boomer nursing home care, and you have further reinforcement of money management as a daily style of living (note: consider different marketing techniques one might have to employ to attract a ‘tour bus’ of shoppers trucked from an assisted living facility vs. consumers with discretionary shopping travel routes). With healthcare legislation in the U.S. likely to increase the cost of healthcare for Boomers (regardless of politics, it’s unlikely that healthcare will be cheaper for any segment other than niche groups), even less will be spent on discretionary purposes.
Third, people are living longer. The average lifespan increases regularly. This means the Boomers will have to continue to recalibrate their annual expenditures to make their life savings, investments, and annuities last longer. They’ll spend less each year as a result. In effect, this dampens the spike of the Boomers. The Boomers are acting like a population with less purchasing power, because of the increased lifespan.
These factors are likely to dampen the variability in population segments. Even so, there may be some ‘lumpiness’ to age segments. So what is a brand company, retailer, service company, or any business-to-consumer company to do? The most important factor is to increase product breadth to cover a wider age range. Imagine a cross-section view of water waves moving toward the beach. With a very thin slice in view, the up and down motion is very stark. But with a wider lens, it’s clear that the same volume of water is moving across that lens. The company with a broader product mix in effect can leverage the total volume, and leverage higher sales of products targeted at the population mix that is high at that point in time. The added advantage is that fluctuations in purchasing are not completely age-driven. A broader mix will have hedging effects helping manage any spikes.
While it may be true that increased growth can come from ‘listening to your core customer’ and being more precise with that group, a better strategy for consistent earnings over time is to spread your demand across different ages, races, countries and other dimensions—you can’t ‘ignore’ any segment without risk.
Product Portfolio Management (PPM) is a discipline that originally emerged in part from the famous Boston Consulting Group (BCG) matrix that helped corporations decide how to best manage the economic lifecycle of their lines of business or products. While thinking about products through the lens of the simple BCG matrix is valuable, extending the PPM definition and process has value. Unfortunately, in the domain of consumer products and retail, PPM is most often associated with making decisions around new product introduction and product lifecyle management. PPM has been co-opted by innovation choices and has veered off its natural path far too early and missed the value inherent in its roots as a management, not tactical, tool.
It should become standard to extend the definition of PPM to absorb all the value and experiential learning of the portfolio management discipline in finance. As such, today’s consumer products and retail definition of PPM should be more like this: The management practice of seeing a company’s set of category or brand products as individual constituents of a single portfolio that has risk, hurdle rates, a role in the store and the household, and is a chief instrument in delivering the shopper strategy of the company, including new product introduction and rationalization decisions.
Today’s consumer products company or retailer can become dependent on PPM as a tool for innovation and PLM, and loses a much bigger benefit in the process. In fact, a more holistic approach is appealing, if only because a new product-centric approach to PPM puts less focus on existing products, corporate goals, and the short and medium-term realities of sales, supply chain, collaborative retailer-vendor business practices, product assortment, and the range of disciplines involved in filling a place in the shoppers’ lives. Unfortunately, it is the rare consumer product company or private label retailer who has actually put a cross-functional PPM team in place that is thinking strategically about the product mix without the mandate of using these learnings solely to decide the new products to launch. To corporate governance, the value of such a team or business practice can be copied almost verbatim from a list of benefits of actively managing an investment portfolio. Classic Portfolio Strategy is as applicable to the strategic and outcomes-based goals of managing a product portfolio as to successfully identifying which products should be released (or retired) next. In fact, taking a set of best practices from any reasonable investment portfolio management handbook may extend the value of the traditional consumer products and retail PPM practice. The typical business values of portfolio management can all be realized when a portfolio theory approach is adopted within PPM: reducing risk through diversification; fine-tuning levels of aggressiveness or conservatism; leveraging unique economies of experience or scale to realize higher than market returns in specific areas of the market; and realizing strategic goals using the portfolio as a whole as a delivery mechanism.
One can imagine PPPM for the consumer products executive, brand manager, category manager, or their peers on the retail side, engaging in three levels of PPM, which may build on each other:
- New Product Strategy and Introduction. The basic de rigeur approach or even working definition of PPM.
- Strategic Delivery. Alignment of a portfolio mix of products as an instrument to broader company financial, strategic, and branding goals.
- Functional Optimization. A practice of measuring and tuning impact on all areas of the business resulting from product portfolio decisions, including manufacturing, sales, supply chain, and execution at the store.
While this layered and more expansive approach should be and is not yet practiced in the market, there is something much more transformative at hand that ideally should completely change these practices. That is, instead of corporate financial goals, marketing initiatives, cross-functional realities and even competitive responses, the chief driver of the process—reversing it from end-point to starting-point—is putting a shopper-engineered PPM process in place with the household as the driver of risk, demand, mix, price, brand, usage, and long-term relationship dimensions.
- Healthcare . In this visualization, GE took 500,000 records from the millions in its electronic medical record database, and calculated the out of pocket and insurer cost of a handful of chronic conditions by age. One of the few both uses of and displays/designs of a radar graph, the visual changes as one increases and decreases age. This also has a predictive analytics component in that it answers a user’s question “What if I develop a case of hypertension, what will that cost me when I’m 65?” Like any good analytic, being able to see the data brings up more actionable and specific questions that this analytics doesn’t answer but the data set could.
- Digital Media. This visualization is part of The New York Talk Exchange, a visualization project developed by the Senseable City Lab at MIT. Perhaps the potential applications of the analytics are as explosive or more than the specific data they used in this case. The analytic shows starting or sourcing neighborhood within NYC and to where their communications via the AT&T network were destined. Users can see the frequency distribution of endpoints, and by comparison who was talking to whom geographically across sister boroughs. Imagine creating a site map of your web site, application, team or workflow, and seeing the frequency of where the user, function, business process or capital goes next. Predictive analytics can say: “If we invest in area x where is that capital, profit opportunity, or waste most likely to go next, and how does that change if I make another investment?” Perhaps only this specific visualization is best for that type of comparative predictive analytic.
- Retail. According to Well Formed Data, Sankey Diagrams and stacked bar charts informed this (4MB .pdf download) time series visualization of how medical journals in related fields merged into a cohesive ‘basket’ of journals in the emergent field of neuroscience. While on the surface not retail related, it points to a very compelling—and as yet a visualization I’ve never seen produced— which would explain how specific products that drive volume and profit affinitize into specific types of market baskets. Replace a) each journal with a specific retail product, which the user can color code at run-time for visualization, perhaps color-coded for on/off promotion, b) the eigenfactor of each journal which is represented by the weight/width of the specific line with the amount of profit or volume the item produces—again the user can choose profit or volume or some other measure at run time, c) the ten or so portfolio of ending blocks of journals as specific types of market baskets, and d) the breadth of starting lines moves from medical disciplines to aisles or departments in a store. In effect what you have here as a decade long time series one has compressed into a single, specific shopping trip. Data can include many trips, a single store, one day or years of data. The predictive angle is to be able to answer questions like “If I promote this item, does it move away from its core ‘7 items per basket, quick trip’ basket into a ‘destination item weekly stock-up’ basket?” One can also look historically how different store consumers shop categories, volume vs. profit items, and more.
These predictive analytic visualizations start with healthcare as the least complex and become more complex. Depending on how frequently the user needs the data updated, the amount of data (in the digital media/telco example clearly tens or hundreds of gigabytes), the processing (in the retail example, calculating eigenvalues in an analytic that could be analyzed hourly, especially for promotional out-of-stocks, for example), and the speed required means a predictive analytic visualization is not something to try at home with an off-the-shelf database platform and hardware.
In the mid 1990’s, hearing about someone with a 1 terabyte data warehouse (DWH) was a sort of mystical, illusory event, engendering doubt or even suspicion as being a ‘fish that got away’ story. The person telling the story was never the one who actually built the DWH, they were just exposed to it in some way, and they threw the story around as if it was nothing, loving the awed look on the faces of their audience. Invariably this would be someone from the Information Technology (IT) field, since the business users would be unlikely to know, care, or be surprised that a very large amount of data is needed to answer their questions. So the IT person would also carelessly throw out a rejoinder such as ‘You know, at that size, you can’t simply [insert technique IT people do every do with a ‘normal’ large DWH].’
Fast forward a decade. Today, terabyte+ warehouses are common. However, one hears the same stories with one small difference: replace the word terabyte with petabyte . A petabyte, at 1000 terabytes, is a seemingly unreachable stretch of data. However, as we all witness the increasing power of processing and decreased cost of storage, we seem to be seeing enough examples of PB+ warehouses to say, “yesterday’s terabyte is today’s petabyte”.
Before you get a petabyte DWH, you need a petabyte of operational data. When a petabyte of data is present to ‘run’ your business, only then can someone say ‘we need to analyze all this data’. Today’s petabyte-operational business is much more likely to be communication or information based. For example, AT&T reported one year ago that “AT&T currently carries about 16 petabytes of total IP and data traffic on an average business day”. (With our log scale growth in storable communication, presumably it’s on its way to doubling…) Other companies with petabyte businesses include Google, all the major telecommunications companies, all the major web businesses—digital media and telecommunications. It’s nice to know the exception that proves the rule is the PB+ data collection at the Large Hadron Collider.
In a recent conference, a member of Facebook revealed the accelerating growth of their DWH. They reported that in March 2008 they were collecting 200 gigabytes (GB) of data per day. In April 2009, they were collecting 2+ TB of data per day, and in October 2009, they were collecting 4TB+ data per day. If you chart this, you see something approaching a classic logarithmic curve. While Facebook reports its DWH is closing in on 5 PB today, by the time a reader is absorbing this sentence, it has likely long surpassed that.
Does this mean in 2020, more than half of the Fortune 100 will have petabytes size data warehouses? Probably not. However, they’ll all have TB+ warehouses, and a herd of businesses will be PB+:
• All large and mid-size digital media, social media, and web businesses
• Large and mid-size telecommunication firms, driven by their Call Detail Record databases
• Financial market-based companies (think of tracking all stock market transactions to the microsecond level of granularity), and more and more bricks and mortar companies (e.g. banks) who have done as little as dipped a toe into financial markets, social media, streaming communication, and the like.
• Large energy companies recording all seismic and atmospheric ‘communications’ to a very specific latitude/longitude
• The energy grid will be getting close. It’ll be likely that cars are talking to the grid to reduce congestion and to enable metered driving in the fast lane, so chances are the cars will be talking to each other spitting out signals every second. Just like that, we’ve added another 100M four-wheeled ‘people’ in our country communicating and someone will want to analyze it.
And, you know, when your car’s antenna is a source for an exabyte data warehouse, you can’t just change the wiper blades, you have to……
In the business intelligence world, there’s less focus these days on the ‘big’ commercialized logical data models. The Wikipedia link here has only a single external reference: an article written fourteen years ago. Conversely, there has been a consistent acceleration of the number of commercial off-the-shelf physical data models by industry and application type. The reason is simple.
Physical data models are required for BI applications to work.
Logical data models are like appetizers: they should be cheaper than the main meal, small, and take less time to eat. It’s when vendors try to sell you a ‘with such a big appetizer, why do you need an entrée?’ that you should look for the door.
The best blog post—and perhaps the only one—of the year on the LDM concept is by Dan Linstedt. Dan tries to create a fair and balanced view, but his cons list is longer than the pros.
His cons:
- Consulting organizations don’t often understand how to implement properly
- They have been mapped 1 to 1 to the physical (without necessary changes)
- The models are super typed, and often times are too high a grain for customers to really grasp ALL the elements that are combined within
- The super-types try to handle too many levels of “grain”, mixing corporate customers for example with individual customers and changing the outcome of the queries on the reports
- Often require years of expertise and training to implement properly
- Sometimes are a nightmare to tune, partition, and get performance from
- Sometimes require tons of staging areas to “prepare” the data before integrating into supertyped table structures
- Nearly always need “extending” and customization, but don’t fare well with customers, as this becomes a multi-year, high cost effort – turning into a huge consulting services gig for big dollars.
- “I believe that if you “customize” the logical data model, that you will have trouble when the vendor puts out a new version (that alters the version you have in place). I’ve seen cases where a 2 and 3 year bid effort to customize becomes a 7 year services contract in order to maintain customizations across multiple releases.”
Many of Dan’s pros can be viewed as negatives. For example, “Easy to Buy vs. Build” he lists as a pro. In the world of IT, just about anything is easier to buy vs. build. If it weren’t, whoever is commercializing the buy option would go out of business. The trade-off on a buy build is usually cost and time to market. Since Dan acknowledges that except in rare cases these LDM’s take long to implement (and as you can guess, anything where a 2-3 year consulting effort to implement it has to be expensive) then a buy model doesn’t make sense. It may be easier to buy a coffee table than build one, but if your coffee table to build is from IKEA and the only one you can buy is an 1850 antique in moderate condition, it’s a no brainer.
My colleague and BI expert Mike Roney had some measured comments about the relative value of LDM’s vs. PDM’s:
“LDMs can engulf a large number of subject areas in a short period without doing any of the hard work associated with instantiating anything. At the cursory level they feel very complete and don’t box me into anything. That can feel compelling with a higher audience and wraps me in a warm blanket of safety. However, when the rubber hits the road it becomes a very challenging task to materialize a broad LDM for my uses. It’s really ground zero, looking up at a considerable amount of work to implement anything. PDMs on the other hand, especially once they are coupled with ETL and reporting IP, will be attacked as rigid and inflexible to my “special” circumstances. They do enjoy the development speed and build vs. buy arguments but struggle with flexibility. This is where a properly positioned GAP analysis would come into play. It does however provide a basis for development to begin and a strong message toward rapid development. Working backwards from a PDM to a LDM is not a challenging thing and can be done in collaboration with a prospect as a catalyst to gain mind share and understanding of their considerations. I would not go so far as to call LDM obsolete as they are used very effectively all the time.”
Fortunately for end-users looking for BI, a few things have happened since LDM’s started appearing and vendors attempted to use them as differentiators:
- IT groups were buying BI because of user pain and hunger, not to define a long-term strategy. BI reports, dashboards, alerts and queries needed data in tables to run: they needed physical data models. With a physical data model (required), a tool like MicroStrategy or SAP Business Objects can be implemented. For less strategic or time-sensitive projects, LDM’s were either a nice-to-have, or they were an unnecessary expensive addition: ‘bull horns on a Cadillac’.
- Theoretically, a relational database scientist might have an ordered thought that the tables and fields should flow from a physical model which should flow from a schema which should flow from a logical data model. Unfortunately, this ‘top-down’ approach is not how BI systems are built (or how wars are fought). In reality, different pockets—where-ever need and budget are greatest—of user need is addressed first. Doing the LDM ‘step’ in the process was one of the quickest to throw out when timelines went awry. In reality, the end users with pain and budget can list out their requirements in Excel, those can be tied to specific metrics as well as the data to build the metrics, and the data itself leads to the model. The relationships between the data (e.g. one-to-many, many-to-many, recursive, etc.) define the best model, sometimes independently of the tool or queries, sometimes in conjunction with them. What works with BI tools, what is fastest, is this ‘bottom-up’ approach. Sometimes a quick LDM can work—I’ve seen some of the most effective be documented and used in PowerPoint and Visio.
- Then there were some vendors who went off on a side track, expanding their LDMs and positioning them as if they had unique technical value. These LDM’s quickly became some industry expert’s Jackson Pollock brain-dump of every possible subject area in that vertical business. The ultimate ‘science experiment’ is the scene where technology vendor subject matter experts unroll their LDM like a maid throwing open a sheet about to make a bed, and then stooping over it next to an analyst end-user, saying “Do you have this in your business? Do you have that in your business?”. At the end of the day, that model was never used. Why? Because 70-90% of it was irrelevant for the few subject areas needed to solve today’s business pain. And the rest was useless because the information needed to set up hierarchies, attributes, levels, master data and reports were all instantiated in Excel (unfortunately), Access, and other previous reports or systems—at the granular level.
The final straw is the argument the vendor with an LDM will give you on why being so extensive is such a positive. “You can just delete what doesn’t apply to you.” While this sounds nice—it sounds like buying a car with every option and you can cancel it once you find out you don’t need it—in reality it’s more like buying a set of encyclopedia’s when you’ll be doing one school report a year (and you have the Internet, and the vendor’s going to force you to read, ahem ‘skim’, the entire set of encyclopedia’s so you can throw away, ahem ‘recycle’, the rest of the volumes). A better model is what everyone else does when they need a subject area: buy a single book on that topic or find the same on the Net. A set of encyclopedias is a thing of the past: so are Logical Data Models. On the other hand, find a vendor with a tight physical model that’s easily understandable and you have something that can speed you up.
Building on the basics of Shopper Insights, there is a rush of hype about social media as the source of next-generation shopper insights. Not so fast. While social media is a) plentiful, b) telling, c) real-time, and d) insightful into motivations, it is also e) not tied to specific items generally. Consumer product brand companies (makers of everything from light bulbs to TVs to toys to cheese) and retailers managing their private label efforts become interested when the consumer insights can be tied to action—and actionability always comes back to a specific product. For example, it might be nice to know that a shopper ‘likes Store 100’, it is much more actionable when the shopper says she ‘likes Store 100 because they always have her XL size of Aroma Jeans’. Category-level and especially item-level insights work. Generalities fit into a landscape of interest to market research, but because they sense general trends, they don’t always provide enough focus to influence specific decisions.
So what do market research, brand teams, and anyone digesting shopper insights really want? They all want insights to be four things:
- Constant. They want a stream of insights that is constant, they don’t need to initiate. A good example of something constant is a market research bureau report—it shows up every month or season once you’ve subscribed. An example of something not constant that needs to be re-initiated is a consumer survey.
- Searchable. The users of insights want to be able to pile up the insights and search the shopper quotes and insights by terms like retailers, product names, price, feature type, and other actionable variables.
- Speed of Business. The timeframe for actionability to make a change relevant is within a week. Ideally, the consumer of shopper insights will identify a relevant insight within days of it happening, digest it and translate it into the right meaning and action, and act on it at the ‘moment of truth’—the point of purchase—or with some other engagement with the shopper. An example of something that’s not speed of business are those market research bureau reports. They may arrive consistently, but might be providing insights about next year’s fashion or shopper events that occurred two quarters ago. An example of speed of business is feedback that comes to a customer service deck in a store.
- Reflects My Shoppers. A huge amount of social media—in fact most of it—won’t have anything at all to do with my shoppers. In fact, finding my shoppers in the deluge of social media data is finding a needle in a haystack. The good news is that there are tools today that can find needles. The problem is that there aren’t enough needles (more on that shortly). An example of a channel for shopper insights that reflects my shoppers is a survey focus group. While not consistent nor real-time, it definitely reflects my shoppers as I can hand-pick the attendees.
One can easily see that it seems many of the current channels for shopper insights don’t hit on all cylinders. In fact none do. And while social media is constant, real-time, and searchable (but not reflective of my consumers), there’s not a lot of mass there. The number of consumer insights documented is low. This is due to the venue. People on Facebook and Twitter are not focused on writing product reviews. Once in a great while someone will write some product commentary, but it’s so rare as to not be useful. It’s so rare that it doesn’t reach any statistically significant n—it could be a trap to ‘generalize from a sample of one’. Surprisingly, there is a better way. And one company in Virginia is quietly talking for the first time about using its technology to mine for shopper insights in the unlikeliest of places.
The ideal shopper insight is driven by the shopper. The most engaged shoppers actually want to talk to the retailer and manufacturer about their opinions, experiences, frustrations, and tastes. Until now, no one’s really been listening for shopper insights within that noise. Where do consumers go when they want to talk? Yes, some go to places like The Consumerist, and some go write a review if the product can be reviewed—but if you’re selling something other than books, movies, or electronics, the number of reviews you’ll get is slim. The most common place, by far, shoppers go is the email forum and the customer service phone line from the manufacturer. The brand companies get hundreds of thousands of calls a year. That’s right. Hundreds of thousands! For a single product, the brand company may get ten thousand calls a year, vs. three product reviews on a web site. I’d put my money on a weighted average and a sample of insights from the call center rather on the three reviews. In this case, social media—while cool—just doesn’t have the volume, not even a sliver, that call center email and phone call data does.
The main beauty of the call center data is that it’s constant, speed-of-business, and it reflects my consumers—it is my consumers! The company mentioned previous makes it searchable. (Fair warning: Clarabridge is a partner of Netezza). Clarabridge has found that this call center data is rich. The callers mention specific products, what retailers they shopped to purchase it, and are happy to answer questions about their age and other demographics—because they want to engage. Their area code is captured, as well as more detailed (i.e. mine-able) data. So what do you want to act on: market research that’s a month old for a segment of the population that is mostly not my shoppers, or yesterday’s quotes from specific shoppers. The call center data is not coupon seekers—those calls can be filtered out of the insight analysis.
The clear challenge here for brand companies and retailers is that instead of spending time looking for needles, it’s like getting a carton of needles in all colors. Suddenly you need a) a very large database, and b) a way to get out all the red needles today, but all the blue ones tomorrow. That processing has to be done at run-time…when the question itself is being asked about the product at hand. Since you already own the call center data, the investment isn’t in purchasing other peoples surveys or opinions, it’s investment in processing power to sift through the wealth of insights you already own and that keep coming.
While call center data isn’t glamorous, or handled by marketing today, it’s the best source of actionable, fresh shopper insights.
One of my favorite quotes from Star Wars was Han Solo saying “”Well, if they follow standard Imperial procedure, they’ll dump their garbage before they go to light-speed. Then we just float away.” Today’s retailers and brand companies often have a product portfolio that is hampered with ‘garbage’: unprofitable products that serve little value to consumers, take up valuable shelf space, are unnecessary in the assortment, and have failed. The business process for ‘dumping their garbage’ is called SKU rationalization or product rationalization, and it’s a business process that has not taken advantage of software coming on to the market today that hits the inflection point between low cost, high profit, and repeatable intervention. While a more systematic, process driven approach might reduce the amount of product ‘garbage’ coming in from new product introductions, and have more active harvesting activity, so often companies find themselves very over-assorted and in need of winnowing many products out of the mix in a serious way. They find themselves in a ‘dump their garbage’ mode in mid-flight. This sku rationalization process is far too often done too crudely (with a crayon) or too expensively (with a laser) when the best approach is for a middle-level investment (with a highlighter).
The prevention model is perhaps where the highest ROI can occur. Unfortunately this methodology is very hard, rare, and impossible to pull off consistently. Reducing the number of unprofitable new product introductions is the best approach. A slew of Product Portfolio Management software solutions and consulting offerings have been introduced of late—unfortunately they have ended up focusing on new products and not helping with sku rationalization efforts for an existing too-fat assortment.
Almost every retailer is in the situation of needing intervention. In fact, even if prevention were twice as good on the part of brands and retailers, intervention would be needed. There’s no way to predict changes in shopper preferences, cultural and sociological trends, fashion, and competitive product introductions that invalidate a product’s role and impact. Surprisingly, very few sound approaches exist.
On the simple side is a far-too-common ‘let’s rank our products by profitability and cut the bottom 5%’. While this is sometimes better than sticking with the current assortment, it can have negative impact on consumers. An even graver reality is that the product profitability on which the ranking occurs is often incorrect. Very few companies have consistent activity-based costing in place, nor do they have basic product costing in place so they can look beyond cost of goods sold into a relative product costs that takes into account a series of very findable back-end costs. By putting a little more effort and investment into getting costs together can make all the difference between a ‘slash and burn’ simple approach, and an elegant cut. On the opposite side of the spectrum are rationalization projects by consultancies such as Bain (example here) or the Corporate Executive Board (example here) that can cost $10M and beyond.
What is really needed is a middle ground. What about an approach that is an owned process and asset so it’s repeatable, not a one-off consulting project. And what if it is more elegant than ignoring a better product costing. Just this approach is available using a mix of data collection assets and standard business intelligence software. A company should be able to use their owned software and develop a more sophisticated rationalization approach, including a business process that’s right for them. Or a company could use an off-the-shelf software designed for product rationalization—one of the best is offered from Pi Solutions (fair warning: Pi Solutions is a partner of Netezza). The ROI return for item rationalization is crippled in an expensive management consulting approach–the cost of ‘dumping their garbage’ outweighs the profit; the ROI investment is too small to reach a return in the do-it-yourself ranking approach–the babies get thrown out with the bathwater. The right approach is a middle-ground: using business intelligence, data mining, and business process in a repeatable, sensible way.
In the 1990’s, very soon after the widespread expansion of standalone business intelligence (BI) software implementations, professionals began talking about extending this power to operational applications. Operational applications were front-office, back-office, horizontal, ERP—they created purchase orders and instructed plant machines to start and stop and automated customer communications. All of these software applications that automated and made companies run suddenly became the loci for future instantiations of BI. And the promise was that once BI was integrated into the very business processes operationalized by this software, that the optimization and smarts would happen automatically.
The promise was not immediately realized. At first, some believed it was because the operational software providers didn’t ‘know BI’. However, after a time, these business process software modules included a spectrum of BI modalities. Everything from hard-coded SQL, to high-science algorithms, to OEM’ing top BI platforms such as Business Objects or Cognos. No one could say these tools had not stumbled on the ‘right’ way of integrating BI into their workflow. Even today, we have yet to see the promise of possibility latent in the fact that operational software is helping companies act, but without organized data-based intelligence outside of the pre-coded ‘rules’ of the human operators.
Have we finally gotten it right? Indeed, after all this time, Gartner in 2009 said in discussing their BI Magic Quadrant:
“Areas that have traditionally been under corporate performance management (CPM), such as business planning and forecasting, are increasingly being embedded with BI capabilities. This, together with a trend of embedding analytics into business processes, will drive further investment in BI.”
Gartner went further, with a vision that by 2012:
“…business units will increase spending on packaged analytic applications, including corporate performance management (CPM), online marketing analytics and predictive analytics that optimize processes, not just report on them” (emphasis mine).
If we’re still not there after all this time, and the vision has been obvious more than a decade ago, then there has to be one or more significant barriers to this happening. Clearly the barrier is not simply technical. My hypothesis is that the largest barriers are:
a) Managers and analysts like the idea of a black box giving them an answer, but not the idea that the answer will be used—without their intervention—to do their job for them. Professionals would still rather type in the number of widgets to be ordered or destroyed, the amount to budget for x, y, or z, and the number of spin cycles to run the toothbrush vat. Workers love the idea of BI laying out the answer for them, but it’s as if we still want to copy the right answer onto the test, rather than letting the robot take the test for us. Maybe it’s because it’s us—not the robot—who gets stuck with the grade.
b) Our communication culture at work and a continuing dedication to transparency at low levels of grain means the chance that the professional will have to answer the question ‘Why?’ is very real. One can imagine being in a meeting and being asked: ‘We always order one truckload of tomatoes. This week you ordered two. And we had so many left over. Why did you do that?’ Imagining our answer as ‘I let the computer decide’ doesn’t have a nice ring to it.
What does work in these situations is having a crib sheet at hand—BI output—that influences the worker to make better decisions. And this is what some of the more innovative BI applications have migrated toward. There are beginning to be applications that support business processes. They don’t give you a BI palette and ask you to envision and create a picture as much as they say: you have to paint this room in your house, and here are the questions you will have or ought to have, and their answers. Instead of giving you a servant asking ‘What should I do’, they give you a vacuuming robot, a making-coffee robot, and a start-the-fire robot.
An example of a software application with this approach is QuantiSense (fair warning: Quantisense is a partner of Netezza). QuantiSense invented what they call ‘Playbooks’. Playbooks are just specific work-flows that any retail merchandiser, planner or allocator is or should be doing anyway. The Playbooks define the workflow, the process of the workflow and the points within the workflow that could be optimized if a BI process were inserted at just that point. As a result, the regular analyst can run the ‘plays’ you might see on Monday Night Football rather than something closer to a little kid quarterback saying ‘Go Long…’ as the extent of their vision to the right play. This sitting of a standalone BI app next to the professional and their operational app, with the BI tool coaching them through a workflow—rather than giving them some output—seems to be the only method that will get today’s workers around the barriers that have plagued knowledge workers for a decade.
One of the hottest buzzwords in consumer and retail marketing is ’shopper insights’. Companies such as Nielsen and IRI have been working with shopper insights for many years. But not until 2009 has the term shopper insights become so mainstream. Walking through the headquarters of any large retailer or consumer product company, one might stumble upon a Shopper Insights Center of Excellence, or a Customer Insights Department. As a mark of its stature, the group—for the first time—may even sit outside the watchful eye of the market research department.
So what is a shopper insight? Wikipedia tells us:
Unilever defines a shopper insight, an insight upon which shopper marketing is based – as a “focus on the process that takes place between that first thought the consumer has about purchasing an item, all the way through the selection of that item.”
As far back as 2004, the Hartman Group defined Shopper Insights as: “That which is necessary to properly understand the role of the shopping experience with regard to purchase behavior (in specific) as well as brand loyalty (in general).”
Performing a search on monster.com for ‘Shopper Insights’ jobs brings up 76 roles from a blue chip list of companies including Johnson & Johnson, Dannon, Wyeth (Advil, Robitussin), IRI, Kimberly Clark, MARS, Cadbury and Sara Lee. Changing the search term to ‘Customer Insights’ brings up a list of 410 jobs, with more retailers represented, such as Home Depot, ShopRite, eBay and Walgreens.
As plentiful as jobs, conferences, white papers, and more, there are painfully few examples of shopper insights sitting around for general and free consumption. Companies—professionals with ‘insights’ on their business cards—are paying millions of dollars for these insights. With such value, contemporary insights rarely find their way to the Information Superhighway. Nevertheless, a sample of shopper insights might include:
a) Unacculturated Hispanics do not buy/buy fewer beverages when they eat at a quick-service restaurant
b) In ‘The New Frugality’, Mom has taken on more responsibilities and bears the brunt of deciding, for example, no Disneyland vacation this summer.
c) Regarding ‘green’ shoppers: “For most shoppers sustainable considerations become a tie-breaker, when other factors are in relative parity. Because of this effect, sustainability characteristics drive a relatively large amount of product switching. Once a more sustainable product has captured the shopper’s commitment it tends to create brand stickiness by retaining the shopper’s loyalty through repurchase.
All the details point toward a shift away from managing a portfolio of products that managers believe shoppers would want to select from (e.g. Category Management), toward identifying shopper motives, needs, and behavior patterns and understanding the mix through the eyes of a portfolio of consumers. More traditional customer segmentation enabled us to learn the language of different income, psychographic, or ethnographic clusters. But satisfying needs and watching changes in shopping behaviors will give the retailer or brand manufacturer the signal when to start engaging with their customers, not only with the right language, but even more critically—at the right time.
In May, 2009, North Korea conducted its first nuclear test in three years. While testing ballistic missile range and capability are more tangible for the press and public, the underground nuclear test is a more specific marker that a state is progressing toward usable nuclear weapons. Since 1963, the only marginally acceptable way to test nuclear weapons has been underground. While there does not seem to be conclusive evidence that underground tests have a negative health impact, it is clear that underground tests are a marker for serious risk of global stability indexed by the weapons capability of less stable states. An hour before its recent underground test, North Korea notified the U.S. and Beijing the test would be occurring. If no notification had arrived, would developed nations have known the test occurred? With today’s technology, one would think identifying the presence of a nuclear detonation would be easy. Not so.
The volume of recordable seismic activity is beyond enormous. One seismic survey might total terabytes of data. Ideally, a monitoring country would be leveraging existing third-party seismic data acquisition sources or even owned monitoring stations. The concept of looking for patterns in trickle-feed or real-time data streams is a petabytes-per-day challenge. Add to this the advent of 3-D or 4-D models and the data sizes increase exponentially. Add to the general wealth of seismic events the additional human events such as aircraft impacts or building collapses among others. Together, it’s a high data volume challenge, with data continuing to present, needing pattern recognition and modeling using historical data as a baseline, with a national security risk urging fast recognition so action can be taken in a meaningful time frame.
The Lawrence Livermore National Laboratory summed this problem up succinctly:
Nuclear tests are no longer frequent. However, there are 30- 40 earthquakes of magnitude 4 and greater every day — about 10,000 per year. A magnitude 4 earthquake releases energy on the order of a one-kiloton nuclear explosion. Identification and location of the rare, and possibly covert nuclear test, within the cacophony of natural and man-made background seismic activity, is a major national security scientific challenge that NNSA and its labs are in a unique position to meet.
(Fair warning: Lawrence Livermore National Laboratory is a Netezza customer)
Between 1945 and 1975, six countries conducted nuclear tests. A twenty-year period followed where global security prevailed and no countries emerged as additional nuclear states. In 1998, the first of three additional countries prevail, and many believe Iran will soon be the fourth. A simple historical histogram juxtaposed with the increased flow of information, communication, and development—and the failure of the contempory preferred state action of prevention being diplomacy—that additional nuclear countries will emerge in the current cluster of emerging nuclear states. As such, seismic monitoring for underground tests—and the likelihood that these tests will not be communicated in advance or at all—becomes critical for global security. For better or worse, the data warehouse technology that can and used to serve businesses cannot handle the imperative for sensing global security risks. Whether to goal is diplomacy or action, states need terabyte-throughput that only leading edge technology can deliver if the goal is mining seismic data to identify underground nuclear tests.
