Courtesy of Intelligent Enterprise With my kids in the midst of their teenage years, I am reminded every day of the delicate job of giving advice. Even gentle observations that don't strike me as controversial--that cake is not a nutritious dinner, for example--can, at the wrong moment, be met with an emotional tirade. "You ate cake for dinner when you were a kid!" my daughter screams, among other jousts questioning my legitimacy. Actually, I preferred Cocoa Puffs, but this isn't about me. Few adults, much less teenagers, are going to be persuaded by a detailed argument supported by voluminous research that clears up any dispute about the poor nutritional value of chocolate cake. In many ways, the data management community suffers from the same problem. Data management does a lot of things for an organization, but the ultimate value remains its power, through business intelligence tools and analytics, to overturn irrational decision-making and to steer behavior toward actions that bring positive results in line with business goals. Just ask the Oakland A's, a baseball team that used data smarts to overturn conventional scouting wisdom based on misleading statistics, as described so well by Michael Lewis in Moneyball. But it takes more than simply making the data argument. Satisfying users with advice from data mining, which employs advanced algorithms to discover valuable information from data, has "been a challenge," says Amir Netz, SQL Server product manager at Microsoft and a long-time business intelligence (BI) visionary. "Conceptually, it sounds great: Let the machine look at the data, see patterns and make predictions. It's amazing, almost like science fiction. But people can be overwhelmed by data-mining results that are filled with statistical concepts; they say, 'take it away and give it to a professional.'" Microsoft's game has always been to deliver productive technology to the masses. Thus, it has been grappling with how to deliver data mining, potentially data management's most impressive value generator, in a way that doesn't befuddle users and cause them to reject the insight. Netz says the company is focused on enabling developers to build applications that make data mining transparent. Amazon's cross-selling features and other self-service e-commerce sites that use predictive models to tune cross-sell offers while customers are shopping are examples of successfully "buried" data mining. Along with SQL Server's engine-level algorithms, Microsoft's Office developers are adding data mining to Excel "without exposing any concept of what data mining is," Netz says. BI's challenge, shared by all the major vendors, is to provide benefits without expecting too much of users or IT departments. "A lot of growth is going to come from embedding BI in applications," Netz asserts. "But BI has to adjust to the idea that we don't own all the real estate on the screen. We have to work in a very small window and make what BI provides useful without distracting from business processes." Netz believes the community is only at the start of a "long-haul" effort to embed BI and analytics into nearly all processes and applications. The imperative to integrate terabytes of increasingly varied information, often under real-time pressures, will make data management an exciting field for years to come. But as managers step up to the challenge with exuberance, they must also step back and let users determine how data-driven insight fits their world. Then users can have their cake--and eat it, too. David Stodder is the editorial director and editor in chief of Intelligent Enterprise.
