As 2011 drew to a close, the IDC team of business analytics analysts worldwide put together a list of ten stories that in our view had and will continue to have a significant impact on the business analytics market. The ten stories include:
1. Business analytics technology sales surpassed $92 billion
Organisations demonstrated an increased willingness to invest in improved decision-making capabilities. In 2011 the business analytics technology market, which includes software, hardware, and services for data warehousing, query, reporting, analysis, advanced analytics, content analysis, analytics spatial information management, and analytic applications, surpassed $90 billion in worldwide revenue. IDC predicts this market will reach $100 Billion either by the end of 2012 or in early 2013. Business analytics is being driven by the need to support or automate decision making processes that can be positively influenced by the analysis of ever growing volumes of multi-structured data.
2. Cloud based analytic applications became the new must-have addition to the portfolio
Acquisitions of specialised software vendors in 2011 pointed to the desire among large vendors to expand their cloud-based offerings. Examples included IBM’s acquisition of DemandTec, SAP’s acquisition of SuccessFactors, and Deloitte’s acquisition of Oco. These acquisitions of relatively small vendors were made with the eye toward the future when a portfolio of industry- and/or process-specific analytic applications with deep domain expertise and valuable content will become a source of differentiation. Large IT vendors have tried but largely failed to market their own portfolios of analytic applications. This was partly because it required a go-to-market strategy that required sales and marketing to divide their attention among many small market segments and partly because even the largest of vendors did not have enough internal domain expertise to tackle highly specialised analytic applications. The new strategy is to acquire and manage a portfolio of specialised analytics and seek development and administrative “synergies” where possible. We expect this trend to continue as large vendors look for niche analytic application vendors in untapped market segments as new acquisition targets.
3. Oracle acknowledged that RDBMS can't solve all problems
At Oracle Open World 2011, Oracle announced the Oracle Big Data Appliance, which is an engineered system. The Oracle Big Data Appliance includes an open source distribution of Apache Hadoop, Oracle NoSQL Database, Oracle Data Integrator with Application Adapter for Hadoop, Oracle Loader for Hadoop, an open source distribution of R, Oracle Linux, and Oracle Java HotSpot Virtual Machine. The Oracle NoSQL Database is a distributed, scalable, key-value database. We single out this announcement among hundreds made by business analytics vendors during 2011 because it highlights the acknowledgement by Oracle, the bastion of relational database technology, that RDBMS isn’t appropriate for all information management workloads. The announcement is therefore likely to have a significant positive impact on the adoption of NoSQL databases alongside RDBMSs.
4. The database solidified its position as the foundation of business analytics solutions
At the same time as the industry’s attention shifted to non-relational information management and analysis technologies, the database vendors quietly continued to enhance their products ensuring that relational database management technology will continue as the foundation of business analytics solutions for years to come. Features such as in-database advanced analytics, in-database OLAP, in-database data transformation (ELT), columnar data orientation, hardware/software optimisation, and automation of database and system administration functions, among others, were the foci for RDBMS vendors in 2011.
5. Microsoft abandoned LINQ to HPC (Dryad)
As the rest of the world was learning about, experimenting, and deploying Hadoop (and related technology), Microsoft was busy developing its own set of technologies for support of data-intensive computing applications that run on a Windows HPC Server cluster. The technologies included LINQ to HPC, formerly known as Dryad (a high-performance, general-purpose distributed computing engine). In November Microsoft announced its decision to discontinue work on LINQ to HPC. A post on the Windows HPC Team Blog stated:
In line with our announcement in October at the PASS conference we will focus our effort on bringing Apache Hadoop to both Windows Server and Windows Azure. Hadoop has emerged as a great platform for analysing unstructured data or large volumes of data at low cost, which aligns well with Microsoft’s vision for its Information Platform.
Microsoft’s decision to embrace Hadoop is another indication of the momentum of this open source technology and the community that is developing around it. At PASS, Microsoft announced that there will be Hadoop integration in the forthcoming release of SQL Server 2012. We expect more news from Microsoft on this topic and a means of addressing the needs of .Net developers tasked with Big Data projects.
6. IBM Watson clearly demonstrated the strong potential of question answering based on analytics
On Feb 14th, 15th, and 16th, IBM's Watson challenged Jeopardy's top winners, Ken Jennings and Brad Rutter, to a 3-day round of Jeopardy. Watson won. IBM invested four years of work by scores of IBM Research computer scientists, not to mention a significant amount of hardware and off-the-shelf, open source, and custom software, to create Watson. Watson is "a massively parallel probabilistic evidence-based architecture" for answering questions. In Watson IBM combined deep NLP with machine learning, a voting algorithm, a method of interpreting the questions and assessing them by formulating parallel hypotheses, and Hadoop and UIMA for pre-processing, as well as the usual search, fuzzy matching software and an in-memory caching system to save time in retrieval. The strength of this system is the combination of all of these capabilities.
At the end of 2011, IBM announced that it will be commercialising Watson to address real world requirements for its technology. At the same time IBM challenged other vendors to combine technology components in new ways to create game-changing applications.
The broader theme surrounding the IBM Watson is the entry in 2011 into the unified access and analysis technology market of additional large IT vendors. At IDC we refer to unified access and analysis technology as the software that combines the capabilities of management, analysis, and access of structured and unstructured data. Historically, the technology for handling each of these data types has developed along parallel paths. In 2011 HP bought Autonomy and Vertica, and Oracle bought Endeca, to merge unstructured and structured information into a single modular analytics/search/BI platform on which to build new types of applications. Finally, a critical mass of large vendors exists to move the market forward.
7. Business intelligence appliances emerged
Over a decade ago the business analytics appliances market started out with an exclusive focus on data warehousing, Netezza and Teradata defined a market that now includes products from EMC, HP, SAP, and Oracle. 2011 marked the year when the appliances expanded to incorporate BI tools. Examples of these BI appliances include Oracle Exalytics, IBM Smart Analytics System, HP Business Decisions Appliance (developed with Microsoft). We expect broad availability (and demand) for highly specialised appliances for process and industry-specific analytic use cases. In addition to on-premises adoption, many of these appliances will power cloud-based analytic services.
8. Decision management surfaced as a more mainstream topic
Before business analytics and before business intelligence there were decision support systems (DSS). Along the way the word decision fell out of favour in the marketing circles of related technology and services vendors (as well as among industry analysts). In 2011 that trend seems to have reversed. Large vendors such as FICO, SAS, IBM, and SAP as well as specialists such as Sparkling Logic, Phasics, and Lumina brought their decision management solutions to the front of their messaging priority lists. Some of these vendors had been marketing decision management solutions for years, but the increase in the number of vendors and the frequency of mentions of decision management marks 2011 as the new beginning for the decision management market.
9. The gamification of the business analytics market began
Gamification is all the buzz in several consumer focused industries, but the concept and practices are also finding their way into the enterprise software market. Gamification includes the use of game design techniques to engage audiences. According to the Wikipedia definition Gamification works by making technology more engaging, by encouraging users to engage in desired behaviours, by showing a path to mastery and autonomy, and by taking advantage of humans' psychological predisposition to engage in gaming. In 2011 we saw some gamification techniques, specifically focused on collaborative decision making and visual discovery, begin to show up in business analytics software. Vendor such as QlikTech, Tableau Software, TIBCO Spotfire, Kaggle, and others are likely to push the boundaries for incorporating more gamification functionality within business analytics technology.
10. The shortage of analytics experts shot up in importance
Applying predictive or descriptive analytics to big or small data requires a set of specialised skills that are not always available for many organisations. A third or organisations reported in 2011 that the lack of staff with appropriate analytical skills is a significant challenge. To address this skills supply shortage new specialists will need to be trained worldwide. In response to the universal cry for more math and statistics skillsets combined with business acumen, universities have begun to offer courses and degrees in business analytics. Some of the examples now include North Carolina State University’s Institute for Advanced Analytics, which has started offering an MS degree in Analytics. Northwestern University’s McCormick School of Engineering offers the Master of Science in Analytics degree and the Master of Science in Predictive Analytics. Villanova now has a Centre for Business Analytics at its School of Business. DePaul University offers a MS degree in Predictive Analytics. For some future graduates the ultimate career goal will become the position of the chief data scientist - a role that emerged in the headlines of 2011 analytics stories.
It will be years before these and other similar higher education efforts start paying off, but industry and academia are making the right decisions and working together to address market demands while promoting their own self-interests.
Finally, here are two unofficial entries to the list of 2011 significant business analytics events:
a. Although the movie Moneyball highlighted the value of analytics in baseball, lead actor Brad Pitt failed to displace Tom Cruise as the Hollywood celebrity most associated with analytics. The latter’s 2002 movie Minority Report had, according to IMDB twice the box office ($102 vs. $50 million) and grossed almost five times more ($353 vs. $74 million).
b. In 2011 every end user conversation I had about BI or analytics included some mention of end-user self-service. How to achieve it, how to support it, how to enhance it, and how to control it were all topics of lively discussion. End-users want answers to their questions at the speed of thought. Therefore, we nominate Queen's 1989 "I Want It All (I want it now)” as the 2011 theme song for the business analytics market.
Entry posted by Dan Vesset