Big Data in the Enterprise

  • Vote This Post

    1
Posted on May 17, 2012 by Chris Lau Posted as Big Data Blogosphere Discussion

In the corporate sector, Enterprise Resource Planning (ERP) and Business Intelligence (BI) were buzz words in the late 1990’s to early 2000. Data Warehousing was a concept that gained ground in that time. By the time social media gained ground after 2005, a massive amount of data was being stored in an unstructured way. With Facebook going public tomorrow, big data is clearly a serious deal in the consumer space. The consumer space is of significance for the corporations wanting to market in social media networks.

After attending a Gartner local briefing earlier this week, the limitations of how corporate data is structured is clear. This is not to say that Big Data will ever replace BI, ERP, or data warehouses. It is to say that Big Data will need to work alongside existing systems.

Gartner weighed in on Big Data, recognizing that unstructured data decentralizes the business analyst function in corporations. The Business Intelligence function is pushed outside of the IT unit to the business unit.

Corporations should anticipate that Big Data projects start as a proof of concept. The cost of Big Data software (Hadoop) is very low. The rising acceptance for open source software contributes to the low cost for Big Data implementation. Large volume, complex, loosely structured data are characteristics that would fit a Big Data project.

The growth in Big Data is illustrated with these recent developments:

  1. Six government agencies in the US will be spending over $200 million to help find ways to store, analyze, and organize large data
  2. The significantly lower costs of Big Data prototypes as compared to BI/ERP/Data Warehouses is creating jobs in this specialty
  3. A Toronto hospital is using big data for the clinical detection of infections in premature infants

Big Data is getting a dose of legitimacy, through the participation of large IT companies. Microsoft acquired Fast Search and Transfer in 2008, making the tool available in its SharePoint. HP acquired Anatomy (although paying a significant premium), a company based in the U.K. Oracle acquired Endeca. Endeca provides tools for managing unstructured data. The company also supports web commerce and BI. EMC has a tool, Lucene Solr, to support text analytics.

Despite acquiring Cognos many years ago, IBM is building its own BI tool for Big Data. The company chose to organize its tool in the format of a tabbed spreadsheet for linked analysis.

Further discussion: where is your company at with Big Data? 

Chris Lau Chris Lau (80 Posts)


Related posts:

  1. Big Data is This $97 Billion Elephant in the Room
  2. What is big data?
  3. Suppose You Didn’t Need Big(ger) Data
  4. Big Data is the start of Systemless Information
  5. Does Your Organization Need Big Data?
Tags:
  • DonSheppard

    Good points.

    Clearly, a search function for big data is necessary.

    Perhaps this is where a cloud solution also works – bursty processing, large amounts of non-private data possibly shared by a community, pay as you go, etc.

    Where on the hype cyle do you think big data is?

  • http://twitter.com/chrispycrunch Chris Lau

    Hi Don,
    The hype for big data is at an early phase. The reason? The best known implementation of big data is with facebook, so the hype is masked by social media, not the data architecture behind it. As IT turns its attention away from advertising on social media, and towards the application of big data in other sectors, then hype will start to build.