What is Your Big Data Story?

Posted by on in Data Modeling & Architecture

Bigdata

I've been speaking, teaching and ranting on big data and NoSQL technologies recently. I've noticed that when I chat with many data modelers, I've met with a lot of skepticism about big data.

You may be that "guy" if you've ever said: • It's just mainframe all over again.

• It's a fad to get out of data modeling tasks

• It's not something I need know about

• I don't have big data, so I don't need to know NoSQL

I think what's missing from that thinking is the fact that modern data architectures use technologies that are the best fit for solving a data "problem". I like to use the term "data story" instead of "problem." It's not that the newer technologies are replacing traditional relational database systems; they are complementing them.

We've Seen This Before

I'm experienced enough to remember when the big controversies were brewing over whether or not to build denormalized data structures to support reporting and data warehousing. What eventually came out of those conflicting points of view was a good understanding that we need to optimize structures for consuming data differently than we do for creating and maintaining data. I see no difference between that optimization of a data story than the use of NoSQL and big data technologies to solve the stories for those data uses. The industry tends to lump these all under the term "big data", but I don't believe all the data stories in this area are applicable just to high volume datasets.

If a modern data architecture makes use of a variety of technologies', then a modern data architect/modeler needs to understand what those solutions and provide modeling services for them.

So instead of asking are you team SQL or team Big Data, you should be asking "I have this data, which technologies should I be looking at to tell its story.

What is your big data story?

Tags: Big Data


About
Gold User, Rank: 13, Points: 268
Digital Smartieskirt & Architect, Consultant | Speaker |Microsoft MVP SQL Server I love talking about data, data management and data quality across all kinds of platforms. Love Your Data!

Comments

  • Sue G2555
    Sue G2555 Tuesday, 18 November 2014

    I do feel that the term "big data" is really misleading. When you ask an "expert" there appear to be as many definitions and descriptions as there are for customer. Which leads me to agree with you that when we hear someone saying "I have Big Data", surely we should be replying - "OK, so what is it you want to do with this data you have?" There are many tools and technologies that can and should be able to handle massive amounts of data that has been structured differently to our traditional RDBMS's ... depending on knowing what the outcome should be is key to understanding and selecting the right kind of tool/ technology for the task. I wouldn't cut a birthday cake with a double handed Viking axe, I'd use a chef's knife. The same rule must apply!

  • Karen @DataChick
    Karen @DataChick Thursday, 20 November 2014

    I love that analogy. Now I need to find a suitable axe to carry around.

  • Joey Dantoni D2316
    Joey Dantoni D2316 Tuesday, 18 November 2014

    Interesting column. I do feel like the paradigm is very similar to that of data warehousing. It's not always a high volume scenario--in my experience it's where things aren't a good fit for a traditional relational system for whatever reason (weird data, pure size, or rate of change) and need another solution to work them an ideal fashion. One system I tried to migrate tracked all network events for a large enterprise.

  • Karen @DataChick
    Karen @DataChick Tuesday, 18 November 2014

    I think all of that is why I see people using "big data" and "NoSQL" almost interchangeably. Data with multiple formats don't do well in RDBMs no matter what the size.

    I so wish both communities (big data and NoSQL) could some up with less misleading labels.

  • Please login first in order for you to submit comments
  • Page :
  • 1

Check out more tips and tricks in this development video: