How big is ‘Big Data’?

You are thinking about leveraging data from your internal applications, web/mobile applications or social media to better understand your customers need, but can’t seem to figure out if a big data solution would be a good fit? Then continue reading …

As we all know that data is the new gold, but we seldom ask ourselves whether is this really BIG data? Some IT professionals may classify data as Big data just by looking at the size of the data …”Is it in ‘Gigabytes, Terabytes, Petabytes’, ..?”  The real answer lies not just in the size of the data however, but other critical factors i.e. Velocity with which we receive data, variety or the type of data.

Factors for profiling Big Data 

“Whether you need Big Data Solutions or not?” depends on your need. However big or small the data is,  the data need to be captured, ingested, stored, processed and analyzed to transform data into information. We leverage that information to drive business value. Key drivers to be considered while looking at Big data and I like to remember it as the industry calls it..3 V’s…

1.      Volume (size of the data), which will determine whether the data is big enough or not for a Big data/ Data Management solution.

2.      Variety (Format of the data) which begs the question of the different formats your data is in, which is coming from the several disparate sources of Data. Is the data structured, unstructured (emails, video, images)?

3.      Velocity (Speed) is the rate at which the data is being generated, captured and shared.

If you need to further derive business value from what you are doing today some other questions we could ask would be…

1.      Is the volume, velocity and variety of the data affecting your ability to extract value from it?

2.      Do you have numerous sources of data proliferating, hindering your ability to get business insights? 

3.      If the volume of data is small enough, then why can’t you analyze it? Does it need data integration or data processing work before reporting or distribution?

4.      Is the rate of data capture so quick that the systems are having trouble ingesting, processing and analyzing it all together? 

Reflect on Big Data Solutions 

All of the aforementioned begs the question of the state of your current systems including processes, databases and technologies and their ability to derive intelligence from existing or new data. 

Do you currently use the modern-day solutions that allow systems to easily scale,  are reliable and economical or still use traditional environments in which the data outgrows?

Data has seen an unprecedented growth and it has never been easier to handle data as with the recent cloud technologies like Amazon Web services (AWS). Amazon offers the following solutions for data..

1.      AWS solutions for data capture and storage -Kinesis streams (for streaming real-time data), Simple Storage service (S3- can be used for Log data or any flat file), Dynamo DB (for No SQL storage), RDS (relational Databases service for relational Databases) 

2.      AWS solutions for Processing and analyzing Data- AWS Lambda (to process streaming data), Amazon Redshift (for Data warehousing), Amazon Kinesis firehose (load streaming data into data warehouse for near-real time analytics), Amazon Elastic MapReduce (EMR- transform, cleanse and analyze the data) and Amazon Kinesis Client Library (KCL)

All the above provide easy cloud-based solutions for data and can resolve issues around volume, variety or velocity. Still confused on how to use them or which solutions you need for your growing data? For a consultation-->Contact Us

About the Author

Supriya is an expert Sales Professional, she has been in the industry circles for 10+ years and has worked with many startups. Coaching and Mentoring is her passion and she sees success in others as her success.

 

 

Areas: Sales Channel Building, KPI’s to Startups, Tools Startups can use. Idea Validation.