Adopt Big Data Technology to outsmart your competitors
Admin Joes July 11, 2020
507
89
Adopt Big Data  Technology to outsmart your competitors
It is generally accepted that big data can be explained according to three V's: Velocity, Variety and Volume. However, I would like to add a few more V's to better explain the impact and implications of a well thought through big data strategy.
Adopt Big Data  Technology to outsmart your competitors
HOW TO DO THIS ?
Big data has a lot of potential to benefit organizations in any industry, everywhere across the globe. Big data is much more than just a lot of data and especially combining different data sets will provide organizations with real insights that can be used in the decision-making and to improve the financial position of an organization. Before we can understand how big data can help your organization, let's see what big data actually is:

Velocity:

The Velocity is the speed at which data is created, stored, analyzed and visualized. In the past, when batch processing was common practice, it was normal to receive an update to the database every night or even every week. Computers and servers required substantial time to process the data and update the databases. In the big data era, data is created in real-time or near real-time. With the availability of Internet connected devices, wireless or wired, machines and devices can pass-on their data the moment it is created.

The speed at which data is created currently is almost unimaginable: Every minute we upload 100 hours of video on YouTube. In addition, over 200 million emails are sent every minute, around 20 million photos are viewed and 30.000 uploaded on Flickr, almost 300.000 tweets are sent and almost 2,5 million queries on Google are performed.

The challenge organizations have is to cope with the enormous speed the data is created and use it in real-time.

Variety:

In the past, all data that was created was structured data, it neatly fitted in columns and rows but those days are over. Nowadays, 90% of the data that is generated by organization is unstructured data. Data today comes in many different formats: structured data, semi-structured data, unstructured data and even complex structured data. The wide variety of data requires a different approach as well as different techniques to store all raw data.

There are many different types of data and each of those types of data require different types of analyses or different tools to use. Social media like Facebook posts or Tweets can give different insights, such as sentiment analysis on your brand, while sensory data will give you information about how a product is used and what the mistakes are.

Volume:

90% of all data ever created, was created in the past 2 years. From now on, the amount of data in the world will double every two years. By 2020, we will have 50 times the amount of data as that we had in 2011. The sheer volume of the data is enormous and a very large contributor to the ever expanding digital universe is the Internet of Things with sensors all over the world in all devices creating data every second.

If we look at airplanes they generate approximately 2,5 billion Terabyte of data each year from the sensors installed in the engines. Also the agricultural industry generates massive amounts of data with sensors installed in tractors. John Deere for example uses sensor data to monitor machine optimization, control the growing fleet of farming machines and help farmers make better decisions. Shell uses super-sensitive sensors to find additional oil in wells and if they install these sensors at all 10.000 wells they will collect approximately 10 Exabyte of data annually. That again is absolutely nothing if we compare it to the Square Kilometer Array Telescope that will generate 1 Exabyte of data per day.

In the past, the creation of so much data would have caused serious problems. Nowadays, with decreasing storage costs, better storage options like Hadoop and the algorithms to create meaning from all that data this is not a problem at all.

Veracity:

Having a lot of data in different volumes coming in at high speed is worthless if that data is incorrect. Incorrect data can cause a lot of problems for organizations as well as for consumers. Therefore, organizations need to ensure that the data is correct as well as the analyses performed on the data are correct. Especially in automated decision-making, where no human is involved anymore, you need to be sure that both the data and the analyses are correct.

If you want your organization to become information-centric, you should be able to trust that data as well as the analyses. Shockingly, 1 in 3 business leaders do not trust the information they use in the decision-making. Therefore, if you want to develop a big data strategy you should strongly focus on the correctness of the data as well as the correctness of the analyses.

Variability:

Big data is extremely variable. Brian Hopkins, a Forrester principal analyst, defines variability as the "variance in meaning, in lexicon". He refers to the supercomputer Watson who won Jeopardy. The supercomputer had to "dissect an answer into its meaning and [... ] to figure out what the right question was". That is extremely difficult because words have different meanings an all depends on the context. For the right answer, Watson had to understand the context.

Variability is often confused with variety. Say you have bakery that sells 10 different breads. That is variety. Now imagine you go to that bakery three days in a row and every day you buy the same type of bread but each day it tastes and smells different. That is variability.

Variability is thus very relevant in performing sentiment analyses. Variability means that the meaning is changing (rapidly). In (almost) the same tweets a word can have a totally different meaning. In order to perform a proper sentiment analyses, algorithms need to be able to understand the context and be able to decipher the exact meaning of a word in that context. This is still very difficult.

Visualization:

This is the hard part of big data. Making all that vast amount of data comprehensible in a manner that is easy to understand and read. With the right visualizations, raw data can be put to use. Visualizations of course do not mean ordinary graphs or pie-charts. They mean complex graphs that can include many variables of data while still remaining understandable and readable.

Visualizing might not be the most technological difficult part; it sure is the most challenging part. Telling a complex story in a graph is very difficult but also extremely crucial. Luckily there are more and more big data startups appearing that focus on this aspect and in the end, visualizations will make the difference.

Value:

All that available data will create a lot of value for organizations, societies and consumers. Big data means big business and every industry will reap the benefits from big data. McKinsey states that potential annual value of big data to the US Health Care is $ 300 billion, more than double the total annual health care spending of Spain. They also mention that big data has a potential annual value of € 250 billion to the Europe's public sector administration. Even more, in their well-regarded report from 2011, they state that the potential annual consumer surplus from using personal location data globally can be up to $ 600 billion in 2020. That is a lot of value.

Of course, data in itself is not valuable at all. The value is in the analyses done on that data and how the data is turned into information and eventually turning it into knowledge. The value is in how organizations will use that data and turn their organization into an information-centric company that bases their decision-making on insights derived from data analyses.
Comments
Rekha
July 27, 2020
Waw, the best brief explaination I have read so far.
Pallavi
December 14, 2019
I love coding in Python. It is much lesser time consuming to code & makes my job simpler.
Sanjana
December 14, 2019
Essay writing is fun, I love writing essays, it helps me to broaden my Vocabulary. I will start writing essays with the given tips in the blog.
Rekha
December 14, 2019
Artificial Intelligence, Machine Learning is the flavour of the season, waw excited to take up the course
John
December 14, 2019
IOT has brought in newer dimension to mobile apps, this article has helped me to know about IoT in a better way
Santhosh
December 14, 2019
Digital transformation has resulted in a tremendous change in the way of working, a very nice article which has helped me to understand the aspects of Devops
Ajay
December 14, 2019
Got to know the important aspect of writing
Leave A Comment
Your email address will not be published. Required fields are marked.