Big data and it's evolving role in shaping our future

Data is everywhere these days, and almost all of us are consuming and producing some type of data on an everyday, regular basis. That tweet you just sent out? Data. Your most recent internet search featuring lists of the nearest takeout pizza? Data. Asking your smart device what the weather is looking like, yeah that’s data too and someone or something will eventually analyze it. Simply put, if you’re connected to the internet and using a WiFi or Bluetooth connection, you’re producing data that will exist until the power runs out and the phones no longer work.

Before you hide from big brother and draw the curtains, it’s important to understand how big data works. Yes, there is such a thing as data brokers, and it is a billion dollar industry that operates on harvesting and selling your data to corporations that use it for their competitive advantages. But these days, the quantity and rate of which we produce data has grown at such a rapid pace that often times the analysis behind our personal data is factually wrong and missing key information.

Where does all that data go?

The majority of data that you consume or create will be stored in massive data holding centers. These holding centers are almost always owned and under the control of the government, a fortune 500 company or a leading global corporation such as Facebook, Microsoft, Apple, or Amazon. The United States hosts over 3 million data centers, but fewer than 10 of those facilities will account for 72% of the country's data. The U.S. government boasts some of the world’s largest data centers, with a handful of individual locations that cover more than 600,000 square feet.

Where your data is specifically stored depends on what device you’re using, the search engine behind it, your current location, and the securities behind your device or cloud. For example, if you’re using Google Chrome or you search anything through the corporations widely used database then that data will be stored at one of their 15 locations across the world, ranging from Council Bluffs, Iowa to Hamina, Finland. It’s important to know that data is not singular in where it travels, and in the majority of cases it will reach a diverse set of holding centers, ranging from the NSA to Microsoft.

Uncle Sam isn’t the bad guy

Believe it or not, the U.S. government is pretty transparent when it comes to data collection. The majority of their centers publicly reveal the data that they collect, which is often times focused on climate control or monitoring global affairs. The data centers that you should be worried about are the ones that are held through private companies and corporations, as they’re the ones that constantly find themselves in controversial, national headlines.

Leading industry corporations such as Google and Facebook use your data to make a whole lot of money by selling your personal data to large advertising agencies.They also use that data for themselves by creating tailored in-house marketing campaigns designed to meet your specific needs, which are mined through smart algorithms running throughout their data processing centers. Uncle Sam doesn’t pimp out your data to the world’s highest bidders, unlike the makers of your beloved smartphone and online streaming service.

Amazon and Facebook are perhaps the two least transparent corporations that store and mine your everyday data usage, as they refuse to reveal exactly what they store and to whom they are selling it to. Users of Amazon’s Echo should be careful of what they say to the smart device, as the company has yet to disclose where the data is stored and who has access to it, leaving many people concerned that the company is selling voice recordings to the U.S. government or private agencies offering the most money.  

Investment of Big Data continues to skyrocket, but positive results are lacking

In the early 1990’s computer scientist John Mashey coined the term big data through a series of academia articles that explained how important of a role big data will play in shaping our future societies. The term referred to data that would be so large and expansive that our current tools would not be able to properly store, manage, and analyze the incredibly large data sets. Since then, the term has gained popularity through the rapid growth of technological advancements, and entrepreneurs have capitalized, creating data farming firms that have raised over hundreds of millions of dollars.

The market for big data began to take off in the early 2000’s, when our data usage through social media platforms and the internet of things became so overpopulated that our current tools could no longer keep up with the output, paving the way for future companies to create tools and algorithms specified to meet big data demands. Since the early 2000’s, the market share of big data has increased by a minimum of 10% per year, and is expected to reach 200 billion in 2018 as predicted by IDC. Economic analysts don’t expect the investments to slow down anytime soon, and forecast that the entire market could reach 300 billion dollars as soon as 2020.

The investments have been pouring in, but what about actual results? As of now, big data is one of the riskiest investments to make as an angel investor or company because the evidence of positive ROI after initial investments is extremely cloudy, no matter what Silicon Valley is saying. The majority of fortune 500 companies cannot handle big data on their own, and the data mining firms are so new to the market that there isn’t enough supporting evidence out there to prove that these investments are worth the expensive price tag.

It’s not only private firms struggling. In 2008 Google became heavily interested in big data, and announced its plan to use data sets to predict important crises or epidemics, with their first project (GFT) focusing on tracking and predicting flu outbreaks on a regional basis. The program failed miserably as it failed to predict the outbreak of H1N1 in 2009. Google apologized for the poor results, and pledged to create a smarter system. However, they once again failed to live up to their promised potential, and GFT met its final demise in early 2013 when it once again failed to predict a global flu outbreak.

How big data can help improve our future

Self driving cars. We’ve all heard about them by now, but the majority of us don’t own one. As of now, these advanced machines of transportation rely on what is called partial automation, acting on SAE level 2. What that means in English is that the self driving cars that are currently on the road are using data, but do not fully depend on it. When the magic really starts to happen, is when these advanced cars can rely solely on big data sets to get from point A to point B. In the near future, cars will be able to take real time traffic data to create the quickest routes, and eventually every car will be working through the same system, leading to the possibility of no more traffic jams. LA just got a whole lot more attractive!

The healthcare industry has been a leader in big data investments, as there are a widespread of complex communication and data logging systems that are used by hospitals across the globe. Four hospitals in Paris are currently using big data sets to predict patient traffic on a 24 hour basis, helping them properly staff according to their patients needs. Positive results are already starting to pour in, as the four hospitals are experiencing much shorter waiting lines compared to local hospitals who are not using big data.

Big data can also help improve societal infrastructures by reporting problems before a human would ever take notice. One example of how this is currently playing a role in cities across the globe is through the tracking of potholes by using big data. Just 5 years ago potholes and dangerous road conditions were all tracked through paper, but now large cities like Chicago and Kansas City are using real time big data to improve their cities infrastructure. The city of Chicago placed sensors on all of their public service vehicles, which would relay important information back by tracking bumps or sensitive areas in the street, allowing them to track dangerous road conditions without the driver having to ever step out of the vehicle.

Big data could potentially create more jobs

Perhaps the biggest worry within big data and artificial intelligence is that it will terminate millions of employment opportunities. While this is true, it’s important to consider how many jobs it will create in return. The firm Gartner predicts that by 2022 AI and big data will create more than 2.2 million jobs, exceeding the 1.8 million jobs it will replace. However, the report did mention that the shipping and manufacturing industry will see a total net loss in employment, but this is due to the automation and artificial intelligence that will soon hit all major shipping corporations.

Titles such as Chief Officer of Data, or data miner did not exist in the early 2000’s, and college degrees that focus on big data will continue to rise as more companies begin to see the value in understanding all of their data. However, one thing to keep in mind, is that the jobs created through big data will be highly skilled and trained professions that require years of higher education, which could lead to societal problems because the jobs that big data and AI are replacing are lower-level labor jobs that do not require master degrees.

In the end, it’s impossible to predict if big data will result in a positive or negative effect to society as a whole. But one thing is for certain, which is that the role that big data will play in our everyday lives will only increase as time goes on.