08-18-16 | Blog Post
Big data has seen big growth during the past few years. But what is it? Google simply says it’s “extremely large data sets that may be analyzed computationally to reveal patterns, trends, and associations, especially relating to human behavior and interactions.” But big data can never be that simple. Rather than just providing information about people’s beliefs through something such as an election vote, big data also gives insight about people’s behaviors—who they are, who they associate with, the things they buy, and how they act around others. By treating it in a social rather than informational context, data scientists can actually tell more about people, and more accurately. According to data scientist Sandy Pentland, this is because “people are so enmeshed in the surrounding social fabric that it determines the sorts of things that they think are normal, and what behaviors they will learn from each other.”
How can IT take advantage of this growing trend? As more and more devices are connected to the Internet of Things (IoT), (6.4 billion devices this year alone, and up to 21 billion by 2020, according to Gartner) the amount of data transmitted from them will grow exponentially and provide an overload of information about consumers and their behaviors. This means storage and security solutions are needed, which companies recognize and have begun to provide. Easier access to this data allows scientists, companies, and governments to provide better human services, higher quality of health care and be more in tune with the needs of their patients, customers, and constituents. IT companies are already taking advantage by providing analytics about big data. But where is that information kept?
The question of storage has already been addressed: Cloud computing. It’s only growing, and the market needs space to store all of that data. Cloud environments, with their flexibility, ease of use, and scalability, are a natural fit.
Of course, with all of the information potentially available in the cloud, privacy and security concerns arise. Who gets control of all that data, and how can it be kept secure? Some of the challenges big data faces include:
These challenges and others present security problems that are not solved yet. Vendors that use big data platforms such as Hadoop are starting to differentiate themselves based on security. However, researchers have pointed out an alarming lack of concern toward security of Hadoop and other platforms.
The possibilities for societal advancements are endless thanks to big data, and its future is very bright. With big data, a more stable, efficient world is possible, and data scientists are just starting to understand what it can do. However, the security risks are still quite large. It’s up to IT organizations to determine how to store and analyze big data efficiently and securely so society can truly take advantage of it.
Online Tech will be at the Indianapolis Big Data conference, speaking about how to protect your data on Sept. 1 at the Indiana Convention Center and Lucas Oil Stadium. For more information, visit https://web.otava.com/indy-big-data-conference-2016.
