Big data and virtualization: a formidable defense
All those terabytes of data from so many different platforms can be made available on demand, Delphix’s Ted Girard writes.
Our national security is becoming increasingly reliant on the ability to rapidly manage and quickly use the growing amount of military and intelligence data available at any given moment. As part of President Obama’s 2012 Big Data Research and Development Initiative, the Defense Department was among six major agencies that committed funding to develop tools and techniques needed to access, organize and glean discoveries from huge volumes of data. The end goal: to improve situational awareness to help warfighters and analysts, as well as provide increased support to military operations.
Nearly three years later, military forces continue to face complex and intriguing challenges related to the dramatic increase in data – they are simply overloaded with information. U.S. military intelligence analysts continuously sift through vast and varied data sets, from cellphone records, email and text files to more advanced satellite imagery and intelligence, surveillance and reconnaissance (ISR) data from drones. Consider just the massive data growth from drones and other surveillance technology:
- Video streaming of the military’s Distributed Common Ground System totals over 7 terabytes daily.
- The Air Force ISR Agency collects about 1,600 hours of video per day.
- A single 14-hour drone mission can generate about 70 terabytes of data.
These new sources of data are providing a wealth of new information but they also pose problems with respect to data management and integration challenges. Much of the data that military and intelligence communities collect and analyze exists on different platforms, in different forms and in different environments. In addition to the volume, sharing relevant and critical data is growing more complex by the day for both soldiers out in the field and for defense professionals operating in command centers.
Why deploy data virtualization?
The need to turn big data into actionable insights that can be swiftly disseminated to military leaders and key decision-makers can be a matter of life or death for our warfighters. Today’s harsh realities of digital warfare, cybersecurity, global terrorism and homeland security demand that critical data from multiple sources – structured and unstructured – be integrated in order to provide better intelligence, information sharing and timely delivery.
Virtualization involves creating a virtual version of a specific item – this includes a server, application or database. Data virtualization is the process of abstracting different data sources through a single “virtual” data access layer to provide unified access and integrated data services to users and applications in real time. A big data analytics platform becomes much more valuable when you can access, analyze and disseminate the data “on demand.”
With virtualization, you can update and refresh large databases in a fraction of the time, create multiple copies of a reporting system in parallel and at different points in time, and share the information with relevant stakeholders instantly to facilitate better decision-making. Historical snapshots provide mission teams with the best information to detect changes in behavior, position and/or other relevant activity.
Agility is a big benefit when dealing with big data analytics. With data virtualization, DOD agencies can have access to a comprehensive, accurate and constantly changing view of military and intelligence data that can be shared across organizations – ensuring that the right data is delivered to the right people at the right time.
More big reasons
In addition to increasing agility in data management, virtualizing the data eliminates the bottlenecks associated with provisioning new environments and backing up or refreshing applications and databases. Other game-changing advantages for defense and intelligence agencies include:
- Increased project output by up to 80 percent.
- 90 percent reduction in infrastructure space requirements.
- Greater quality assurance with significantly fewer data-related errors.
- No production system downtime.
To create real value from big data, data virtualization can be an integral part of DOD’s use of open source technologies such as Hadoop and data visualization tools. Virtual data management also enables agencies to more effectively migrate to the cloud or maintain hybrid environments. Virtualization ensures data integrity and can help resolve migration issues quickly by creating data sets from any point in time while keeping storage requirements to a minimum.
A defensive approach
In order to protect our national security at home and abroad, U.S. military and intelligence agencies must find ways that utilize the best technologies to improve the integration and dissemination of disparate pieces of data quickly. The adoption of data virtualization is proving to be a viable and cost-effective solution within DOD as it strives to increase situational awareness and enhance military operations. An agile approach to data management is now possible providing huge cost savings and greater access – enabling DOD to execute at a higher rate of success and stay ahead of the game.