Big Data Cloud Delivers Military Intelligence to U.S. Army in Afghanistan

High Performance Technologies, Inc. (HPTi), a firm that focuses on the technology challenges of America’s federal government, supported the design and implementation of a private cloud that conveys the latest intelligence information in near-real time to U.S. troops stationed in Afghanistan. The cloud’s reach extends all the way to forward operating bases that might serve as local nodes by setting up trailers and collecting data from unmanned aerial vehicles (UAV) or other nearby sources.

The program to develop the Big Data cloud began in April 2009, after the U.S. Army determined that the architecture of a private cloud could fully meet their requirements to handle “intel” analysis, processing, and production.  The cloud went live in March 2011.  

The Army approached HPTi to be a part of the team that worked on the effort, the company told IDC, because of the firm’s reputation for creating and exploiting high performance computing (HPC) platforms.  The Army considers private clouds of the type HPTi designed to be HPC environments.  The Afghanistan private cloud can handle petabytes of data today and is likely to grow quickly. Intelligence clouds need access to information going back a long way, in order to make it possible to see patterns needed for predictive models.

Semantic algorithms and graph analysis are already important components.

 In general, the Army prefers open source environments for their flexibility and agility, and to avoid dependence on large vendors.  Some of the algorithms enabling the cloud were off-the-shelf formulas, while others had to be custom-written by Digital Reasoning and others. 

HPTi assisted putting all the pieces together to produce an integrated solution, including the software stack that manages the way the algorithms securely interact with all the data. The stack uses open-source Hadoop for the file system and MapReduce.  Under the contract, HPTi also provides the support for the private cloud. Software releases happen every four months.  .

Today, the HPC cloud delivers intelligence to multi-function laptop workstations. Plans call for enabling connectivity to mobile devices such as Android-based tablets.  Today, the cloud delivers specific data to users for analysis. In the future, users will be able to mine and search all the data.

According to HPTi, the main pre-deployment challenges were as follows:

• A lot of the software was new. Working with the Hadoop stack was not easy, especially in its early, pioneering stages.
• Synchronizing technology milestones with the fixed DoD acquisition life cycle was another challenge.
• Finally, it was challenging to have to release the entire integrated solution at one time, including a whole series of tightly integrated applications.

Next steps including tuning the analytics to keep “noise” from increasing in certain situations. Another future challenge is true synchronization from one constituent cloud to the next, to enable continuity of profiles and functionality when users change geographic locations.

HPTi said a movement is afoot to standardize private clouds across the U.S. Government, by certifying the cloud to be able to communicate with each other as trusted partners. This will require a common reference implementation, so that the “meta cloud” looks like a continuous data space.

HPTi’s business has been on a roll for some time, and the company will soon become even larger. In June 2011, HPTi signed a definitive agreement to merge with Dynamics Research Corporation.

Please plan to attend IDC’s 47th annual industry business briefing
Directions 2012

Competing for 2020: Delivering on the Promise of the Connected World
March 7, 2012 in San Jose, CA ~AND~ March 13, 2012 in Boston, MA

Contact me for more information or visit: http://www.idc.com/directions12

Related Stories

The New Era of Computing: An Interview with “Dr. Data”

Digital Reasoning Refreshes Text Analytics

GE Addresses Time Series Data Demands