Is MarkLogic the Adult at the NoSQL Party?

If big data is a big party, Hadoop would be at the center and be surrounded by Hive, Giraph, YARN, NoSQL, and the other exotic technologies that generate so much excitement. Lurking near the edge would be MarkLogic, a NoSQL database vendor that has one foot in the big data party, and one foot in the mature (and sometimes boring) world of enterprise software.

MarkLogic was founded in the San Francisco Bay Area in 2001, before the term “NoSQL” was popular. Back then, the company referred to its main product an XML database, which gave a nod to the unique way in which customers could store and access data. This approach was new and different, considering the utter dominance that relational databases like Oracle, SQL Server, and DB2 had in the world of enterprise software.

Obviously, a lot has changed since 2001. In the last five years, the explosion of less-structured data like Tweets, Weblog data, and sensor infuriation has opened up exciting new opportunities in the fields of data mining, predictive analytics, and visualization. Technologies like Hadoop and NoSQL are important because they provide us with a way to store and sort through all that data, because it simply doesn’t fit into the relational constructs of the traditional RDBMS.

MarkLogic has changed, too. The company now refers to its main database product as a NoSQL database engine, which is technically accurate. In addition to providing a place to store less-structured data, NoSQL databases like MarkLogic’s provide the capability to build applications on top of those data stores in ways that the Hadoop File System does not.

Over the years, MarkLogic has evolved the product to enable it to conform to corporate standards of conduct, explains MarkLogic CMO Michaline Todd. “We like to say that, unlike a lot of other NoSQL databases that are just putting enterprise features into their roadmap, from day one MarkLogic has had things like ACID transactions, high availability, transaction recovery, and government-grade security,” she tells Datanami. “We believe that makes us a differentiator in the enterprise.”

MarkLogic’s database is used for both analytical and transactional applications–in some cases, providing both workloads from the same database, which the company views as a key component of the future of enterprise computing. It is particularly strong in the financial services and publishing sectors, and is primarily used for custom, one-off applications.

MarkLogic’s software backs some pretty major enterprise applications. It powers the applications that are responsible for clearing, correlating, and performing risk analysis on about one-third of the word’s derivatives transactions, Todd says. It is also the core back-end database powering the new federal healthcare exchange that is set to open October 1. That system has been called the world’s largest information integration project, says Aaron Rosenbaum, MarkLogic’s director of product management.

Getting the NoSQL database to that point required years of work, Rosenbaum says. “To be an operational database over billions of dollars of transactions…there’s just a lot of hardening. It takes years,” he says. “It took Oracle years. It took Sybase years. It took MySQL years. It took MarkLogic years.”

Does that make MarkLogic the grownup in the exciting and fast-paced world of NoSQL databases? “We are, definitely,” Rosenbaum says. “”We have the feeling of [being] the grownup. The analysts have validated that, that we’re the only ones who are ready and grown up for those deployments.”

Of course, those other “noisy kids” at the NoSQL party aren’t ready to cede to MarkLogic the title of “Best and Most Important NoSQL Vendor of All Time.” While they might not have the long, 14-year pedigree that MarkLogic can boast, they are undeniably clever, and undoubtedly have a few things up their brash, juvenile sleeves to give MarkLogic a run for its money.

And big money, it will be. According to a 2012 Forrester study of big data trends, high-growth firms are 156 percent more likely to be investing in NoSQL than their peers. That compares with Hadoop, which high growth firms are 54 percent more likely to be looking at, and poor old relational database technology, which high growth firms are only 6 percent more likely to look at than their peers. NoSQL is projected to bring in more revenue than Hadoop for the foreseeable future, with the combined Hadoop/NoSQL market growing from just over $1 billion in 2013 to about $3.5 billion in 2017, according to Wikibon’s recent market forecast.

 ”NoSQL has become viable because of the new compute infrastructure, the speed of commodity computing, and the trust in commodity computing,” Rosenbaum says. “The rise of relational has had it’s time in the sun for two-and-a-half, three decades, and I see NoSQL playing out over two to three decades. This is not like ‘Oh, it’s suddenly hot after three years.’ This has been many years in coming and it will take time to play out and it’s not likely to get displaced.”

MarkLogic hopes to ride the wave for some time. The company is finalizing a major new release of its technology, called version 7.0, which will debut in early October. Among the big new capabilities that MarkLogic 7 will bring include the capability to store and search semantic triples; support for utilizing HDFS as a storage mechanism; and the capability to deploy a single MarkLogic database across multiple, heterogeneous platforms, including Amazon S3.

“We’re doing grown-up, dull sorts of things,” Todd says, “but also cool things that other NoSQL players aren’t able to start thinking about because they’re still playing catch up on the enterprise stuff.”

Related Items:

MarkLogic Secures $25 Million in Growth Capital

U.S. Patent and Trademark Office Launches Big Data Application on MarkLogic

StorageReview.com Publishes Results of MarkLogic NoSQL Database Storage Test