NoSQL: The future of data economy

The increasing influence of social, mobility and cloud, and the humungous volumes of unstructured data generated, is not only challenging the traditional RDBMS (relational database management system), but also paving ways to raise innovative queries. The new—aged non-relational database— “NoSQL” has created a universe of opportunities through data collation, management and analysis.

Database technology has come a long way since IBM began its research project “System R” in 1974. This was followed by Oracle, which made relational database commercially available for the first time in 1979. For the next 30 years, which is a fairly long period of time considering the pace of technological advancements, database technologies kept evolving but more or less in isolation. It failed to adapt to new changes that the programmer community was demanding in the early Internet era. Database technology vendors were happy with their dominating market share but were oblivious to emerging technology requirements of data-driven applications.

NoSQL is now providing competition to traditional RDBMS and is posing a serious challenge to its three-decade monopoly. Interestingly, NoSQL databases existed even in late 1960s but its popularity surged only in the first decade of the 21st century due to adoption by companies such as Google, Yahoo, Amazon. These companies realized that they cannot depend on RDBMS technology for their vast data volumes. Hence, they started innovating by adopting open source culture and developed end-to-end ecosystems to manage their business requirements.

The open source movement made these innovations highly collaborative and at a blistering pace. And the gap between traditional database technology and these innovations widened further.

There are numerous differences between SQL and NoSQL databases.

Firstly, SQL databases represent data in the form of tables, comprising rows and columns, while NoSQL databases are document-based and do not have standard schema definitions representing key-value pairs, graph databases or wide-column store-based data categories, thus making them ideal to deal with unstructured data.

Today, we have around 300 exabytes of unstructured data and it is increasing by the day. The predicted rise of unstructured data is at 56% per annum, compared to 12% for the structured data; this is why “NoSQL database” is believed to be the database of the future. According to CBRonline.com, the database market is estimated to be worth $46 billion globally, a total that is expected to go up to $50.1 billion in 2017.

SQL databases are vertically scalable, while NoSQL scales horizontally. SQL uses structure query language that defines and manipulates data whereas NoSQL queries are focused on the collection of documents. SQL databases are typically a good fit for a complex query-intensive environment, something that NoSQL doesn’t excel in. This doesn’t mean they are bad but NoSQL is just better suited to different workloads. It works better with hierarchical data storage but SQL databases aren’t well-suited to this.

Some of the leading SQL databases are Oracle, Microsoft SQL Server, IBM DB2, while the leading NoSQL databases are MongoDB, Redis, Cassandra.

NoSQL addresses needs of non-relational data model, distributed data stores, horizontal scalability and more importantly, design agility demanded by programmers. MongoDB (earlier known as 10gen) is a leader which provides NoSQL in open source model along with subscription-based technical support and other services.

Eliot Horowitz, co-founder and chief technology officer of MongoDB Inc., explains the fundamentals of its design philosophy. He says “MongoDB wasn’t designed in a lab. We built it from our own experience building large-scale, high-availability robust system.” Interestingly, his team didn’t start from scratch. They considered what was broken and missing in another open source database, MySQL.

Which way to go?

It finally boils down to “horses for courses”. If you require transaction processing applications that can generate hundreds of reports, then RDBMS works well. However, if you are planning to add an analytics flavour to your transactional applications, then you will be exposed to limitations of the underlying RDBMS foundation. The real consumers of your applications, your business users, are never happy with the transactional nature of reporting. They need intelligent reports generated by mashed up data.

That’s exactly where NoSQL will come in and delight you, and more importantly, your customers.

The author is director (analytics and Big Data) at Clover Infotech Pvt. Ltd.