Home Data MarkLogic’s NoSQL challenges traditional RDBMS systems
MarkLogic’s NoSQL challenges traditional RDBMS systems Featured

Every successful IT start-up begins with a good idea – there must be a better way to do this. So began MarkLogic and its journey to what is now known as a NoSQL database.

Fifteen years ago Christopher Lindblad, a post-doctoral researcher in high-speed networks and real-time video processing at Massachusetts Institute of Technology (MIT), began to seriously question the limitations of traditional relational database management systems (RDBMS), especially about handling large, unstructured data and search capabilities. Lindblad got his Bachelors from Caltech, and both a PhD in computer science, and a Masters from MIT.

Without getting too technical, NoSQL is almost the opposite of an RDBMS (like Oracle). In an RDBMS, data is organised into tables that relate back to a master record via index keys. It uses structured query language (SQL) to query the database.

NoSQL does not use tables and relationships. Rather it stores a larger variety of structured and unstructured data of different formats with the ability to index not only the data but also what is in each data type. For example, the BBC used it at the 2012 London Olympics to power Olympic data services storing audio, video, text, results, and other disparate data and delivering 2.8 petabytes of data on its peak day, including more than 100 million video requests.

iTWire interviewed Gary Bloom, its chief executive officer and president, and Tim Macdemid, vice-president of sales in APJ, about MarkLogic’s meteoric rise to fame as the largest global NoSQL vendor.

MarkLogic Gary BloomBloom is a 14-year executive veteran of Oracle, vice-chair and president of Veritas following its acquisition by Symantec, and has held other high-level positions in investment firms.

 

 

Marklogic Tim MacdemidMacdermid has spent almost his entire career in the data field. He is ex-Oracle, SPSS, ISYS, and recently unstructured data specialist QSR International. The remainder of the interview is paraphrased as answers often came from both men.

 

What is good and bad about RDBMS and NoSQL?

It is not about good and bad but about what each can do well. RDBMS has its place where you have structured data in silos and NoSQL has its place where you have disparate data types and no silos.

RDBMS shines when you have lots of formatted, structured data and low requirements for database changes.

NoSQL shines where you have a mix of data types, need database design flexibility, and need to extract data quickly without having to write complex and often one-off SQL statements. The key to MarkLogic’s offering — its secret sauce is in a few areas.

First, right from the start in 2001 it has had enterprise features built-in so that for example it can run ObamaCare (US Department of Health) or the world’s largest library – Library of Congress. It passes the security, robustness, data recovery, availability, and speed tests.

Second, it is about a 360° view of the data you have. Most organisations have too many data silos from RDBMS to Excel spreadsheets. MarkLogic overlays those “gold” data sources not necessarily replaces them with the ability to generate incredible indexing that reveals so much more than siloed data can. The value of a database is not what data you have in silos but when combined the whole is so much greater than the sum of the parts.

Third, is the ease of accessing data. As data types change you don’t have to rebuild SLQ queries, redefine data relationships, and often go right back to square one – MarkLogic’s interrogation system is as flexible as the data it stores.

Finally, it is the development time. NoSQL implementation can be considerably faster – the BBC project had to be built in less than 12 months. Given the timescales, this project would not have been achievable using an SQL database, which would have pushed the design towards a complete modelling of the data first.

How has NoSQL taken on RDBMS?

At least half our revenue comes from finishing projects that started as RDBMS. And it is not an argument about “structured versus unstructured” or “RDBMS versus NoSQL” either. Many clients reach the understanding that RDBMS limitations will hold their particular type of project back, but often have gone so far that cannot abandon it either – there are very few greenfields like ObamaCare or BBC.

And MarkLogic must be distinguished from other NoSQL, which has become more a generic term covering five different models – column, document, key-value, graph (RDF triples), and multi-model. We are the only enterprise class NoSQL and are primarily a document and multi-model type of database.

We started using open standard XML (extensible markup language) to read various data types. It is simple to use, Internet-friendly, and there are lots of application programming interfaces (APIs) for it. Our original database server was written in C++, and we now use a lot of Web language – Java. That means we support Java, JSON, and more.

Being multi-model means we are good in the graph area as well. What that gives us is the highest performance, scalability, flexibility, and functionality of any NoSQL database. We also have RDBMS features like ACID and JOIN support.

Where did MarkLogic get its first big break?

Lindblad’s passion was in search, high-speed networks, and real-time video processing so he started looking at the technology of the time – RDBMS. In his opinion, it was too limiting, inflexible, and held back an organisations ability to handle their current structured data, let alone the new data sources that often considered too hard to define, store and make use of.

More than 1000 clients later we have specialised in media which has become a digitise or die industry, finance and insurance, health, and government. MarkLogic is used in almost all industries so please don’t typecast us. Once you have used MarkLogic it tends to go viral within an organisation and then as people move on to other organisations.

Remember that we may be 15 years old but comparatively, we face the same issues as Oracle did in its development 30 years ago. We have learnt well from Oracle's experience.

You mentioned security – what do you mean?

Because our database can be on-premise, hybrid, public, or private cloud, we have developed enterprise-grade security to protect personally identifiable information (PII) and privacy. We use the term “redaction” a lot. It gives users the ability to combine all the data but to mask data on a need to know or authorised basis. For example, if a hacker ever got in with stolen credentials all they would see is what that person was able to see to do their job – no-one needs to see PII information to analyse it.

In part, that is because our search engine is part of the database so you can set up almost any rules about what can be searched and how.

You mentioned cloud?

MarkLogic runs on commodity x86 servers so it is easy to use in the cloud, virtualises, runs on bare metal, and combinations of all. We run very well on AWS, Google Compute, and Microsoft Azure. We take our “app” to the data wherever it is, and we are not locked into any specific cloud.

You mentioned a 360° view of the customer?

First, data needs to be at the heart of any organisation. It needs to be valued and put on the balance sheet. Next, it needs to be taken out of silos like marketing, accounts, distribution, warranty, etc., and overlaid with MarkLogic to expose that hidden data.

Governments at federal, state and local levels are an amazing example. What if you could combine rate data, with power and utilities, and use of government services to map Australia? A holistic view offers so much more capacity to do good.

Back to RDBMS – do you find scepticism about NoSQL when tendering for a project?

Yes, of course – RDBMS has been around since the 70s and NoSQL since the noughties. But what we say is if in doubt let us validate the data – give us some samples and let us show you how it works. Usually, a proof of concept can be done in weeks — not months or years and the client is sold.

Most clients simply say the greatest value is in what MarkLogic unlocks from existing data – not to mention the speed and agility.

What key messages do you want to leave readers?

  1. Data should be at the heart of all business.
  2. There is a major shift in the database market from RDBMS to NoSQL.
  3. You don’t need to abandon RDBMS and SQL investments – just overlay MarkLogic on it.
  4. The transition will give huge increases in agility and the ability to undertake digital transformation.
  5. MarkLogic is proven, enterprise-strength technology.

47 REASONS TO ATTEND YOW! 2018

With 4 keynotes + 33 talks + 10 in-depth workshops from world-class speakers, YOW! is your chance to learn more about the latest software trends, practices and technologies and interact with many of the people who created them.

Speakers this year include Anita Sengupta (Rocket Scientist and Sr. VP Engineering at Hyperloop One), Brendan Gregg (Sr. Performance Architect Netflix), Jessica Kerr (Developer, Speaker, Writer and Lead Engineer at Atomist) and Kent Beck (Author Extreme Programming, Test Driven Development).

YOW! 2018 is a great place to network with the best and brightest software developers in Australia. You’ll be amazed by the great ideas (and perhaps great talent) you’ll take back to the office!

Register now for YOW! Conference

· Sydney 29-30 November
· Brisbane 3-4 December
· Melbourne 6-7 December

Register now for YOW! Workshops

· Sydney 27-28 November
· Melbourne 4-5 December

REGISTER NOW!

LEARN HOW TO REDUCE YOUR RISK OF A CYBER ATTACK

Australia is a cyber espionage hot spot.

As we automate, script and move to the cloud, more and more businesses are reliant on infrastructure that has the high potential to be exposed to risk.

It only takes one awry email to expose an accounts’ payable process, and for cyber attackers to cost a business thousands of dollars.

In the free white paper ‘6 Steps to Improve your Business Cyber Security’ you’ll learn some simple steps you should be taking to prevent devastating and malicious cyber attacks from destroying your business.

Cyber security can no longer be ignored, in this white paper you’ll learn:

· How does business security get breached?
· What can it cost to get it wrong?
· 6 actionable tips

DOWNLOAD NOW!

Ray Shaw

joomla stats

Ray Shaw ray@im.com.au  has a passion for IT ever since building his first computer in 1980. He is a qualified journalist, hosted a consumer IT based radio program on ABC radio for 10 years, has developed world leading software for the events industry and is smart enough to no longer own a retail computer store!

 

Popular News

 

Telecommunications

 

Sponsored News

 

 

 

 

Connect