Why Choose Node.js to Code Your Database?

A lot of people have asked me the question “Why did you build Harper in Node.js?” or something similar in a less polite way. To be honest when I first saw Node.js, I hated it. I resisted, hard. I felt that Java developers like myself were REAL developers, and folks developing in Node were just banging on a keyboard producing random outcomes.  Eventually, I gave in to the wave - it was AWS Lambda functions which really pushed me in the Node direction, and I started to realize the error of my ways. 

A Node.js Learning Curve

For the first 3 months using Node.js, I HATED IT. I hated how variables were not typed and a string could become an int and back to a string. I hated the callback hell. I remember debugging

if(let i =1){
  do something 
}

for over an hour before realizing that i was becoming 1 in the if statement. It drove me crazy.  

That said, over the first three months, I began to learn Node's quirks. My hate slowly turned to acceptance, and my acceptance quickly turned to love. Coding in Node.js has renewed my passion for software development, and has brought enjoyment back to the process. 

Benefits of Node.js

As we dove deeper into Node.js we continued to discover things about it that we found incredibly helpful.

Kyle found async, probably the single greatest library in npm that solved some major problems for us. We no longer were trying to do looping callbacks, or using massive sets of nested code. We were able to use things like async.each and async.waterfall to solve our problems. Now we can handle callbacks elegantly with a reliable library that does the heavy lifting for us.   

I began to see that we were coding at a speed that was much much faster than we had in Java. Things that would take us months previously, now took us days. Additionally, most code schools, like Turing here in Denver, are teaching some Node.js framework so it made recruiting a lot easier.  

I also loved that my computer was not eaten alive by the JVM every time I spent a few hours coding something or had accidentally left my server running. One of the major benefits of Harper as a database is that it uses almost no resources while at rest. In fact, the first time I went to look for it in top Zach had to explain to me that you could page down in top, because not only was Harper not in the first set of results, it was the last (Richard Clayton discusses this difference between Node.js and Java in more detail here). 

From an IoT perspective, this is huge, as the hardware on the edge doesn't always have massive specs and utilizing database management software that can more elegantly and efficiently utilize that hardware is critical. Because Node.js is cross platform, we were able to release our first version with distributions for Linux, Mac, ARM6, ARM7, and ARM8.  

Harper and Node.js

From a scaling perspective, Node’s resource management is important to Harper as the product can scale massively without a huge hardware budget. In fact, in some of our initial testing, we were able to handle the same load and more as SAP HANA running on hardware that was 32x smaller (more about that in a future post). Furthermore, the package cluster allows Harper to better utilize hardware by vertically scaling with the machines hardware. 

Additionally, for IoT and IIoT, Harper being written in a lightweight framework like Node.js allows it to be at time of publishing, under 65MB with the entire Node engine and all its dependencies included in the build. This is great as you can run Harper directly on your micro-computing devices.  

I am an impatient man. For those who know me, they would probably say that is an understatement. Eclipse and even IntelliJ drove me crazy when coding in Java. I couldn't stand how long it took for my server to spin up. With Node, my server is up and running in sub-second time allowing me to do more robust unit testing and producing higher quality code. Furthermore, in the event that a Node.js app crashes in a production environment, it can be back up and running in sub-second time.  

NPM is Awesome

I love how easy it is to get anything done in Node.js. I have always hated how difficult package management is in Java. With Node.js not only does npm, a node repo management solution, make things amazingly simple; the Node.js community is incredible. In fact according to Stackoverflow's developer survey 47% of projects last year were written in Node.js! Npm has over 350,000 packages currently available and is the largest package management solution in the world.

Npm is an incredible resource. Almost every time we are trying to solve a problem we look to npm and the solution is there. For example, when I was tackling our clustering solution; we spent a lot of time thinking about how to design it. We realized our design looked a lot like a chat messaging app. Enter socket.io, the backbone of Harper's clustering capability. Connection management is built in and hardened, and messaging is reliable and easy.    

Another example - Kyle was trying to tackle complex mathematics for our SQL capabilities. As Harper is an HTAP database, not only are we focused on scale but we are also focused on providing robust analytical capability. Again we looked to npm and by leveraging Math.js, Kyle was able to implement some truly incredible mathematical capability within our SQL functions. As a result, we have things like unit conversions, and shortly will have GEO spatial query as well. 

Node.js is the Future

A lot of folks see Node.js as a fad or as something for startups, but not a good fit for enterprise applications. Those folks are wrong. At Harper, we are excited to be on the forefront of bringing Node.js to the enterprise. Harper is the first enterprise class database written in Node.js. Node has allowed us to quickly develop a product that is lightweight, easy to use, and incredibly stable.   

If you want to learn more, you can download our Community Edition today.