Why eBay adopted Node.Js

Whats eBay?

Well, most of you must know what eBay is but let’s just get it once more, eBay is an online shopping site best known for auctions and customer to customer sales. eBay is a multibillion-dollar business with operations in about 27 countries.

Some stats for eBay:

• eBay has around 180 million active users.
• Which can go through around 1 billion of products at eBay.
• All this data is spread over 90PB of memory storage. (as of 2013)
• And for this data, there are around 200 of the Database servers and to support search they have 20 servers for search.

What was eBay using before?

For the most part, eBay was running Java-based tech-stack. And java was used to generate XML, XSLT(extensive stylesheet language transformation) that was used to convert from one type of XML to other to support HTML templates and to later convert them easily to PDF or images.

Java was not only being used to convert XML to HTML markup code but it was also used to generate Javascript as well as HTML. Hence, it was taking quite a lot of resources and time to generate and convert the data/templates.

Getting all this and all considering the stability and the scale of a site like eBay they started to think some options as being a tech giant they are always open to new technologies, and at that time Node.Js was getting great highlight because of its behavior.

Why Node?

So there were these 5 basic items that attracted them to the node, that were:

Speed of development:

So having the frontend and the backend layers with the same tech stack would increase the productivity and the speed. Also, there wouldn’t be more effort as to have the Java learning curve.

Performance:

Performance-wise Node had this upper edge because of its Non-blocking I/O, being asynchronous and single-threaded.

Scalability:

• The engineers started with the basic Java infrastructure, but it consumed many more resources than expected, raising questions about scalability for production. Hence giving an upper edge to Node.

Open source community:

• Js is an open-source community made it easier to have packages download and reuse them instead of writing them again.

CPU profiling:

• As JS being used it was easy to get the CPU profiles though the asynchronous nature of JS created some issues that were later fixed and catered.

ql.io

ql.io was the first project that put Node.js into the spotlight. ql.io works like an HTTP gateway to aggregate data from different services into a centralized point. ql.io is an integral part of the eBays architecture for web content and information delivery. So, why there was a need for such kind of thing?
Because there were cases that required accessing multiple APIs at a time which involved making more network calls. And most of the times those API where not even dependent on each other so it made it easier to make calls parallel were needed and sometimes to get in sequence so as to satisfy dependencies

What to support at eBay?

• User tracking (As this was the main criteria for searching as well as to show the relevant data to the customer.
• Reading cookies (This was being used for the ads as well as to keep the user likings in order to suggest the user's products that are relevant to them)
• Encrypts / Decrypts (For data retrieval and reading cookies )

What to do with Java code?

They came up with 4 options as stated in the images with there pros and cons (left to right respectively). They totally canceled the last two approaches as they were difficult for debugging and also being resource-intensive and very hard to maintain.

After taking all into consideration they came up to have it as a web service as it was easier to use them later. Not only reuse but also they can be called from anywhere be it node or anything and also the calls can be made parallel for better performance.

eBay Goals ?!

To avoid “Magic”

• No idea why and how the code works!
• Making proper documentation for each module.

File Watching

• Should only rebuild the particular module.
• By unloading and recompiling on those.

High performance

• Node.Js can be extended with C/C++ libraries.
• Socket sharing
• High-performance evented I/O at its core (LibUV)
• Single-threaded, So no sync issues or locks.

Async UI rendering

• Immediate template rendering
• Let the template drive the data
• Minimize the idle time on the server

Robust

• Cluster2 (Multi-process cluster management)
• Automated testing using Mocha. These also included Netmorphic testing.
• Continuous deployment using Jenkins.

Configuration management

• Single source
• Per module configuration
• Configuration repo based on MongoDB

Conclusion:

With all that done and said to the factors that became the reason to have to port towards JS were more and better as of what they have. In order to have the best user experience and to have a maintainable and scalable code as well.