Testing Content Server browse performance (Part 3)
This is the third in my series covering Content Server browse performance.
In Part 1 I examine the setup and tooling I will be using
In Part 2 I run through some of the elements of the tooling that I used to capture performance timings.
So where we are — tests created, code running and successfully executed so let's have a look. For the sake of speed, I will be looking at the end to end test results. That is, the total time taken to perform the following tests:
Test One
- Browse to Enterprise Workspace (or
/nodes/2000in Smart UI) - Browse to Performance Test folder
- Browse to 50 Folders folder
Test Two
- Browse to Enterprise Workspace (or
/nodes/2000in Smart UI) - Browse to Performance Test folder
- Browse to 100 Items folder
Results may vary
Let's be clear this is not meant to be empirical evidence of how Content Server performs — for instance very few people will deploy a single instance with a Postgres database into production. I am just attempting to offer something better hearsay and conjecture when it comes to comparing the performance of the two competing UI’s in Content Server. There will be better ways to improve your performance by:
- Adding additional resources at a database level (IO, CPU)
- Hire a DBA and performance tune your database
- Keeping your Content Server regularly patched — cluster management has been a thing for years now. There is simply no excuse for a system admin not to be attempting to patch and improve their system.
- Examing your taxonomy and metadata model. In these tests, I have not added any additional metadata into the mix — but as soon as you start adding columns, facet trees there is a tax that comes with that.
- Permissions — as much as some clients go ‘metadata mad’ — there are as much that goes‘ permission mad’ adding complex and circular permission models.
- Brava — this is another usual suspect (and something I might test later), but the viewer that is not given out for free may also impact on the user experience
Smart UI vs Classic
There is no other way to put it. Smart UI is an easy 2 seconds slower than classic.
I would imagine if there was tuning there might be a chance to get this lower. But the reality is that Smart UI is more dynamic (more stuff gets done on the fly), and the network traffic is more chatty (more REST API calls).
CGI vs LLISAPI
This one interested me. In poor network environments (think WANs back in 2008) lliapi.dll was always the most performance in that area. You might suffer server side as you are wrapping the CGI object but I was surprised to see that is was quicker (however the margins are small).
Apache Tomcat vs IIS
This stood out the most from my results. Especially if you are looking for ways to shave time off your Smart UI performance time. Note: The tests below were done using Apache Tomcat without compression.
The margins for performance improvement are smaller with the Classic UI however.
Oh hai Apache Tomcat
Apache Tomcat’s performance was a nice surprise to me. More so as it is completely stock (apart from running over HTTPS). What that means is that this server was:
- no content expiry configured.
- No compression- but default Tomcat does not enable compression
I will need to perform more tests — however for now here is a quick chart of how compression and / no compress compare
Even those the no compression test run did beat out the compression test. It is worth remembering that the compression result still compares well the IIS result.
Overall results
For the 50 Items test the results were:
- Classic UI (Apache Tomcat 8.5/CGI/No Compression) AVG 4.30472 sec MEDIAN 4.263 secs
- Classic UI (Apache Tomcat 8.5/CGI/Compression) AVG 4.51155 sec MEDIAN 4.433 secs
- Classic UI (IIS/CGI) AVG 4.524 sec MEDIAN 4.522 secs
- Classic UI (IIS/LLISAPI) AVG 4.37972 sec MEDIAN 4.308 secs
- Smart UI (Apache Tomcat 8.5/CGI/No Compression) AVG 6.71571 sec MEDIAN 6.666 secs
- Smart UI (IIS/CGI) AVG 7.27989 sec MEDIAN 7.2165 secs
For the 100 Item test the results were:
- Classic UI (Apache Tomcat 8.5/CGI/No Compression) 4.36972 sec
- Classic UI (IIS/CGI) 4.64389 sec
- Classic UI (IIS/LLISAPI) 4.38877 sec
- Smart UI (Apache Tomcat 8.5/CGI/No Compression) 7.12088 secs
- Smart UI (IIS/CGI) 7.42007 secs
Conclusion times
For a customer demo — the Smart UI will wow audiences. And for new customers who have never used Content Server, the smart UI performance may be ok. However for existing customers who have come to expect a level of performance having to tell them that they must wait ~2 seconds for their content can be a surprisingly hard sell.
As for deploying Apache Tomcat over IIS — it is potentially something worth examining as most deployments will need it (e.g OTDS, Archive Center, Brava). However the win here in Classic UI terms in perhaps a 0.2-second difference using my test methods.
Finally — whilst I have not covered it here. There is some value in determining where in the test the slow down occurs for smart UI (i.e. is it browsing the enterprise workspace or rendering the final test location). Additionally, I have yet to work through the windows performance timings in detail, it is likely that in some aspects the smart UI is quicker under some measures by virtue of less being sent over the wire and better pagination. Something for another day.
Can Smart UI be fixed?
Yes, it can. But the fix won't just be client side — however that said, there are a number of things client side that can also be fixed.
Ultimately, the Smart UI is very chatty. And each one of those queries is hitting a threaded server consuming a thread. When you are firing these queries asynchronously it doesn't take much to overwhelm Content Server if it is not scaled appropriately (with the contention at the database layer notwithstanding). It would appear that the Smart UI is geared toward flexibility — whilst a more opinionated approach would hardcode more options in code to prevent the chatter.
But it is not all bad
With the rise of the Smart UI, has given rise to a richer REST API which can now allow more to be done with the Classic UI.
Thank you if you have managed to read down this far. Should you have any questions you can ping me on Twitter.
