How to Benchmark Cloud-based BIM Platforms?
bimU.io vs. Autodesk BIM 360/Forge vs. Trimble Connect
Disclaimer: I’m the lead developer of bimU.io. Performance might vary depending on model size and complexity.
TL;DR
- Our clients have been asking about how good bimU.io’s performance is compared to other cloud-based BIM platforms. They like the capability to access BIM without software installation. But sometimes 3D navigation in browser is a little bit clunky.
- The following three medium-sized BIM models were used for the testing. Details of the models can be found in the Test Conditions section.
1. Autodesk Navisworks file (“NWD”) exported from Autodesk Revit
2. Autodesk Revit file (“RVT”)
3. IFC file (“IFC”) exported from Autodesk Revit - We tested the above three models on the following three BIM platforms:
1. NWD, RVT, and IFC were uploaded to bimU.io Viewer (“bimU.io”) — a freebie offered by Transformosa to access and share BIM via browser effortlessly and securely.
2. NWD and IFC were uploaded to Autodesk BIM 360 (“Autodesk”) — a product line owned by Autodesk that also empowers its developer tools, Autodesk Forge.
3. RVT and IFC were uploaded to Trimble Connect (“Trimble”) — a project collaboration tool created by Trimble for BIM-enabled document management. - Autodesk BIM 360 is the fastest for model upload and conversion. bimU.io can load model in browser blazingly fast. Trimble Connect is good at processing IFC files more quickly.
- bimU.io has the best graphics performance. Graphic settings have to be tweaked a bit for Autodesk BIM 360 to be smoother. It might be a bit laggy when a BIM model is bigger on Trimble Connect.
- bimU.io has the least amount of network bandwidth consumption (similar to mobile data usage), followed by Autodesk BIM 360 and Trimble Connect.
Test Criteria and Results
The three tested platforms are fundamentally different in terms of functionality and product positioning. So we needed to think of some objective criteria to do fair comparison. As far as overall smoothness is concerned, three questions came to mind:
- How quickly can I push up a BIM model to the server and view it online?
- How responsive can it be when navigating a 3D model in browser?
- How big would a processed model be in terms of file size?
Time
To make a BIM model available for online viewing, it is a process involving exporting from BIM authoring software, uploading to BIM platform, and converting to a web-friendly format. Autodesk BIM 360 was generally the fastest because it had the ability to read native files and no need to export. But it might not display the same graphics and visibility as in modelling software. When a model was ready to view, bimU.io could load it in browser much more quickly.
Frame Rate
Frame rate greatly impacts the viewing experience in 3D environment. Frames per second (FPS) is a unit that measures display device performance. The greater the FPS is, the smoother the 3D scene appears. FPS fluctuates from time to time as you navigate in 3D. Below FPS values were given as ranges after taking out extreme values that were too large or too small during the operations of sectioning, orbit, zoom, and pan. Obviously bimU.io hit the highest level in terms of the overall graphics performance.
Download Size
The smaller the “compressed” download size is, the less “data usage” it consumes. Normally it includes some static web resources (e.g., images, code libraries, etc.) and the processed model file converted by BIM platform. A smaller “uncompressed” download size might imply lower memory usage, too. In this regard, bimU.io noticeably outperformed the other two BIM platforms.
Interested to know where these numbers came from? Scroll all the way down!
Graphics Performance
Four 3D navigation tools were used to test graphics performance: sectioning, orbit, zoom, and pan. Below screenshot GIFs showed the real-time FPS values which were self-explanatory. You would only notice significant difference when you literally navigate a large BIM model in 3D and it becomes a bit laggy. On the other hand, we had to tweak a bit Autodesk BIM 360’s graphic settings to make it more responsive. Having edge, line, shadow, reflection, etc. turned off resulted in higher FPS. This might be something of interest to the users of Autodesk BIM 360. Below images take time to load. Please be patient : )
NWD File
RVT File
IFC File
Network Activity
Download sizes were shown in the screenshots below. The metrics of “transferred” and “resources” are basically compressed and uncompressed sizes. More explanation can be found here. bimU.io and Trimble Connect both use a single blob to transfer model files. Autodesk BIM 360 splits it into a number of smaller blobs. So we filtered them out by .pf extension.
The three time spans (Finish, DOMContentLoaded, Load) shown below were NOT accurate enough to measure how long it would take to load a BIM model completely. Instead, a timer app was used for all time measurements, including export and upload time, conversion time, and load time.
NWD File
RVT File
IFC File
Test Conditions
- Models
The NWD and RVT files are sample models created by Transformosa. They are available online for every bimU.io user. Unfortunately, we can’t provide the source files. The IFC file is publicly available under cc-by-sa-3.0 licence and can be downloaded here.
- Machine
My three-year-old laptop with the following spec:
CPU: Intel i7–6820HQ 2.71 GHz
GPU: NVIDIA Quadro M1000M
RAM: 32 GB
OS: Windows 10 - Network
Speed test result of my home fibre optic broadband, provided by University of Oxford, IT Services:
Download speed: 19.26 Mbps
Upload speed: 15.70 Mbps - Location
Tester is in the United Kingdom. Server locations of the three platforms are all in the United States. - Time
All tasks were done on a Sunday evening. - Software
Google Chrome 78.0.3904.97 and its Developer Tools and FPS Meter.
Wrapping it Up
There are so many factors that influence the overall user experience (UX). Performance is just one of them. To help you inspect a particular location of interest more efficiently, bimU.io Viewer also features some 3D navigation tools that come in quite handy, such as “Section around Selection” , “Open Viewpoint in BIM Software”, etc.
bimU.io turns out to have the best performance according to the test. With the current basic functionality, it can be a lightweight alternative to other more complicated BIM platforms. We are also working hard on some new features that are coming next year, including supporting large models, more model formats, issue tracking, open API/SDK, etc. Please stay tuned. If you have any problem using bimU.io Viewer, please feel free to email support@bimu.io or log a ticket on our support centre. Below are some useful links for you to get started.
bimU.io Official Site
https://bimu.io/
Log into bimU.io Viewer
https://viewer.bimu.io/
User Guide
https://docs.bimu.io/
FAQ
https://support.bimu.io/portal/kb
Support Centre
https://support.bimu.io/
