Some metrics about ‘large’ models

We are getting a lot of questions about the ability of BIMserver to handle large models. Most of the time our answer is that you have to allocate more heap memory. We’ve never seen a model that cannot be handled by BIMserver because of its size.

Recently some users asked us to perform a quick research on how BIMserver handles large models. We’ve received several different IFC models varying in size between 500Mb and 3Gb. Some of them so complex that they don’t open in any known IFC viewer.
This gave us the opportunity to measure the performance of BIMserver and get some metrics. We are happy to share these with you in this blog.

We’ve used a 122Gb RAM cloudserver from Amazon with 16 cores for this test.
We decided to do the test in a sequence, meaning it is not a ‘checkin one model and stop’, but a ‘ checkin all models one after another and pray’.

We are using several different IFC files for this:

  1. Model 1 is 1.5Gb of IFC step file;
  2. Model 2 is 474MB of IFC step file;
  3. Model 3 is 1.9GB IFC step file;
  4. Model 4 is the 3.85Gb IFC step file;
  5. Model 5 is a 991Mb IFC step file;
  6. Model 6 is a 685Mb IFC step file.

We started the test at November 7th at 01:28:02. The test ended at 05:42:09. The time of the test was of course influenced by the upload speed of the internet connection. What is interesting is the database and memory usage of the server.

The log:

07-11-2014 01:28:02
Database size: 350.91 KB (359330)
Used: 2.12 GB, Free: 1.29 GB, Max: 200.00 GB, Total: 3.42 GB
Done checking in Model 1
Creating project model 2
07-11-2014 02:10:50
Database size: 15.02 GB (16131952680)
Used: 53.13 GB, Free: 23.70 GB, Max: 200.00 GB, Total: 76.83 GB
Done checking in model 2
Creating project model 3
07-11-2014 02:23:54
Database size: 7.82 GB (8391386817)
Used: 38.05 GB, Free: 39.22 GB, Max: 200.00 GB, Total: 77.27 GB
Done checking in model 3
Creating project model 4
07-11-2014 03:20:47
Database size: 22.25 GB (23893972378)
Used: 68.99 GB, Free: 52.90 GB, Max: 200.00 GB, Total: 121.90 GB
Done checking in model 4
Creating project model 5
07-11-2014 05:16:25
Database size: 42.89 GB (46053250623)
Used: 121.55 GB, Free: 29.69 GB, Max: 200.00 GB, Total: 151.23 GB
Done checking in model 5
Creating project model 6
07-11-2014 05:42:09
Database size: 32.35 GB (34733942862)
Used: 67.11 GB, Free: 45.70 GB, Max: 200.00 GB, Total: 112.81 GB
Done checking in model 6

About 2 hours later, the database is 30GB big, CPU back to normal and 70GB of heap memory used. 56GB of that is probably cached database data. We’ve explained many times why the storage of BIMserver is higher than the original IFC file. We believe this is needed to profit from the benefits of using separate objects in a database instead of files. Of course all models were intact after download from the server.

What this test shows is that BIMserver is perfectly capable of handling large models, and even tolerates the checkin of a model while another checkin is still not fully processed by the database. This proves that stability of BIMserver as a strong base for every kind of IFC model.

What surprised us was the caching and memory usage of the database. We decided to spend some time on that and expect to have better performance metric in the next release.