Developments, research projects, roadmap and all other news of the BIMserver.org platform

Some metrics about ‘large’ models

We are getting a lot of questions about the ability of BIMserver to handle large models. Most of the time our answer is that you have to allocate more heap memory. We’ve never seen a model that cannot be handled by BIMserver because of its size.

Recently some users asked us to perform a quick research on how BIMserver handles large models. We’ve received several different IFC models varying in size between 500Mb and 3Gb. Some of them so complex that they don’t open in any known IFC viewer.
This gave us the opportunity to measure the performance of BIMserver and get some metrics. We are happy to share these with you in this blog.

We’ve used a 122Gb RAM cloudserver from Amazon with 16 cores for this test.
We decided to do the test in a sequence, meaning it is not a ‘checkin one model and stop’, but a ‘ checkin all models one after another and pray’.

We are using several different IFC files for this:

  1. Model 1 is 1.5Gb of IFC step file;
  2. Model 2 is 474MB of IFC step file;
  3. Model 3 is 1.9GB IFC step file;
  4. Model 4 is the 3.85Gb IFC step file;
  5. Model 5 is a 991Mb IFC step file;
  6. Model 6 is a 685Mb IFC step file.

We started the test at November 7th at 01:28:02. The test ended at 05:42:09. The time of the test was of course influenced by the upload speed of the internet connection. What is interesting is the database and memory usage of the server.

The log:

07-11-2014 01:28:02
Database size: 350.91 KB (359330)
Used: 2.12 GB, Free: 1.29 GB, Max: 200.00 GB, Total: 3.42 GB
Done checking in Model 1
Creating project model 2
07-11-2014 02:10:50
Database size: 15.02 GB (16131952680)
Used: 53.13 GB, Free: 23.70 GB, Max: 200.00 GB, Total: 76.83 GB
Done checking in model 2
Creating project model 3
07-11-2014 02:23:54
Database size: 7.82 GB (8391386817)
Used: 38.05 GB, Free: 39.22 GB, Max: 200.00 GB, Total: 77.27 GB
Done checking in model 3
Creating project model 4
07-11-2014 03:20:47
Database size: 22.25 GB (23893972378)
Used: 68.99 GB, Free: 52.90 GB, Max: 200.00 GB, Total: 121.90 GB
Done checking in model 4
Creating project model 5
07-11-2014 05:16:25
Database size: 42.89 GB (46053250623)
Used: 121.55 GB, Free: 29.69 GB, Max: 200.00 GB, Total: 151.23 GB
Done checking in model 5
Creating project model 6
07-11-2014 05:42:09
Database size: 32.35 GB (34733942862)
Used: 67.11 GB, Free: 45.70 GB, Max: 200.00 GB, Total: 112.81 GB
Done checking in model 6

About 2 hours later, the database is 30GB big, CPU back to normal and 70GB of heap memory used. 56GB of that is probably cached database data. We’ve explained many times why the storage of BIMserver is higher than the original IFC file. We believe this is needed to profit from the benefits of using separate objects in a database instead of files. Of course all models were intact after download from the server.

What this test shows is that BIMserver is perfectly capable of handling large models, and even tolerates the checkin of a model while another checkin is still not fully processed by the database. This proves that stability of BIMserver as a strong base for every kind of IFC model.

What surprised us was the caching and memory usage of the database. We decided to spend some time on that and expect to have better performance metric in the next release.

Automated validation for IFC building models

Chi Zang recieving the 'Best Phd paper' award at ECPPM 2014

Chi Zang recieving the ‘Best Phd paper’ award at ECPPM 2014

Chi Zang is a Phd student at the Eindhoven University of Technology. His researchtopic is about “Automated validation for IFC building models“. He published a paper about this topic at the ECPPM conference in Vienna (Austria) this month. He won the ‘Best phd Paper’-award during that conference.

In the development of this research he was using the BIMserver.org platform. More information about his work can be found on youtube and slideshare, or find the pdf of the paper here.

 

 

1000 dollars for your IFC file?

dollarsAs you know, at BIMserver.org we intent to build a stable and reliable foundation for other developers to build their tools on. We are very proud and quite confident about the work we released so far. Of course we never rest and keep testing on a constant base. We need the help of the users for that because we need IFC data to test our tools. That is why we created the bug bounty program. We are still not sure on how to shape that program, but decided to at least give a reward of 1000 dollars to anyone who provides us (in the next couple of months) with an IFC model that crashes the BIMserver platform.

Of course the IFC model you provide should be a valid IFC model. We have some known issues that we think are not affecting this challenge, but nonetheless those are excluded (just to be on the safe side). The challenge is to be conducted on the latest stable release of BIMserver (so not the IFC4 beta release). Needless to add that this is about BIMserver. Most users add IfcOpenShell, bimvie.ws and other plugins to BIMserver. These plugins could cause issues, but obviously we are talking about just BIMserver here.

Please send the files to ifc@bimserver.org. We understand that sending large files can be a problem, but tools like wetransfer and dropbox can help you with that.  We strongly encourage to share your models public under a creative commons license. In that case, please send a pull request to the IFC repository on github. That repository is used to test all tools of the open source BIM collective so every contribution is highly appreciated. We even appreciate it so much that we randomly draw a winner from the contributors to collect the 1000 dollars when nobody collects the money from the bug bounty. All contributors will get a BIMserver.org coffee mug, a towel and our eternal gratitude.  People that add IFC4 models to the github repository will receive an extra surprise.

 

 

 

 

IFC4

ifc4 We’ve been experimenting with IFC4.  Because the core of BIMserver is generated  from an IFC schema it is not very difficult to create a BIMserver with IFC4 database instead of the current IFC2x3 (the database schema is also generated from core). The challenge is to have IFC4 ánd IFC2x3 files in the same core and database. After some initial research we decided to implement a first concept where a project can be IFC2x3 ór IFC4. Because subprojects are also ‘just’ projects, this creates the possibility to have a main project with subprojects that hold different versions of IFC data. For example a project with subprojects ‘architect’, ‘construction’ and ‘MEP’  where 1 of these subprojects has a different IFC version dataschema than the others, but can still be merged, queried, etc.. (and all the other BIMserver functions).

The code has been checked into github.

At this moment we do not have enough IFC4 files to test this and turn it into a release. Please help us out with IFC4 files. You can post them in this github repository or send them to us via e-mail. Github submissions will be public (creative commons license) and submissions via e-mail will not be made public but only used for internal testing purposes. IFC aspect models with different IFC versions (FOR EXAMPLE the architecture model in 2×3 and construction/mep in IFC4) are highly appreciated.  As always we send you a BIMserver coffee mug as a token of our appreciation. Thank you very much!

 

Release 1.3.0 final

The final 1.3.0 release has been build. For those of you who followed the release candidates there is not much new. For those who are still on 1.2 we have some huge changes. The biggest changes are the improved integration of render plugins, and the new plugin type ‘modelchecking’. Download it from our github page and have a look at the introduction video for setup instruction. Other documentation can be found on our wiki. We are very thankful to all the contributors and financers.

Geometry
Our most active users and developers know that we see BIMserver as a server. Not a viewer. We don’t handle rendering, Boolean operations and other geometry calculations. The plugin type ‘render engine’ is introduced in a very early development state of BIMserver to have geometry processed by tools that are specialized to do just that. In the latest versions of BIMserver there were 2 render engine plugins shipped with BIMserver: IFC Engine and IfcOpenShell. IFC Engine used to be set as the default. The biggest change in 1.3.0 is that IfcOpenShell is now our preferred render engine. For those that still want to use the IFC Engine there is no need to panic: the plugin is still in there, it is just nog set as the default render engine anymore. We strongly believe IfcOpenShell has made major stability improvements and is now in front of most closed render engines [update: read their blogpost].
Parallel with improvements of IfcOpenShell, the use of geometry from the engines is also optimized. BIMserver administrators can now set the option to render geometry during check-in of data and store it in the database. This one-time store, multiple read principle creates great performance improvements for users that want to view of download explicit geometry. Just before the final release of 1.3.0 we even added re-use of recurring geometry elements to improve the speed and agility of the model in online viewers like BIM Surfer. All in all some major improvements that we are very excited about. We enjoyed collaborating with the IfcOpenShell team and are proud to have their product in BIMserver.

Modelchecking
Another attractive new feature in this release is the “model checking” capability. We implemented model checking in two places: before data is stored in the database, and before a notification to a remote service is send out (after the creation of a new revision in the database). People who work with modelchecking on BIM know that there are many different standards to define how a model should be checked, and even more discussions about which one is best. At BIMserver we have a reputation to respect opinions of others and seldom choose one direction. We see ‘rulesets’, MVDs, SEMs, XSLTs, just plain code, and lots of other options all as probable directions for modelchecking. Although we see lots of potential in many of these initiatives, but feel that none of them has the combination that is ‘open’ and ‘stable’ enough for an effective implantation in the current release BIMserver. For that reason we created a very general checking plugin, with the intention to create specific checking plugins when the mentioned standards are more stable. At this moment the generic model checking feature only works on java code that is specific for BIMserver. We know, not a good example of open standards, but again: set up to grow into something better along the way, flexible enough to follow different directions depending on the adoption.
To make the model checking feature useful for non-programmers we also added the ability to publish and share snippets of code in an online repository. Again we feel obligated to mention that these snippets only work on BIMserver and are not a good example of interoperability. We hope to connect to other repositories with snippets of model checkers in an(y) open standard.

Small feature improvements
Most of the other changes in this release are “under the hood improvements” (bugfixes, better error messages, e-mail headers, memory use optimization, updates of jetty and berkeleyDB, etc.). Some of them however seem to make our users very happy. With the ‘send notification’ features (find it under settings) an e-mail is send out to project members every time a new revision is checked-in. Another improvement is the ability to configure (remote) SMTP servers more extensively. You can now choose a lot more options such as the protocol (SMTP/SMTPS), username, password and port. This allows for using a 3rd party e-mail service which is probably a good choice (and doesn’t have to cost extra). Examples of these providers are Mailgun, Amazon SES and Sendgrid. The admin interface is also improved. We are happy to have the help of the bimvie.ws team for that.

Coming up
We are also very proud to mention that work on a new version of BIMserver is already started and progressing very nice. Biggest challenge we put ourselves into is to have a project with subprojects that contain data in different IFC versions. First experiments have showed that this might turn out very nice. Combining IFC 2×3 and IFC 4 in one BIMserver would be a major innovation. We’ll keep you tuned.
Another effort we put resources in, is optimization of the EMF core. BIMserver makes intense use of RAM memory. Although prices are dropping rapidly,  the number of objects in IFC models seems to be increasing. This keeps the memory usage in BIMserver to stay an issue for current practice. Therefore we are researching how to use EMF more efficient, or not use it in parts that don’t really need it. Community efforts are also gaining more traction. New to the ecosystem is bimvie.ws as open source commercial initiative that created a GUI for BIMserver. Another initiative is an mvdXML checker plugin for model checking created by Benzcly Zhang from Eindhoven University of Technology. A preview can be found on our github page.

Our main focus at this moment is work on the “Federated BIM” concept and intense collaboration with the projects in the open source BIM collective.

Let us know what you think, or any issues you find, on our support forum.

Why we are involved in BIMSie (and other AEC Hackathon presentations)

As most of our fans know, we are heavily involved in the ‘remote services’ part of BIMSie. During the AEC Hackathon we ‘finally’ had the chance to explain and demo why we spend so much time and effort on this. It is a hackathon presentation, so it is a bit technical.

The other two Hackathon presenations about ‘BIMserver when you first login (as administrator)‘ and ‘introduction to open source BIM tools‘ are also available.

Update: slides now available on slideshare http://www.slideshare.net/berlotti/federated-bim-and-bimsie

Participation in FED iFM

The U.S. Department of Defense (DOD) Military Health System and U.S. Department of Veterans Affairs (VA), combined, manage in excess of 210 million square feet of healthcare facilities and just completed a series of projects through the National Institute of Building Sciences centered on planning, design and operations of facilities. The team developed strategic plans, road maps and proofs of concept using BIM, geographic information systems (GIS) and facility management (FM). It delivered a strategic vision for FED iFM (Federal integrated facility management). FED iFM is an initiative to create shared and common practices for integrated facility management in federal agencies and the private sector. The vision is a technology hub of software and applications that can be used for rapid and agile development of tools or innovative practices for moving data from early planning through design, construction and into operations and facility sustainment. Open source as well as proprietary technologies will be evaluated within an integrated platform of cloud and server-based environments.

(C) Onuma

FED iFM Ecosystem

Onuma Systems and the Netherlands organization for applied scientific research TNO have joined forces to evaluate the BIMserver.org platform for use in the FED iFM ecosystem. Join this webinar on February 18th to learn more about the concept [update: watch a webinar about the concept here]. Important part of the strategic vision of FED iFM is to ‘get the data flowing’ between applications. This was the theme of a workshop that was organized last month in Washington D.C. The goals of the workshop were to share a vision on FED iFM; share the results of current federal agency initiatives that are striving for the FED iFM vision; engage with private sector owners, architects and professional organizations to build a bridge of collaboration and shared interests in achieving the FED iFM vision; and enlist technology service providers to build platforms, applications (apps) and app marketplaces to access agency data repositories and to foster and enable the realization and success of the FED iFM vision. During this workshop TNO gave a presentation about BIMserver and their vision on how to ‘get data flowing between applications’. The presentation was recorded and is available here.

Evolvement in BIMSie

BIMserver.org is involved in the new BIMSie initiative from BuildingSMART Alliance. We intent to build BIMserver in compliance with the BIMSie standard API.  BIMserver.org is specifically involved (and interested) in the development of the ‘RemoteServiceInterface’ and the ‘LowLevelInterface’. Léon van Berlo presented his view on BIM in the cloud and BIMsie during the “Building Innovation Conference 2013”  in DC a couple of weeks ago. You can see the two video’s over here:

 

 

New version of BIM Surfer and integration in BIMserver platform

The BIM Surfer project is, just like BIMserver.org, part of the open source BIM collective. It is developing an open source WebGL viewer for BIM models based on open standards (for now only IFC). The project was an initiative from our team, but created by Rehno Lindeque. Due to several reasons there was not a lot of progress at BIM Surfer, until some guys from Vienna picked it up. During the Google Summer of Code students worked on improving BIM Surfer with cool new features. The most visual ones are the two sliders. You can see all about it in de youtube movie below. There are also numerous other improvements. Read all about it on github.
The BIMserver.org team, in close collaboration with MOST and TUWien, is working hard on the integration of this new viewer with BIMserver. We are using the new JSON interface to interact with BIM Surfer and load the objecttypes one by one to speed up the loading time. More info will be available in our new beta release which is coming up in a couple of weeks.

UPDATE November 13: Integration with BIMserver platform.

The BIMserver.org team has intensified the communication between the open source BIM Surfer viewer and the BIMserver.org platform. The ‘get model from server’ now works, and there is much more user feedback with progress bars and partial loading of models. Have a look at the video above.

Developers release 1.2.0 (beta release)

Today we released a beta version of 1.2.0. To get things clear right away: it has bugs. Please do not download this if you are an end-user. The main reason to release this new version is to inform developers of changes and, again, a new concept.

The new concept we would like to introduce is still a bit fuzzy. We are developing at it every day and it is changing every day. This is also the reason why we will keep updating the beta releases of 1.2.0 the coming weeks. The most important change you will notice is the ability to add ‘extended data schemas’ to projects and a new plugin type for queries (including the new BimQL plugin). More detailed info and documentation will follow soon.

When you are a developer, things will probably make sense when you dive into the code. When you are a user, a demo that we are preparing will say more than a thousand words. So please be patient for another little while.

After looking at the new release, please send your feedback through http://support.BIMserver.org