FOSS

Last week I was privileged to be invited to speak at a meeting about molecular modelling with a focus on tools, GUIs and visualisation. The meeting was held at the Holiday Inn in Runcorn and the Daresbury Laboratory (England). I wasn't expecting to be back in England quite so soon, having only just returned to Pittsburgh at the end of January.

The meeting was a great opportunity to present some of the latest work I and others in the Avogadro and OpenBabel communities have been doing to create tools that enable the building of molecules and structures, as well as their visualisation. It was also a great chance to hear some very interesting talks by the developers of other building tools and some quantum codes. Donald and I were also invited to Daresbury Laboratory to work with some of the CCP1GUI developers.

I presented my talk on Avogadro on Wednesday morning and have made the slides available here. Donald gave an introduction to Avogadro, some of the history and the architecture before I gave my presentation. We finished by taking questions while I demonstrated the Avogadro application. I think it was extremely productive. We had many more conversations over dinner and drinks later as well as in a workshop setting on Thursday afternoon.

It was great to be able to put a face to a few of the names and discuss current issues more informally in the evening. The talks were all of a very high quality and from a varied list of speakers from other open source projects, some of the free quantum codes as well as commercial products. I have come away from the meeting with a much better appreciation of the needs in the community and I feel that Avogadro is in a great position to fill the apparent void.

I am glad that we were able to get surface and orbital support working in Avogadro before the meeting. Right now we only support Guassian cube files but the implementation is general enough that I will be able to add support for further formats. I really think that if we can get enough people collaborating on a common project everyone can get the tool they need to effectively do their research at a much lower investment than could be achieved by working on many separate projects.

I met Tristan Youngs, the developer of Aten, who had implemented some really nice features in his molecular builder that is much more focussed on molecular mechanics. It is well worth checking out. As is Zeobuilder which was developed by Toon Verstraelen. They both implement some great features and have strengths in different areas. Of course my dream is to integrate many of these features via Avogadro plugins and have one editor which is capable of being used in a diverse range of applications.

It was also great to speak to Mario Valle who is doing some very interesting work in the area of new visualisation methods and supports a large user base of computational chemistry users. There were of course so many other talks but you can look at the schedule yourself and I think the slides of all the talks should be available in the near future.

I feel sure that many good things will come out of this meeting and hope to be able to attend similar meetings in the future. I would like to thank Jens once again for hosting the meeting and taking care of everything. I hope to see some patches and/or commits from him in the near future

I am pleased to announce that I tagged and released Avogadro 0.6.1 yesterday evening. This is a bug fix release which fixes one pretty large bug that slipped through - the OpenGL context was lost if switching between virtual desktops, multiple views etc rendering the OpenGL window useless unless the application was restarted. As such I would encourage anyone running Avogadro 0.6.0 to upgrade to this new version. It also features several smaller bug fixes and feature enhancements.

The screen shot above shows Avogadro 0.6.1 running in a KDE 4 session. One of the small visual tweaks I made was to add a second light source to our default OpenGL scene which really helps to illuminate the other side of the scene. Thanks go out to Albert for his suggestion of adding another light source. Hopefully there are no really big bugs remaining but Avogadro is still in the beta stages of its development. It is rapidly approaching a stable release though and I am very happy with our progress so far.

We would love to hear what you think of Avogadro. I had one person question why we always have to use the latest and greatest version of OpenBabel and felt I should offer some explanation. Many of the features exposed in Avogadro use functions and structures in OpenBabel. I myself was quite heavily involved in improving OpenBabel's support for Gaussian cube files and the cube format so that we could load and display orbitals for example. As such we often add new features or fix bugs in OpenBabel trunk and so a new release of OpenBabel must be used in order for everything to work.

There are already ebuilds for this latest version in the Gentoo tree. Ubuntu/Debian builds are in the process of being built. We should hopefully have Mac and Windows binaries very soon too.

I am headed to a meeting in the UK where Donald and I will be talking with other scientists about visualisation in chemistry and related areas. We will of course be showing off Avogadro as well as talking with many other people working in this area. I am very much looking forward to it and hope that this will lead to further innovation in the Avogadro project as well as the open source chemistry movement in general. It will of course be great to have a full English breakfast and some real ale too!

Earlier today we released Avogadro 0.6. This release is quite overdue and we had originally planned to make a release around the new year. This release has many, many new features that have been implemented since the last release.

I am especially proud of the new support for displaying surfaces. This initially led to the surface engine that displays a Van der Waals surface. Tim added the ability to map the electostatic surface potential onto that. I then began work on adding support for displaying orbitals. This led to me getting commit access to OpenBabel and making quite a few commits as I got Gaussian cube loading working. While I was there I couldn't resist improving the Grid classes (still a little more I would like to add in).

Another new feature I am really pleased with is the QGraphicsView based periodic table. I think it works very nicely and I am hoping to add this code to Kalzium. I think the Avogadro library is in great shape for me to begin porting Kalzium to use it. Tim and Geoff have been doing some amazing work in OpenBabel on improving the force fields used for geometry optimisations.

We have also been doing a lot of work to improve Avogadro's performance when rendering large systems. We have added quickRender functions and OpenGL display lists to improve interactivity as well as threading calculations where this made sense. We have a cool little colour widget Geoff coded too that allows colours to be displayed and picked easily, improved default layout and one of my personal favourites - persistent settings in most of Avogadro.

There are certainly some bugs remaining. I would like to get to the bottom of as many as I can and hope to make one or two bug fix releases in the 0.6 branch. We would love to get feedback from people. We have already made a source release and a Mac binary. I will be adding Gentoo ebuilds shortly and we will hopefully get a Windows binary out by early next week.

Hope you enjoy this release. I think we are getting close to a stable API that we can call 1.0 and have most of the core features I was hoping to get into Avogadro. Our framework is extremely modular and extensible and I hope that we will be able to build up a community around Avogadro. It was also great to be able to make this release on 29 February - we have so few of them

So over the weekend I spent quite a lot of time hacking away at OpenBabel working on the Gaussian cube format support I needed to get working in order to be able to visualise electronic orbitals. The initial support was taken from Molekel but contained no error checking and it was not reading in the example cube file I had.

Thankfully a colleague pointed out a page with details on the Gaussian cube format and I used this as a basis to get it working in OpenBabel with better error checking and a more resilient tokenisation of the cube points. I made my first few commits to OpenBabel over the weekend and got at least simple Gaussian cube files loading that only contain one cube. More will follow I am sure.

After I got the file loading sorted a few more changes to Avogadro got me my first ever orbitals in Avogadro - the benzene HOMO! I believe the cube file came from the JMol test files. Geoff informed me that chemists for some reason see positive as blue and negative as red. That seemed very strange to me and I had initially put it the other way around. Growing up playing with electronics it seemed to me that positive should be red just as the wires in a circuit have a red positive... Other than that orbitals seemed to be working well and I was very pleased.

Next I started to load up some other cube files I had been given as examples. I thought the above image of the electrostatic potential of CH₃Cl looked very nice. When I showed Geoff he also liked it and we wondered if it had been visualised in this way before. I know normally the cube would be mapped onto the Van der Waals surface of a molecule.

Anyway I was pleased and also felt more productive as I now have my laptop back up and running and am developing on Gentoo again. It would be great to hear what other people think of this new support. I am already working on various improvements to the code and getting ready for another release of Avogadro!

I am pleased to announce that we now have working surface support in Avogadro. We originally began working on support for surfaces around November time when Geoff improved the support in OpenBabel for grids and added some initial code to the Avogadro repository. Quite quickly it became evident that this algorithm did not deal well with data sets containing discrete objects that you wanted to polygonise.

I began a search of the available algorithms and came across a very nice guide to the marching cube algorithm with example code. I had a brief look for a C++ implementation and didn't find anything initially, began implementing what was on his page and then came across Zhu3d which is a Qt 4 based program that actually uses an implementation of the algorithms described in the guide and references them. As it is GPL licenced code I imported the relevant class into our repository and began the task of adapting it to fit into our framework.

With some code from Geoff, more pointers from Geoff, some reading and quite a bit of help from Tim (not sure if he has a web page/blog) we got it working. Since then Tim has actually fixed a big bug in OpenBabel that was causing crashes and done more work on the IsoGen class. The screenshots above show the results of our early work where initially we got points and then lines working. I was very pleased at this stage as it adds a feature that had been missing for quite some time to the Avogadro framework.

I then got filled triangles added with transparency support, allowing us to visualise the underlying structure of the molecule and map other parameters onto the surface. Tim then added electrostatic charge mapping to the colour of the surface which I hope you agree is already looking very good.

There is still quite a bit left to do. I need to move the class into libavogadro and out of the engines, integrate it into our Painter API, implement caching and am thinking of using the Qt Concurrent framework to do intelligent multithreading once Qt 4.4 is released as surface generation is quite slow for big molecules. I also need to link the mesh size to the global quality level and see what other optimisations might be possible. Another big one is adding support for visualising molecular orbitals. Geoff has already done a lot of work on the back end in OpenBabel for this and so hopefully it will not take me long to get it added in.

Lots to do but some great progress already. I would love to know what you think. Once we have been able to polish this new feature a little I hope we can get a release out of the door so that more people can test out these new features and let us know what they think, point out bugs and shower us with compliments Until then I hope you enjoy the screenshots!

Note: For some reason the exported graphics shown in this post were not antialiased. They are on screen and I need to look into why they are not when I export graphics in this way.

I just made a few commits using my trusty git svn dcommit (I am still really loving git and it looks like the bug I reported may be fixed in it now too). Then I saw the commit numbers - we just hit commit 1000. Development has really been accelerating recently and I haven't had time to write about as much as I wish I could.

Hopefully I will get a chance to take some screenshots and show off some of the latest stuff. I have coded a QGraphicsView based periodic table I am pretty happy with. It is compact, allows elements to be chosen, shows our colour scheme and even displays a little extra information on the selected element. Then there is the ribbon engine which is coming along but needs some loving attention from some bio people.

I am very pleased with the new quick rendering stuff we have added. It still has its issues. I had fixed some but some of Benoit's recent commits seem to have reintroduced the cache problem upon engine changes. Tim has added a new hydrogen bond engine which looks great thanks to some stipple line stuff from Geoff. There is some great stuff in the properties dialogs too along with lots of improvements to our force fields.

There is certainly lots more for us to do. The biggest thing I still haven't gotten in yet is surface support! I really want to get that sorted as I think it will be both useful and should look great. Along with some extra dialogs for graphics export, POV-Ray rendering etc.

I recently added a couple of new engines to Avogadro and the screen shot below shows them both. The ring engine finds all the rings of the molecule and draws a transparent plane through the ring. The polygon engine finds atomic centres with three or more atoms bonded and draws a polygon around that atom. This is only done for atoms that are not the common organic types.

I think these are useful engines that have been prompted by user requests. May be not as exciting as the ribbon engine or the surface support we are working on but useful to display certain structures. The ring engine has not been without its problems though as you can probably spot in the image below showing the ring engine rendering a carbon nanotube.

The lighting just flips at certain points - the same happens as you rotate molecules that are using the ring or polygon engine. I guess the lighting is flicking on at certain angles. I would be interested if anyone with more OpenGL knowledge than I might know how I can improve the rendering. Some rings are near white and some of the back rings don't seem to be drawn - that could actually be drawing order though which is always tough to get right.

Comments and tips are always welcome. The big challenge right now is getting surface support added in. This is something I think we really need and I would use in my daily work as I am sure many others would. I have been reading up on marching cubes and stuff but would welcome tips in that area too!

Over the last couple of days I have made a few commits to improve the ribbon support in Avogadro. Many thanks to Geoff for fixing some bugs we found in Open Babel and to Thomas Margraf for his helpful suggestions and for letting me take a look at a little of his code that renders ribbons.

Right now the ribbon is drawn as a tube between the carbon atoms of the backbone and so isn't really a ribbon at all. Then I looked at the image and it is looking quite a bit like a ribbon. The code is checked in and so feel free to take a look at it. I am using NURBS but am far from an expert on their use. My laptop failure has lost some of the tuning I had done too - this is my first screenshot from Apple Mac OS X Leopard!

Points if you can tell me why the alpha helix looks pretty good despite me having not yet implemented anything to find the correct plane! Still needs some tuning but I hope you will agree that this is looking pretty nice now. It has also led to some optimisation of molecule loading but I am sure that there are lots of biologists who will tell me it still isn't quite right.

Update: Geoff has now shown me how to do screen capture and so here is a pretty video of Avogadro rendering ribbons in action!

Today we released Avogadro 0.2 which features many improvements since the 0.1 release. It has lots of the new features I worked on during my Google Summer of Code project along with some great work by other contributors such as the bond centric manipulation tool coded by Shahzad Ali, Ross Braithwaite and James Bunt.

Geoffrey Hutchison announced the release earlier today, on what I have been informed is mole day due to the date format used in the US - 10/23 corresponding to the 10^23 of Avogadro's number. So today seemed like an even more fortuitous time to make a release of Avogadro!

Today also marked my first commit to the Gentoo repository in quite some time to add this release to Gentoo's ebuild repository. If you have been waiting for me to add/update packages and I haven't gotten around to it due to life being so busy now would be a good time to poke me. I will also hopefully be completing the ribbon support I have already talked about and that should make it into a release soon but wasn't ready for this release.

We would love to receive feedback on the latest release of Avogadro. These new features will also make their way into Kalzium in KDE 4.1.

I term it as "ribbon" because it really isn't a ribbon. I haven't figured out how to calculate the plane the ribbon lies on and right now it is a series of cylinders drawn from point to point. I hope you will agree that it already looks fairly effective and can certainly show the nature of the secondary structure of biological molecules.

There is still a lot to do such as draw the tube in a more efficient manner as a continuous OpenGL construct and draw each residue separately. It would also be good to be able to draw ribbons but this may take a deeper understanding of ribbons and how they are visualised than I have. If there are any biologists out there who want to help me out then tips, or even better patches, would be great fully received

Originally I thought I would just attend the aKademy conference on the weekend and return home after that. I submitted an abstract for my talk on Gentoo and KDE and had hoped to meet some KDE developers and see if we could work with KDE better. Later I applied for my Google Summer of Code project working on the Kalzium 3D molecular editor and was accepted. This caused me to change my plans and stay for the entire week and I am really glad I did

I have been to quite a few open source conferences over the years but I have to say aKademy is my favourite so far. There was a great atmosphere and lots of very intelligent and dedicated hackers present. There were also lots of laughs and humour along with important discussions and debates. I got to meet so many people who work on KDE and whose blogs I have been reading for years. My only regret is that I couldn't get out for a few more beers in the evenings...

I thought it would be a good idea to camp as aKademy was in June and the odds were good that the weather would be fair. In the days leading up to my departure Sheffield flooded and I think it rained every day of aKademy (some days solidly throughout the day). On Sunday evening we came back to a rather large pond at the side of our tent that had previously been a patch of grass with a dip in it. I chose where to pitch the tent well though and the tent remained out of harms way

On Saturday and Sunday I attended the Sheffield Open Street Map party. All this is pretty new to me but I was certainly interested in the concept of creating a free and openly editable map of the world. The party was kindly hosted and organised by the IET and ShefLUG in cooperation with the OSM guys who brought along some GPS gear for those of us who don't have our own.

We got a crash course on how to use the handheld GPS devices and then went out to do some mapping. I partnered up with Dan on Saturday and we wandered around the back streets quite close to the Mappin building which was our base. We took it in turns making notes and using the GPS and I think we managed to map just about every street in the patch we covered as well as quite a few footpaths.

Above is a copy of the OSM map of Sheffield as it stands right now after some of our weekends work has been added. It still hasn't rebuilt the tiles with the extra bits I have just added but I am sure it will get on there soon. I really enjoyed doing the mapping and hope to do more in the future. There are so many good reasons to help out with this project if you can.

Marble is planning on adding support for OSM, it is also a great way to do something pretty geeky and get out of the office for a while I love the freedom it gives you to add data, correct mistakes and use the data for whatever you like. Personally I am thinking of adding dive sites as I go to them and using Marble to display all the sites I have been to. I could also use it to map out runs and check on my performance as I train.