Marcus D. Hanwell's Blog - KDE

The Kalzium Molecular Editor

marcus@cryos.net (Marcus D. Hanwell) — Mon, 26 May 2008 18:28:23 +0100

Some of you with long memories may recall that I worked on the Kalzium molecular editor for my Google Summer of Code (TM) project last summer. The observant among you may have also noticed that Kalzium in KDE 4.0 featured a molecular viewer. Well the freeze meant that much of my work didn't make it into KDE 4.0. This is the case with many of our current GSoC students (Naomi among them) who are just starting to code as KDE 4.1 is being frozen and released.

Since last summer life has been really hectic, the shipment of my household possessions from the UK to the US was delayed significantly by an incompetent shipping company which also proceeded to break many things... This led to me having no Linux system to develop on for quite some time, and then less time to develop due to having to deal with the fallout of a botched delivery. Still, I am very pleased to be able to show off some of my latest work which made it into KDE 4.1 and will be in KDE 4.1 beta1 (due out in a few days).

I was able to expose some of the extra display types I added along with support for drawing and editing molecules. You can also measure distances and angles of the molecule you are looking at and do some geometry optimisation. Given a little more time I would have liked to expose some of the manipulation features but this new six month cycle was so short and my time was constrained anyway. It will make it into 4.2 at least.

I am looking forward to seeing the animation support Naomi is working on now and am very happy to be able to mentor her over the summer. I think high quality, open source educatonal programs are very important and hope that this editor will be useful in that role. The Avogadro library and application should hopefully be able to satisfy college students and researchers. With the scripting support and open source nature of the two projects I think they interact very well and complement one another.

As always I would love to hear your feedback on the Kalzium changes and the other work we are doing.

Avogadro 0.8.0 Released

marcus@cryos.net (Marcus D. Hanwell) — Fri, 23 May 2008 10:29:44 +0100

On Monday we officially released Avogadro 0.8.0. I think this is a great release with many improvements over 0.6.1. We have been working really hard on improving the stability of Avogadro and I hope this shows in this release. There have also been some very nice improvements in speed and lots of new features.

The screen shot above shows an imported Gaussian formatted checkpoint file displaying the HOMO molecular orbital. The import of output from quantum codes and subsequent calculation of MOs is at a very early stage but you should take a look at it in the extensions menu if you are interested. It currently only calculates s, p and d orbitals and can only parse formatted checkpoint files from Gaussian. The goal is to add at least f orbitals and many more quantum codes.

I don't think I ever talked about the new periodic table QGraphicsView that I am very proud of. It allows you to select elements that are not listed in the combobox, shows the colour they will have on screen and is about as compact as I can make it. I have already packaged Avogadro 0.8.0 for Gentoo, we have binaries (Windows, Mac) and source available for download and should hopefully have packages available for Debian and Ubuntu very soon.

There were a few features that didn't make it in and a few bug fixes we found just after release. Once we have ironed out a few more bugs and received more feedback we will make a bug fix release. I was amazed to see that Avogadro is currently shooting up the Naomi Fox's application for a Google Summer of Code (TM) was successful. Some of you may remember that I participated last year as a student and was also lucky enough to attend aKademy. This year I am no longer a student but I am very pleased to be able to participate as a mentor and hopefully pass on some of what I learned last year.

I hope you will all join me in welcoming Naomi to our development community. As I moved to the US last year we are actually in the same timezone which will make mentoring easier. It does mean getting to aKademy will be a lot more difficult but may be I can attend the first KDE Americas event. I am really looking forward to working with Naomi over the summer and I think this is a really important set of features still missing in Kalzium.

Looking forward to a great summer, my first in Pittsburgh and my first time as a mentor.

Getting Back to Gentoo and KDE Development

marcus@cryos.net (Marcus D. Hanwell) — Sun, 09 Mar 2008 14:58:00 +0000

After I got back from the UK in January my wife came to join me. She brought with her my old Acer Ferrari nearly working shy of a new hard drive. I got one from NewEgg at a fairly reasonable price and installed it. The drive arrived with a few bent pins, packaging isn't their strong point obviously, and took twice as long as they said but hey at least I had it. I had been stranded in America without Linux for more than three months and so another few days wasn't going to kill me.

Once I got the drive in it seemed the slot loading CD drive and got itself in a twist and would not load any CDs. An hour later and lots of pieces on the table I had reset it, put it back together (a few times) and got a Gentoo LiveCD in there. So I was in business. I had considered installing kubuntu but I wasn't getting on with it in the VM I had at work and I wanted to get back to Gentoo development too.

Everything looked good. Got an X server up and running. The ATI binary blob was as unstable as ever. Thanks to nerdboy (I think) I got the open source r300 driver working. This is much more stable although missing several OpenGL features we use in Avogadro such as smooth triangle shading, changing vertex colours in drawing operations etc. Other than that it works much better than the binary blob and is far more stable. I like stability in a system.

I have KDE 4 trunk running and am using it as my main desktop right now. Still got some rough edges I haven't had time to figure out just yet such as kmail refusing to save my IMAP accounts, kopete refusing to connect to GTalk and general saving issues most of the core seems pretty solid. I am compiling Thunderbird right now so that I have something. I will hopefully be getting back to Kalzium development pretty soon. If I find the time I would love to help out with KDE in a more general sense as well as helping out with the Gentoo packages.

My desktop is apparently in a customs area somewhere in New York. Delay after delay after delay with the shipping of my household possessions from the UK. So I am still without desktop hoping that an "intensive exam" is not too expensive and does not involve breaking any of my stuff :/ We shall see. Fingers crossed I will actually have furniture and my desktop computer back within a few weeks. That would allow me to be so much more productive having my dual core Gentoo desktop back, creature comforts and all that good stuff.

So keep your fingers crossed for me. Hopefully you will be seeing a lot more commits coming from me in the KDE and Gentoo repositories as well as the Avogadro work I have managed to continue to do on the MacBook Geoff so kindly loaned me whilst I was laptopless (is that a word? It should be!) Not sure that was a sentence now either...

Avogadro 0.6.1 Released

marcus@cryos.net (Marcus D. Hanwell) — Sun, 09 Mar 2008 14:30:00 +0000

I am pleased to announce that I tagged and released Avogadro 0.6.1 yesterday evening. This is a bug fix release which fixes one pretty large bug that slipped through - the OpenGL context was lost if switching between virtual desktops, multiple views etc rendering the OpenGL window useless unless the application was restarted. As such I would encourage anyone running Avogadro 0.6.0 to upgrade to this new version. It also features several smaller bug fixes and feature enhancements.

The screen shot above shows Avogadro 0.6.1 running in a KDE 4 session. One of the small visual tweaks I made was to add a second light source to our default OpenGL scene which really helps to illuminate the other side of the scene. Thanks go out to Albert for his suggestion of adding another light source. Hopefully there are no really big bugs remaining but Avogadro is still in the beta stages of its development. It is rapidly approaching a stable release though and I am very happy with our progress so far.

We would love to hear what you think of Avogadro. I had one person question why we always have to use the latest and greatest version of OpenBabel and felt I should offer some explanation. Many of the features exposed in Avogadro use functions and structures in OpenBabel. I myself was quite heavily involved in improving OpenBabel's support for Gaussian cube files and the cube format so that we could load and display orbitals for example. As such we often add new features or fix bugs in OpenBabel trunk and so a new release of OpenBabel must be used in order for everything to work.

There are already ebuilds for this latest version in the Gentoo tree. Ubuntu/Debian builds are in the process of being built. We should hopefully have Mac and Windows binaries very soon too.

I am headed to a meeting in the UK where Donald and I will be talking with other scientists about visualisation in chemistry and related areas. We will of course be showing off Avogadro as well as talking with many other people working in this area. I am very much looking forward to it and hope that this will lead to further innovation in the Avogadro project as well as the open source chemistry movement in general. It will of course be great to have a full English breakfast and some real ale too!

Avogadro 0.6 Released

marcus@cryos.net (Marcus D. Hanwell) — Fri, 29 Feb 2008 22:01:45 +0000

Earlier today we released Avogadro 0.6. This release is quite overdue and we had originally planned to make a release around the new year. This release has many, many new features that have been implemented since the last release.

I am especially proud of the new support for displaying surfaces. This initially led to the surface engine that displays a Van der Waals surface. Tim added the ability to map the electostatic surface potential onto that. I then began work on adding support for displaying orbitals. This led to me getting commit access to OpenBabel and making quite a few commits as I got Gaussian cube loading working. While I was there I couldn't resist improving the Grid classes (still a little more I would like to add in).

Another new feature I am really pleased with is the QGraphicsView based periodic table. I think it works very nicely and I am hoping to add this code to Kalzium. I think the Avogadro library is in great shape for me to begin porting Kalzium to use it. Tim and Geoff have been doing some amazing work in OpenBabel on improving the force fields used for geometry optimisations.

We have also been doing a lot of work to improve Avogadro's performance when rendering large systems. We have added quickRender functions and OpenGL display lists to improve interactivity as well as threading calculations where this made sense. We have a cool little colour widget Geoff coded too that allows colours to be displayed and picked easily, improved default layout and one of my personal favourites - persistent settings in most of Avogadro.

There are certainly some bugs remaining. I would like to get to the bottom of as many as I can and hope to make one or two bug fix releases in the 0.6 branch. We would love to get feedback from people. We have already made a source release and a Mac binary. I will be adding Gentoo ebuilds shortly and we will hopefully get a Windows binary out by early next week.

Hope you enjoy this release. I think we are getting close to a stable API that we can call 1.0 and have most of the core features I was hoping to get into Avogadro. Our framework is extremely modular and extensible and I hope that we will be able to build up a community around Avogadro. It was also great to be able to make this release on 29 February - we have so few of them

Avogadro: New Orbital Support and Gaussian Cube Format

marcus@cryos.net (Marcus D. Hanwell) — Tue, 19 Feb 2008 21:00:00 +0000

So over the weekend I spent quite a lot of time hacking away at OpenBabel working on the Gaussian cube format support I needed to get working in order to be able to visualise electronic orbitals. The initial support was taken from Molekel but contained no error checking and it was not reading in the example cube file I had.

Thankfully a colleague pointed out a page with details on the Gaussian cube format and I used this as a basis to get it working in OpenBabel with better error checking and a more resilient tokenisation of the cube points. I made my first few commits to OpenBabel over the weekend and got at least simple Gaussian cube files loading that only contain one cube. More will follow I am sure.

After I got the file loading sorted a few more changes to Avogadro got me my first ever orbitals in Avogadro - the benzene HOMO! I believe the cube file came from the JMol test files. Geoff informed me that chemists for some reason see positive as blue and negative as red. That seemed very strange to me and I had initially put it the other way around. Growing up playing with electronics it seemed to me that positive should be red just as the wires in a circuit have a red positive... Other than that orbitals seemed to be working well and I was very pleased.

Next I started to load up some other cube files I had been given as examples. I thought the above image of the electrostatic potential of CH₃Cl looked very nice. When I showed Geoff he also liked it and we wondered if it had been visualised in this way before. I know normally the cube would be mapped onto the Van der Waals surface of a molecule.

Anyway I was pleased and also felt more productive as I now have my laptop back up and running and am developing on Gentoo again. It would be great to hear what other people think of this new support. I am already working on various improvements to the code and getting ready for another release of Avogadro!

Avogadro: Surface Support

marcus@cryos.net (Marcus D. Hanwell) — Mon, 11 Feb 2008 11:50:00 +0000

I am pleased to announce that we now have working surface support in Avogadro. We originally began working on support for surfaces around November time when Geoff improved the support in OpenBabel for grids and added some initial code to the Avogadro repository. Quite quickly it became evident that this algorithm did not deal well with data sets containing discrete objects that you wanted to polygonise.

I began a search of the available algorithms and came across a very nice guide to the marching cube algorithm with example code. I had a brief look for a C++ implementation and didn't find anything initially, began implementing what was on his page and then came across Zhu3d which is a Qt 4 based program that actually uses an implementation of the algorithms described in the guide and references them. As it is GPL licenced code I imported the relevant class into our repository and began the task of adapting it to fit into our framework.

With some code from Geoff, more pointers from Geoff, some reading and quite a bit of help from Tim (not sure if he has a web page/blog) we got it working. Since then Tim has actually fixed a big bug in OpenBabel that was causing crashes and done more work on the IsoGen class. The screenshots above show the results of our early work where initially we got points and then lines working. I was very pleased at this stage as it adds a feature that had been missing for quite some time to the Avogadro framework.

I then got filled triangles added with transparency support, allowing us to visualise the underlying structure of the molecule and map other parameters onto the surface. Tim then added electrostatic charge mapping to the colour of the surface which I hope you agree is already looking very good.

There is still quite a bit left to do. I need to move the class into libavogadro and out of the engines, integrate it into our Painter API, implement caching and am thinking of using the Qt Concurrent framework to do intelligent multithreading once Qt 4.4 is released as surface generation is quite slow for big molecules. I also need to link the mesh size to the global quality level and see what other optimisations might be possible. Another big one is adding support for visualising molecular orbitals. Geoff has already done a lot of work on the back end in OpenBabel for this and so hopefully it will not take me long to get it added in.

Lots to do but some great progress already. I would love to know what you think. Once we have been able to polish this new feature a little I hope we can get a release out of the door so that more people can test out these new features and let us know what they think, point out bugs and shower us with compliments Until then I hope you enjoy the screenshots!

Note: For some reason the exported graphics shown in this post were not antialiased. They are on screen and I need to look into why they are not when I export graphics in this way.

KDE 4 Release and Party

marcus@cryos.net (Marcus D. Hanwell) — Sat, 12 Jan 2008 11:42:05 +0000

I am sure you have heard by now about the KDE 4.0.0 release made yesterday. Currently my desktop machine is on a boat in the middle of the Atlantic somewhere, my laptop is broken and I only have access to Apple systems... I was able to check out KDE 4 and test it in a VM and it is looking good.

I haven't been able to help with Gentoo packaging of KDE 4 very much at all which is a shame. I should get my desktop back at the end of the month in my new home in Pittsburgh. I also couldn't attend the release event as I am attending my graduation ceremony on the 17th of January. Hope you guys really enjoy the event and if I could have moved things around to attend I certainly would have.

There is lots of stuff I hope to be able to do for KDE 4.1, and more specifically Kalzium, once I get my desktop back at the end of the month. Avogadro already has lots of improvements including a KGraphicsView based periodic table which I think Carsten is interested in me getting into Kalzium 4.1.

Enjoy the release event and remember to raise a glass to those of us unable to make it for various reasons

Avogadro: Just Hit Commit 1000!

marcus@cryos.net (Marcus D. Hanwell) — Sat, 15 Dec 2007 17:52:42 +0000

I just made a few commits using my trusty git svn dcommit (I am still really loving git and it looks like the bug I reported may be fixed in it now too). Then I saw the commit numbers - we just hit commit 1000. Development has really been accelerating recently and I haven't had time to write about as much as I wish I could.

Hopefully I will get a chance to take some screenshots and show off some of the latest stuff. I have coded a QGraphicsView based periodic table I am pretty happy with. It is compact, allows elements to be chosen, shows our colour scheme and even displays a little extra information on the selected element. Then there is the ribbon engine which is coming along but needs some loving attention from some bio people.

I am very pleased with the new quick rendering stuff we have added. It still has its issues. I had fixed some but some of Benoit's recent commits seem to have reintroduced the cache problem upon engine changes. Tim has added a new hydrogen bond engine which looks great thanks to some stipple line stuff from Geoff. There is some great stuff in the properties dialogs too along with lots of improvements to our force fields.

There is certainly lots more for us to do. The biggest thing I still haven't gotten in yet is surface support! I really want to get that sorted as I think it will be both useful and should look great. Along with some extra dialogs for graphics export, POV-Ray rendering etc.

Git, SVN and Development

marcus@cryos.net (Marcus D. Hanwell) — Sun, 02 Dec 2007 20:42:57 +0000

It seems I have read quite a few posts in the last half a year or so on the virtues on decentralised version control and how good git it. Before my laptop died I had installed git and was using it for development. Unfortunately that was pretty short lived due to the failure of my laptop. Now all of my development is taking place on Apple Mac machines using Leopard (I used Tiger for a while too).

Recently I built git from source on the laptop I am using and checked out the full Avogadro subversion repository using the following commands.

git svn init -t tags -b branches -T trunk https://avogadro.svn.sourceforge.net/svnroot/avogadro git svn fetch

Then I went and grabbed a coffee while it imported over 800 commits which is pretty small compared to many of the other repositories I have heard about being imported. It didn't take too long, but I would happily make my git repository available should anyone want a copy.

I have found git to be very fast which is absolutely great. There are some things which I have found a little confusing while getting to grips with git and its interaction with the subversion repository.

For my day to day work I need very few commands. git svn rebase synchronises with the subversion repository. I really like this way of updating too as it displays the diffs for the commits and the log messages. git commit -a commits all local changes to the local repository. git status shows the current status of the repository, git show shows local changes already committed and git svn dcommit pushes my local changes to the subversion repository.

It is this last feature I think I like the most. I can queue up multiple change sets which might actually break Avogadro, giving nice bite size commits as I make progress, but only committing them once I am done and everything is working (at least reasonably well). Then there is the potential for doing this when off line which is also great.

One of the most confusing things I found was how to nuke local changes if I didn't want to keep them. Deleting the file and updating doesn't work as it does with subversion. Also revert didn't do what I was after. I finally found git checkout which was not what I was expecting. Issuing the command with no arguments resets the repository to the last committed state. Adding filenames as arguments reverts just those files.

Another one I haven't found the answer to is files reappearing that we long since deleted. The avogadro.pro file is the main one left but a set of directories also reappeared. They are not there when checking out using just subversion. If anyone has the answer I would love to know. If I git rm the file the commit to subversion does nothing and it loops round.

Other than that I am really happy with git. It can be a little too easy to use the wrong commands and wipe out changes so I am moving quite gingerly at times but I think it is well worth the trade off. A few guides such as this one, this one and of course this have helped me along the way.

UPDATE: Just to note that I was using the git 1.5.3 release compiled from source.

Avogadro: New Ring & Polygon Engines

marcus@cryos.net (Marcus D. Hanwell) — Mon, 26 Nov 2007 22:58:52 +0000

I recently added a couple of new engines to Avogadro and the screen shot below shows them both. The ring engine finds all the rings of the molecule and draws a transparent plane through the ring. The polygon engine finds atomic centres with three or more atoms bonded and draws a polygon around that atom. This is only done for atoms that are not the common organic types.

I think these are useful engines that have been prompted by user requests. May be not as exciting as the ribbon engine or the surface support we are working on but useful to display certain structures. The ring engine has not been without its problems though as you can probably spot in the image below showing the ring engine rendering a carbon nanotube.

The lighting just flips at certain points - the same happens as you rotate molecules that are using the ring or polygon engine. I guess the lighting is flicking on at certain angles. I would be interested if anyone with more OpenGL knowledge than I might know how I can improve the rendering. Some rings are near white and some of the back rings don't seem to be drawn - that could actually be drawing order though which is always tough to get right.

Comments and tips are always welcome. The big challenge right now is getting surface support added in. This is something I think we really need and I would use in my daily work as I am sure many others would. I have been reading up on marching cubes and stuff but would welcome tips in that area too!

Improved Ribbon Rendering

marcus@cryos.net (Marcus D. Hanwell) — Tue, 06 Nov 2007 15:30:00 +0000

Over the last couple of days I have made a few commits to improve the ribbon support in Avogadro. Many thanks to Geoff for fixing some bugs we found in Open Babel and to Thomas Margraf for his helpful suggestions and for letting me take a look at a little of his code that renders ribbons.

Right now the ribbon is drawn as a tube between the carbon atoms of the backbone and so isn't really a ribbon at all. Then I looked at the image and it is looking quite a bit like a ribbon. The code is checked in and so feel free to take a look at it. I am using NURBS but am far from an expert on their use. My laptop failure has lost some of the tuning I had done too - this is my first screenshot from Apple Mac OS X Leopard!

Points if you can tell me why the alpha helix looks pretty good despite me having not yet implemented anything to find the correct plane! Still needs some tuning but I hope you will agree that this is looking pretty nice now. It has also led to some optimisation of molecule loading but I am sure that there are lots of biologists who will tell me it still isn't quite right.

Update: Geoff has now shown me how to do screen capture and so here is a pretty video of Avogadro rendering ribbons in action!

My Laptop is Fried - Limited Development Activity

marcus@cryos.net (Marcus D. Hanwell) — Tue, 06 Nov 2007 09:49:04 +0000

Since moving out to Pittsburgh I have been using my Acer Ferrari laptop at home for development, keeping in touch with people back home and trying to figure out some of the weirder rules out here such as driving licenses in this state. Last night I booted up when I got home and after logging in everything went a little funky - got I/O errors when typing ls for example.

I tried rebooting but it wouldn't even get to the GRUB prompt. So I got out an old Gentoo LiveCD which I had luckily left in my laptop bag. That got so far then failed to mount the drive. After a few reboots and a little wine (to calm my nerves ) I managed to access the drive. I thought I would try copying some stuff across to my external hard drive and ended up getting more I/O errors and some memory errors for good measure. Half the time when I booted from the LiveCD the kernel panicked when attempting to mount the hard drive partitions.

So today I am a very unhappy English man in Pittsburgh. Due to me having spent far more than I budgeted for moving out here, and then more again on flying my dog out, I don't foresee being able to replace it any time soon. This means my development activity will be limited. Not sure if compiling part of a KDE 4 checkout might have been what finished it off. Looks like some kind of motherboard issue. The trackpad stopped working months ago, now I am getting random memory and I/O errors and the CPU has been running hotter and hotter recently. It served me for just over two years which isn't great but I worked it pretty hard and it travelled a fair part of the globe with me too.

Still can't help being cheered up a little as I will be picking Dax, my overgrown German Shepherd dog up from Dulles Washington International Airport tomorrow (not Dallas as it sounded like on the message they left me which made me very unhappy when I first heard).

Update: My very nice and generous new boss has very kindly offered to let me use the group laptop in the interim. It is a MacBook Pro though using something called Mac OS X Leopard. I can keep hacking on Avogadro with it and I even know how to take screen shots with it now too! It is also capable of checking email and browsing the web so my Internet connection in the apartment won't go to waste. I am considerably happier now but will miss my KDE 4 and Linux fix

Avogadro 0.2 Released Today

marcus@cryos.net (Marcus D. Hanwell) — Tue, 23 Oct 2007 22:05:00 +0100

Today we released Avogadro 0.2 which features many improvements since the 0.1 release. It has lots of the new features I worked on during my Google Summer of Code project along with some great work by other contributors such as the bond centric manipulation tool coded by Shahzad Ali, Ross Braithwaite and James Bunt.

Geoffrey Hutchison announced the release earlier today, on what I have been informed is mole day due to the date format used in the US - 10/23 corresponding to the 10^23 of Avogadro's number. So today seemed like an even more fortuitous time to make a release of Avogadro!

Today also marked my first commit to the Gentoo repository in quite some time to add this release to Gentoo's ebuild repository. If you have been waiting for me to add/update packages and I haven't gotten around to it due to life being so busy now would be a good time to poke me. I will also hopefully be completing the ribbon support I have already talked about and that should make it into a release soon but wasn't ready for this release.

We would love to receive feedback on the latest release of Avogadro. These new features will also make their way into Kalzium in KDE 4.1.