Marcus D. Hanwell's Blog

I am very pleased to be able to announce that Naomi Fox's application for a Google Summer of Code (TM) was successful. Some of you may remember that I participated last year as a student and was also lucky enough to attend aKademy. This year I am no longer a student but I am very pleased to be able to participate as a mentor and hopefully pass on some of what I learned last year.

I hope you will all join me in welcoming Naomi to our development community. As I moved to the US last year we are actually in the same timezone which will make mentoring easier. It does mean getting to aKademy will be a lot more difficult but may be I can attend the first KDE Americas event. I am really looking forward to working with Naomi over the summer and I think this is a really important set of features still missing in Kalzium.

Looking forward to a great summer, my first in Pittsburgh and my first time as a mentor.

Last week I was privileged to be invited to speak at a meeting about molecular modelling with a focus on tools, GUIs and visualisation. The meeting was held at the Holiday Inn in Runcorn and the Daresbury Laboratory (England). I wasn't expecting to be back in England quite so soon, having only just returned to Pittsburgh at the end of January.

The meeting was a great opportunity to present some of the latest work I and others in the Avogadro and OpenBabel communities have been doing to create tools that enable the building of molecules and structures, as well as their visualisation. It was also a great chance to hear some very interesting talks by the developers of other building tools and some quantum codes. Donald and I were also invited to Daresbury Laboratory to work with some of the CCP1GUI developers.

I presented my talk on Avogadro on Wednesday morning and have made the slides available here. Donald gave an introduction to Avogadro, some of the history and the architecture before I gave my presentation. We finished by taking questions while I demonstrated the Avogadro application. I think it was extremely productive. We had many more conversations over dinner and drinks later as well as in a workshop setting on Thursday afternoon.

It was great to be able to put a face to a few of the names and discuss current issues more informally in the evening. The talks were all of a very high quality and from a varied list of speakers from other open source projects, some of the free quantum codes as well as commercial products. I have come away from the meeting with a much better appreciation of the needs in the community and I feel that Avogadro is in a great position to fill the apparent void.

I am glad that we were able to get surface and orbital support working in Avogadro before the meeting. Right now we only support Guassian cube files but the implementation is general enough that I will be able to add support for further formats. I really think that if we can get enough people collaborating on a common project everyone can get the tool they need to effectively do their research at a much lower investment than could be achieved by working on many separate projects.

I met Tristan Youngs, the developer of Aten, who had implemented some really nice features in his molecular builder that is much more focussed on molecular mechanics. It is well worth checking out. As is Zeobuilder which was developed by Toon Verstraelen. They both implement some great features and have strengths in different areas. Of course my dream is to integrate many of these features via Avogadro plugins and have one editor which is capable of being used in a diverse range of applications.

It was also great to speak to Mario Valle who is doing some very interesting work in the area of new visualisation methods and supports a large user base of computational chemistry users. There were of course so many other talks but you can look at the schedule yourself and I think the slides of all the talks should be available in the near future.

I feel sure that many good things will come out of this meeting and hope to be able to attend similar meetings in the future. I would like to thank Jens once again for hosting the meeting and taking care of everything. I hope to see some patches and/or commits from him in the near future

After I got back from the UK in January my wife came to join me. She brought with her my old Acer Ferrari nearly working shy of a new hard drive. I got one from NewEgg at a fairly reasonable price and installed it. The drive arrived with a few bent pins, packaging isn't their strong point obviously, and took twice as long as they said but hey at least I had it. I had been stranded in America without Linux for more than three months and so another few days wasn't going to kill me.

Once I got the drive in it seemed the slot loading CD drive and got itself in a twist and would not load any CDs. An hour later and lots of pieces on the table I had reset it, put it back together (a few times) and got a Gentoo LiveCD in there. So I was in business. I had considered installing kubuntu but I wasn't getting on with it in the VM I had at work and I wanted to get back to Gentoo development too.

Everything looked good. Got an X server up and running. The ATI binary blob was as unstable as ever. Thanks to nerdboy (I think) I got the open source r300 driver working. This is much more stable although missing several OpenGL features we use in Avogadro such as smooth triangle shading, changing vertex colours in drawing operations etc. Other than that it works much better than the binary blob and is far more stable. I like stability in a system.

I have KDE 4 trunk running and am using it as my main desktop right now. Still got some rough edges I haven't had time to figure out just yet such as kmail refusing to save my IMAP accounts, kopete refusing to connect to GTalk and general saving issues most of the core seems pretty solid. I am compiling Thunderbird right now so that I have something. I will hopefully be getting back to Kalzium development pretty soon. If I find the time I would love to help out with KDE in a more general sense as well as helping out with the Gentoo packages.

My desktop is apparently in a customs area somewhere in New York. Delay after delay after delay with the shipping of my household possessions from the UK. So I am still without desktop hoping that an "intensive exam" is not too expensive and does not involve breaking any of my stuff :/ We shall see. Fingers crossed I will actually have furniture and my desktop computer back within a few weeks. That would allow me to be so much more productive having my dual core Gentoo desktop back, creature comforts and all that good stuff.

So keep your fingers crossed for me. Hopefully you will be seeing a lot more commits coming from me in the KDE and Gentoo repositories as well as the Avogadro work I have managed to continue to do on the MacBook Geoff so kindly loaned me whilst I was laptopless (is that a word? It should be!) Not sure that was a sentence now either...

I am pleased to announce that I tagged and released Avogadro 0.6.1 yesterday evening. This is a bug fix release which fixes one pretty large bug that slipped through - the OpenGL context was lost if switching between virtual desktops, multiple views etc rendering the OpenGL window useless unless the application was restarted. As such I would encourage anyone running Avogadro 0.6.0 to upgrade to this new version. It also features several smaller bug fixes and feature enhancements.

The screen shot above shows Avogadro 0.6.1 running in a KDE 4 session. One of the small visual tweaks I made was to add a second light source to our default OpenGL scene which really helps to illuminate the other side of the scene. Thanks go out to Albert for his suggestion of adding another light source. Hopefully there are no really big bugs remaining but Avogadro is still in the beta stages of its development. It is rapidly approaching a stable release though and I am very happy with our progress so far.

We would love to hear what you think of Avogadro. I had one person question why we always have to use the latest and greatest version of OpenBabel and felt I should offer some explanation. Many of the features exposed in Avogadro use functions and structures in OpenBabel. I myself was quite heavily involved in improving OpenBabel's support for Gaussian cube files and the cube format so that we could load and display orbitals for example. As such we often add new features or fix bugs in OpenBabel trunk and so a new release of OpenBabel must be used in order for everything to work.

There are already ebuilds for this latest version in the Gentoo tree. Ubuntu/Debian builds are in the process of being built. We should hopefully have Mac and Windows binaries very soon too.

I am headed to a meeting in the UK where Donald and I will be talking with other scientists about visualisation in chemistry and related areas. We will of course be showing off Avogadro as well as talking with many other people working in this area. I am very much looking forward to it and hope that this will lead to further innovation in the Avogadro project as well as the open source chemistry movement in general. It will of course be great to have a full English breakfast and some real ale too!

Earlier today we released Avogadro 0.6. This release is quite overdue and we had originally planned to make a release around the new year. This release has many, many new features that have been implemented since the last release.

I am especially proud of the new support for displaying surfaces. This initially led to the surface engine that displays a Van der Waals surface. Tim added the ability to map the electostatic surface potential onto that. I then began work on adding support for displaying orbitals. This led to me getting commit access to OpenBabel and making quite a few commits as I got Gaussian cube loading working. While I was there I couldn't resist improving the Grid classes (still a little more I would like to add in).

Another new feature I am really pleased with is the QGraphicsView based periodic table. I think it works very nicely and I am hoping to add this code to Kalzium. I think the Avogadro library is in great shape for me to begin porting Kalzium to use it. Tim and Geoff have been doing some amazing work in OpenBabel on improving the force fields used for geometry optimisations.

We have also been doing a lot of work to improve Avogadro's performance when rendering large systems. We have added quickRender functions and OpenGL display lists to improve interactivity as well as threading calculations where this made sense. We have a cool little colour widget Geoff coded too that allows colours to be displayed and picked easily, improved default layout and one of my personal favourites - persistent settings in most of Avogadro.

There are certainly some bugs remaining. I would like to get to the bottom of as many as I can and hope to make one or two bug fix releases in the 0.6 branch. We would love to get feedback from people. We have already made a source release and a Mac binary. I will be adding Gentoo ebuilds shortly and we will hopefully get a Windows binary out by early next week.

Hope you enjoy this release. I think we are getting close to a stable API that we can call 1.0 and have most of the core features I was hoping to get into Avogadro. Our framework is extremely modular and extensible and I hope that we will be able to build up a community around Avogadro. It was also great to be able to make this release on 29 February - we have so few of them

So over the weekend I spent quite a lot of time hacking away at OpenBabel working on the Gaussian cube format support I needed to get working in order to be able to visualise electronic orbitals. The initial support was taken from Molekel but contained no error checking and it was not reading in the example cube file I had.

Thankfully a colleague pointed out a page with details on the Gaussian cube format and I used this as a basis to get it working in OpenBabel with better error checking and a more resilient tokenisation of the cube points. I made my first few commits to OpenBabel over the weekend and got at least simple Gaussian cube files loading that only contain one cube. More will follow I am sure.

After I got the file loading sorted a few more changes to Avogadro got me my first ever orbitals in Avogadro - the benzene HOMO! I believe the cube file came from the JMol test files. Geoff informed me that chemists for some reason see positive as blue and negative as red. That seemed very strange to me and I had initially put it the other way around. Growing up playing with electronics it seemed to me that positive should be red just as the wires in a circuit have a red positive... Other than that orbitals seemed to be working well and I was very pleased.

Next I started to load up some other cube files I had been given as examples. I thought the above image of the electrostatic potential of CH₃Cl looked very nice. When I showed Geoff he also liked it and we wondered if it had been visualised in this way before. I know normally the cube would be mapped onto the Van der Waals surface of a molecule.

Anyway I was pleased and also felt more productive as I now have my laptop back up and running and am developing on Gentoo again. It would be great to hear what other people think of this new support. I am already working on various improvements to the code and getting ready for another release of Avogadro!

I am pleased to announce that we now have working surface support in Avogadro. We originally began working on support for surfaces around November time when Geoff improved the support in OpenBabel for grids and added some initial code to the Avogadro repository. Quite quickly it became evident that this algorithm did not deal well with data sets containing discrete objects that you wanted to polygonise.

I began a search of the available algorithms and came across a very nice guide to the marching cube algorithm with example code. I had a brief look for a C++ implementation and didn't find anything initially, began implementing what was on his page and then came across Zhu3d which is a Qt 4 based program that actually uses an implementation of the algorithms described in the guide and references them. As it is GPL licenced code I imported the relevant class into our repository and began the task of adapting it to fit into our framework.

With some code from Geoff, more pointers from Geoff, some reading and quite a bit of help from Tim (not sure if he has a web page/blog) we got it working. Since then Tim has actually fixed a big bug in OpenBabel that was causing crashes and done more work on the IsoGen class. The screenshots above show the results of our early work where initially we got points and then lines working. I was very pleased at this stage as it adds a feature that had been missing for quite some time to the Avogadro framework.

I then got filled triangles added with transparency support, allowing us to visualise the underlying structure of the molecule and map other parameters onto the surface. Tim then added electrostatic charge mapping to the colour of the surface which I hope you agree is already looking very good.

There is still quite a bit left to do. I need to move the class into libavogadro and out of the engines, integrate it into our Painter API, implement caching and am thinking of using the Qt Concurrent framework to do intelligent multithreading once Qt 4.4 is released as surface generation is quite slow for big molecules. I also need to link the mesh size to the global quality level and see what other optimisations might be possible. Another big one is adding support for visualising molecular orbitals. Geoff has already done a lot of work on the back end in OpenBabel for this and so hopefully it will not take me long to get it added in.

Lots to do but some great progress already. I would love to know what you think. Once we have been able to polish this new feature a little I hope we can get a release out of the door so that more people can test out these new features and let us know what they think, point out bugs and shower us with compliments Until then I hope you enjoy the screenshots!

Note: For some reason the exported graphics shown in this post were not antialiased. They are on screen and I need to look into why they are not when I export graphics in this way.

I am sure you have heard by now about the KDE 4.0.0 release made yesterday. Currently my desktop machine is on a boat in the middle of the Atlantic somewhere, my laptop is broken and I only have access to Apple systems... I was able to check out KDE 4 and test it in a VM and it is looking good.

I haven't been able to help with Gentoo packaging of KDE 4 very much at all which is a shame. I should get my desktop back at the end of the month in my new home in Pittsburgh. I also couldn't attend the release event as I am attending my graduation ceremony on the 17th of January. Hope you guys really enjoy the event and if I could have moved things around to attend I certainly would have.

There is lots of stuff I hope to be able to do for KDE 4.1, and more specifically Kalzium, once I get my desktop back at the end of the month. Avogadro already has lots of improvements including a KGraphicsView based periodic table which I think Carsten is interested in me getting into Kalzium 4.1.

Enjoy the release event and remember to raise a glass to those of us unable to make it for various reasons

I haven't been keeping up with XKCD since my laptop stopped working - never set up my RSS feeds for the few comics I used to regularly read. I thought I would catch up and this one really made me laugh - I guess geek humour is alive and well!

Along with the one linked to from slashdot about the Googlebomb caused by the died in a blogging accident which I may have just added to now I guess.

I thought I would join in, a little late, with the posts about 2007. I think it is good to reflect at times. Please feel free to skip this whole post if you were hoping for something more focussed. For me 2007 has really been a roller coaster of a year filled with highs and lows. If I were ever to write my memoirs this one would probably go down as the most surreal so far.

I just made a few commits using my trusty git svn dcommit (I am still really loving git and it looks like the bug I reported may be fixed in it now too). Then I saw the commit numbers - we just hit commit 1000. Development has really been accelerating recently and I haven't had time to write about as much as I wish I could.

Hopefully I will get a chance to take some screenshots and show off some of the latest stuff. I have coded a QGraphicsView based periodic table I am pretty happy with. It is compact, allows elements to be chosen, shows our colour scheme and even displays a little extra information on the selected element. Then there is the ribbon engine which is coming along but needs some loving attention from some bio people.

I am very pleased with the new quick rendering stuff we have added. It still has its issues. I had fixed some but some of Benoit's recent commits seem to have reintroduced the cache problem upon engine changes. Tim has added a new hydrogen bond engine which looks great thanks to some stipple line stuff from Geoff. There is some great stuff in the properties dialogs too along with lots of improvements to our force fields.

There is certainly lots more for us to do. The biggest thing I still haven't gotten in yet is surface support! I really want to get that sorted as I think it will be both useful and should look great. Along with some extra dialogs for graphics export, POV-Ray rendering etc.

At the start of October I began my first postdoctoral position, at the University of Pittsburgh in the Chemistry Department working in the newly formed Hutchison Group for Geoffrey Hutchison. Life has been so hectic these last few months finishing my doctorate and preparing for my biggest move ever.

Louise and I arrived on the 27th of September and had less than a week to find a suitable apartment out here ready for me to start work. I am now a legal non-resident alien (I think - someone correct me if I got it wrong). I am very excited to be starting this new job, the move has been really tough but the research looks very interesting and I am sure that my time here will be very productive.

I have already been to quite a few interesting talks in the department and am making the transition from physics to chemistry. It is OK though as I get to do lots of physics and programming along with some more chemistry. Exciting times and I am sure I will talk more about my work in future. I have already posted a tutorial article on installing Sun Grid Engine on a Mac OS X cluster.

Dax made it out here and we are both enjoying the recent snow. This entry sat in drafts for about a month or so as I have been so busy sorting things out, coding, reading and going to physio for my high ankle sprain which I am told is both unusual and the worst kind of sprain...

It seems I have read quite a few posts in the last half a year or so on the virtues on decentralised version control and how good git it. Before my laptop died I had installed git and was using it for development. Unfortunately that was pretty short lived due to the failure of my laptop. Now all of my development is taking place on Apple Mac machines using Leopard (I used Tiger for a while too).

Recently I built git from source on the laptop I am using and checked out the full Avogadro subversion repository using the following commands.

git svn init -t tags -b branches -T trunk https://avogadro.svn.sourceforge.net/svnroot/avogadro
git svn fetch

Then I went and grabbed a coffee while it imported over 800 commits which is pretty small compared to many of the other repositories I have heard about being imported. It didn't take too long, but I would happily make my git repository available should anyone want a copy.

I have found git to be very fast which is absolutely great. There are some things which I have found a little confusing while getting to grips with git and its interaction with the subversion repository.

For my day to day work I need very few commands. git svn rebase synchronises with the subversion repository. I really like this way of updating too as it displays the diffs for the commits and the log messages. git commit -a commits all local changes to the local repository. git status shows the current status of the repository, git show shows local changes already committed and git svn dcommit pushes my local changes to the subversion repository.

It is this last feature I think I like the most. I can queue up multiple change sets which might actually break Avogadro, giving nice bite size commits as I make progress, but only committing them once I am done and everything is working (at least reasonably well). Then there is the potential for doing this when off line which is also great.

One of the most confusing things I found was how to nuke local changes if I didn't want to keep them. Deleting the file and updating doesn't work as it does with subversion. Also revert didn't do what I was after. I finally found git checkout which was not what I was expecting. Issuing the command with no arguments resets the repository to the last committed state. Adding filenames as arguments reverts just those files.

Another one I haven't found the answer to is files reappearing that we long since deleted. The avogadro.pro file is the main one left but a set of directories also reappeared. They are not there when checking out using just subversion. If anyone has the answer I would love to know. If I git rm the file the commit to subversion does nothing and it loops round.

Other than that I am really happy with git. It can be a little too easy to use the wrong commands and wipe out changes so I am moving quite gingerly at times but I think it is well worth the trade off. A few guides such as this one, this one and of course this have helped me along the way.

UPDATE: Just to note that I was using the git 1.5.3 release compiled from source.

I recently added a couple of new engines to Avogadro and the screen shot below shows them both. The ring engine finds all the rings of the molecule and draws a transparent plane through the ring. The polygon engine finds atomic centres with three or more atoms bonded and draws a polygon around that atom. This is only done for atoms that are not the common organic types.

I think these are useful engines that have been prompted by user requests. May be not as exciting as the ribbon engine or the surface support we are working on but useful to display certain structures. The ring engine has not been without its problems though as you can probably spot in the image below showing the ring engine rendering a carbon nanotube.

The lighting just flips at certain points - the same happens as you rotate molecules that are using the ring or polygon engine. I guess the lighting is flicking on at certain angles. I would be interested if anyone with more OpenGL knowledge than I might know how I can improve the rendering. Some rings are near white and some of the back rings don't seem to be drawn - that could actually be drawing order though which is always tough to get right.

Comments and tips are always welcome. The big challenge right now is getting surface support added in. This is something I think we really need and I would use in my daily work as I am sure many others would. I have been reading up on marching cubes and stuff but would welcome tips in that area too!

On Wednesday Dax, my overgrown German Shepherd dog, took his first ever flight. Originally they were supposed to fly him into Pittsburgh International Airport but apparently there were no carriers that could accommodate such an overgrown dog!

So eventually they gave up and told us that they couldn't get him to Pittsburgh as promised. I think it was Thursday night last week I had a voice mail left from what sounded like Dallas International Airport asking me to call them back and confirm that I would take delivery of Dax. I nearly had kittens of my own as I saw how far away this was!

It turned out that there is a Dulles International Airport, and the way it is pronounced sounds very much like Dallas. That was a much more manageable four hour drive away near Washington DC. So I hired a large pickup truck for Wednesday of this week.

Over the last couple of days I have made a few commits to improve the ribbon support in Avogadro. Many thanks to Geoff for fixing some bugs we found in Open Babel and to Thomas Margraf for his helpful suggestions and for letting me take a look at a little of his code that renders ribbons.

Right now the ribbon is drawn as a tube between the carbon atoms of the backbone and so isn't really a ribbon at all. Then I looked at the image and it is looking quite a bit like a ribbon. The code is checked in and so feel free to take a look at it. I am using NURBS but am far from an expert on their use. My laptop failure has lost some of the tuning I had done too - this is my first screenshot from Apple Mac OS X Leopard!

Points if you can tell me why the alpha helix looks pretty good despite me having not yet implemented anything to find the correct plane! Still needs some tuning but I hope you will agree that this is looking pretty nice now. It has also led to some optimisation of molecule loading but I am sure that there are lots of biologists who will tell me it still isn't quite right.

Update: Geoff has now shown me how to do screen capture and so here is a pretty video of Avogadro rendering ribbons in action!

Since moving out to Pittsburgh I have been using my Acer Ferrari laptop at home for development, keeping in touch with people back home and trying to figure out some of the weirder rules out here such as driving licenses in this state. Last night I booted up when I got home and after logging in everything went a little funky - got I/O errors when typing ls for example.

I tried rebooting but it wouldn't even get to the GRUB prompt. So I got out an old Gentoo LiveCD which I had luckily left in my laptop bag. That got so far then failed to mount the drive. After a few reboots and a little wine (to calm my nerves ) I managed to access the drive. I thought I would try copying some stuff across to my external hard drive and ended up getting more I/O errors and some memory errors for good measure. Half the time when I booted from the LiveCD the kernel panicked when attempting to mount the hard drive partitions.

So today I am a very unhappy English man in Pittsburgh. Due to me having spent far more than I budgeted for moving out here, and then more again on flying my dog out, I don't foresee being able to replace it any time soon. This means my development activity will be limited. Not sure if compiling part of a KDE 4 checkout might have been what finished it off. Looks like some kind of motherboard issue. The trackpad stopped working months ago, now I am getting random memory and I/O errors and the CPU has been running hotter and hotter recently. It served me for just over two years which isn't great but I worked it pretty hard and it travelled a fair part of the globe with me too.

Still can't help being cheered up a little as I will be picking Dax, my overgrown German Shepherd dog up from Dulles Washington International Airport tomorrow (not Dallas as it sounded like on the message they left me which made me very unhappy when I first heard).

Update: My very nice and generous new boss has very kindly offered to let me use the group laptop in the interim. It is a MacBook Pro though using something called Mac OS X Leopard. I can keep hacking on Avogadro with it and I even know how to take screen shots with it now too! It is also capable of checking email and browsing the web so my Internet connection in the apartment won't go to waste. I am considerably happier now but will miss my KDE 4 and Linux fix

So I had my first experience of Halloween in the US tonight. As I went grocery shopping I saw all the shops with big bowls full of sweets handing them out to passing children. Dozens of children dressed up as various ghoulish characters wandering around with their parents in toe, helping them to carry their swag.

I walked back to my apartment to witness yet more children wandering the streets with their parents and people up and down my street handing out sweets (or candy as it is known over here). I got home to meet a few of my neighbours work colleagues handing out treats on our doorstep. Lots of the houses around here have been decorated for weeks and it amazes me what a big thing Halloweeen is.

I am really going to miss Guy Fawkes Night on the 5th of November - don't think anyone knows about it over here We always had great parties and gatherings on Bonfire night, although currently my social calendar is emptier than it has ever been due to me being very new in town. Hope you have all enjoyed your Halloween and have a great Bonfire night if you celebrate it!

Today we released Avogadro 0.2 which features many improvements since the 0.1 release. It has lots of the new features I worked on during my Google Summer of Code project along with some great work by other contributors such as the bond centric manipulation tool coded by Shahzad Ali, Ross Braithwaite and James Bunt.

Geoffrey Hutchison announced the release earlier today, on what I have been informed is mole day due to the date format used in the US - 10/23 corresponding to the 10^23 of Avogadro's number. So today seemed like an even more fortuitous time to make a release of Avogadro!

Today also marked my first commit to the Gentoo repository in quite some time to add this release to Gentoo's ebuild repository. If you have been waiting for me to add/update packages and I haven't gotten around to it due to life being so busy now would be a good time to poke me. I will also hopefully be completing the ribbon support I have already talked about and that should make it into a release soon but wasn't ready for this release.

We would love to receive feedback on the latest release of Avogadro. These new features will also make their way into Kalzium in KDE 4.1.

I term it as "ribbon" because it really isn't a ribbon. I haven't figured out how to calculate the plane the ribbon lies on and right now it is a series of cylinders drawn from point to point. I hope you will agree that it already looks fairly effective and can certainly show the nature of the secondary structure of biological molecules.

There is still a lot to do such as draw the tube in a more efficient manner as a continuous OpenGL construct and draw each residue separately. It would also be good to be able to draw ribbons but this may take a deeper understanding of ribbons and how they are visualised than I have. If there are any biologists out there who want to help me out then tips, or even better patches, would be great fully received