Marcus D. Hanwell's Blog

Avogadro, OpenGL and Display Bugs

marcus@cryos.net (Marcus D. Hanwell) — 2008-06-23T21:53:26Z

Before last summer I had never really done any OpenGL programming. Now I feel like I know a reasonable amount but certainly still have a lot to learn. I have an Acer Ferrari laptop with an ATI Radeon X700 discrete graphics card in it. I was using the proprietary driver for a while but it was so unstable it just wasn't worth using. I was hopeful after reading about AMD opening up the specs so that good 3D support can be added to the open source driver.

I have to say on the whole the 3D support has been getting better and better for the r300 chipset. I cannot enable the desktop effects in KDE 4.1 trunk without losing my OpenGL apps though. As you can see in the above screen shots filled surfaces also do not work. For some reason the colouring of the surface is incorrect, i.e. the colour is never changed. I am running the latest git sources of xorg, the ati drivers, x11-drm, mesa etc, thanks to Donald Berkholz. Avogadro seems hit File r300_render.c function r300Fallback line 360, Software fallback:ctx->RenderMode != GL_RENDER every time too.

There is an open bug report that appears to be describing the same situation. I do wonder if it is possible that our OpenGL code could be improved to avoid this bug. I would love any tips on ensuring our OpenGL code is as compatible as possible. The surface rendering works correctly on all other platforms (Linux, Mac, Windows) and with other drivers as far as I know.

We also have an open bug report where Avogadro segfaults on Linux systems using the savage driver. This seems to be a consistent problem. I am afraid I do not have the hardware to test further, I have added further debug output to our initialisation routines - Avogadro crashes very early on. Again, any ideas on how I might fix this bug or at least exit gracefully would be great. Backtraces may help to at least see what functions are called before the crash but this might just be a driver bug.

New Webcam and Linux

marcus@cryos.net (Marcus D. Hanwell) — 2008-06-16T22:29:08Z

As my regular readers will know I moved over to Pittsburgh, PA in the USA at the end of September last year. Before I left I got some of mine and Louise's family webcams so that we could stay in touch. Then it took ages to sell our house and the shipping nightmare began (which I will try and bring myself to talk about one of these days). Eventually I actually got my desktop computer back (a little worse for wear after the shippers "carefully packaged it").

Going back to my original point, I had been putting off buying a webcam as I wanted it to work well in Linux, and every time I looked into which webcams might be my best bet it seemed that even individual models had multiple chipsets, which may or may not be supported. On Saturday I bit the bullet as Louise had been asking about getting Skype working.

I chose the Logitech Quickcam Pro 9000 (photo above) as it seemed to be part of the new USB video class standard I had been reading about and saw there was a healthy Linux UVC project already. I still anticipated having trouble getting it working on Gentoo but thought I should be able to get it working eventually.

Imagine my surprise when I just typed emerge -av media-video/linux-uvc media-video/luvcview, modprobed the new kernel module, ran luvcview and I could see me staring at the camera! Sometimes I worry this Linux thing might be getting a little too easy Another five minutes and I had the microphone working (just usb audio) and was testing it out in Skype. On Sunday we had our first intercontinental video chat with my Mum and two nephews. Skype also integrates into my KDE 4 desktop without any trouble.

I am pretty new to the webcam thing and had resisted it for a while but it was great to be able to chat and see family and friends back home. I would rather use an open source, cross platform video conferencing application but have yet to find a viable one. Open Wengo looks like it might be that application one day but I couldn't get it to work reliably last time I tried (although I did like the look of it). I also don't seem to be able to find a nice application that will let me capture video messages easily.

I was very pleasantly surprised by my experience and am very happy with the performance of the Linux drivers for the webcam as well as the camera itself. Any Linux webcam tips would of course be gratefully received!

UPDATE: Forgot to mention this great blog post I spotted that helped me decide this webcam was probably a good bet...

All American Weekend

marcus@cryos.net (Marcus D. Hanwell) — 2008-05-31T08:37:55Z

What a hectic week. I never got chance to record our first all American weekend on the long weekend. We started off on Saturday morning shopping in the sales and found a great BBQ at Wal-Mart. It is nearly twice the size of the one we had back in the UK with a great labour saving ash collector at the bottom. We then loaded up on meat and other BBQ essentials.

Had lots of friends over on Saturday evening and then exposed our American friends to the Eurovision song contest! It was very hard to explain this phenomenon to them but it was as amusing as ever with expert commentary by Terry Wogan

Then on Sunday we went along to our first ever baseball game and saw the Pittsburgh Pirates beat the Chicago Cubs. It was a really fun day out, beautiful sunny weekend and then we had another BBQ when we got back. I have always loved BBQs and our new one works great.

This weekend we are heading out on a road trip with a few friends from work. I haven't really taken any photos for about a year now so I am really hoping we can see a little of the country around us and get some photos I can show off. I would love more time for hacking on code too, may be Sunday evening when we get back. I have been seriously neglecting my new game too - GTA IV.

Avogadro and OpenBabel Development

marcus@cryos.net (Marcus D. Hanwell) — 2008-05-29T09:48:01Z

I thought I should post a few thoughts on the development of Avogadro and OpenBabel as there seems to be some confusion in replies to some of my posts. Ever since Avogadro began development it has used OpenBabel as its chemical library. OpenBabel provides Avogadro's internal representation of molecules, atoms, bonds, cube data etc.

Avogadro has driven the development of some new features in OpenBabel. In return we have been able to leverage many of the existing library functions to rapidly implement new features in Avogadro. OpenBabel isn't just a library for loading and saving chemical file formats. It provides information on the atoms, bond typing, force fields such as UFF to do geometry optimisation and lots of other very useful features. Many of the recent releases of Avogadro (which is still in beta) have been accompanied by a new beta release of OpenBabel due to each project driving the development of the other. I now work on both (but still mainly on Avogadro) as there were bugs I needed to fix in OpenBabel and Geoff got sick of me sending him patches!

As Kalzium uses Avogadro to do all of its molecular editing and visualisation this also means that the optional molecular editor depends on new releases of the OpenBabel library. I hope this makes it clearer why Avogadro and Kalzium generally rely upon the latest version of OpenBabel - many times we have implemented new features in OpenBabel for Avogadro but they should also be useful outside of Avogadro. As always I am open to suggestions and feedback on the way we do our development but I would love to hear more feedback on the software I am writing rather than the packaging issues. As a Gentoo developer I do appreciate that packaging is extremely important and so am doing what I can to make packaging as easy as possible too.

The Kalzium Molecular Editor

marcus@cryos.net (Marcus D. Hanwell) — 2008-05-26T17:28:23Z

Some of you with long memories may recall that I worked on the Kalzium molecular editor for my Google Summer of Code (TM) project last summer. The observant among you may have also noticed that Kalzium in KDE 4.0 featured a molecular viewer. Well the freeze meant that much of my work didn't make it into KDE 4.0. This is the case with many of our current GSoC students (Naomi among them) who are just starting to code as KDE 4.1 is being frozen and released.

Since last summer life has been really hectic, the shipment of my household possessions from the UK to the US was delayed significantly by an incompetent shipping company which also proceeded to break many things... This led to me having no Linux system to develop on for quite some time, and then less time to develop due to having to deal with the fallout of a botched delivery. Still, I am very pleased to be able to show off some of my latest work which made it into KDE 4.1 and will be in KDE 4.1 beta1 (due out in a few days).

I was able to expose some of the extra display types I added along with support for drawing and editing molecules. You can also measure distances and angles of the molecule you are looking at and do some geometry optimisation. Given a little more time I would have liked to expose some of the manipulation features but this new six month cycle was so short and my time was constrained anyway. It will make it into 4.2 at least.

I am looking forward to seeing the animation support Naomi is working on now and am very happy to be able to mentor her over the summer. I think high quality, open source educatonal programs are very important and hope that this editor will be useful in that role. The Avogadro library and application should hopefully be able to satisfy college students and researchers. With the scripting support and open source nature of the two projects I think they interact very well and complement one another.

As always I would love to hear your feedback on the Kalzium changes and the other work we are doing.

Avogadro 0.8.0 Released

marcus@cryos.net (Marcus D. Hanwell) — 2008-05-23T09:29:44Z

On Monday we officially released Avogadro 0.8.0. I think this is a great release with many improvements over 0.6.1. We have been working really hard on improving the stability of Avogadro and I hope this shows in this release. There have also been some very nice improvements in speed and lots of new features.

The screen shot above shows an imported Gaussian formatted checkpoint file displaying the HOMO molecular orbital. The import of output from quantum codes and subsequent calculation of MOs is at a very early stage but you should take a look at it in the extensions menu if you are interested. It currently only calculates s, p and d orbitals and can only parse formatted checkpoint files from Gaussian. The goal is to add at least f orbitals and many more quantum codes.

I don't think I ever talked about the new periodic table QGraphicsView that I am very proud of. It allows you to select elements that are not listed in the combobox, shows the colour they will have on screen and is about as compact as I can make it. I have already packaged Avogadro 0.8.0 for Gentoo, we have binaries (Windows, Mac) and source available for download and should hopefully have packages available for Debian and Ubuntu very soon.

There were a few features that didn't make it in and a few bug fixes we found just after release. Once we have ironed out a few more bugs and received more feedback we will make a bug fix release. I was amazed to see that Avogadro is currently shooting up the Naomi Fox's application for a Google Summer of Code (TM) was successful. Some of you may remember that I participated last year as a student and was also lucky enough to attend aKademy. This year I am no longer a student but I am very pleased to be able to participate as a mentor and hopefully pass on some of what I learned last year.

I hope you will all join me in welcoming Naomi to our development community. As I moved to the US last year we are actually in the same timezone which will make mentoring easier. It does mean getting to aKademy will be a lot more difficult but may be I can attend the first KDE Americas event. I am really looking forward to working with Naomi over the summer and I think this is a really important set of features still missing in Kalzium.

Looking forward to a great summer, my first in Pittsburgh and my first time as a mentor.

Chemistry Visualisation and Tools Meeting

marcus@cryos.net (Marcus D. Hanwell) — 2008-03-20T18:20:00Z

Last week I was privileged to be invited to speak at a meeting about molecular modelling with a focus on tools, GUIs and visualisation. The meeting was held at the Holiday Inn in Runcorn and the Daresbury Laboratory (England). I wasn't expecting to be back in England quite so soon, having only just returned to Pittsburgh at the end of January.

The meeting was a great opportunity to present some of the latest work I and others in the Avogadro and OpenBabel communities have been doing to create tools that enable the building of molecules and structures, as well as their visualisation. It was also a great chance to hear some very interesting talks by the developers of other building tools and some quantum codes. Donald and I were also invited to Daresbury Laboratory to work with some of the CCP1GUI developers.

I presented my talk on Avogadro on Wednesday morning and have made the slides available here. Donald gave an introduction to Avogadro, some of the history and the architecture before I gave my presentation. We finished by taking questions while I demonstrated the Avogadro application. I think it was extremely productive. We had many more conversations over dinner and drinks later as well as in a workshop setting on Thursday afternoon.

It was great to be able to put a face to a few of the names and discuss current issues more informally in the evening. The talks were all of a very high quality and from a varied list of speakers from other open source projects, some of the free quantum codes as well as commercial products. I have come away from the meeting with a much better appreciation of the needs in the community and I feel that Avogadro is in a great position to fill the apparent void.

I am glad that we were able to get surface and orbital support working in Avogadro before the meeting. Right now we only support Guassian cube files but the implementation is general enough that I will be able to add support for further formats. I really think that if we can get enough people collaborating on a common project everyone can get the tool they need to effectively do their research at a much lower investment than could be achieved by working on many separate projects.

I met Tristan Youngs, the developer of Aten, who had implemented some really nice features in his molecular builder that is much more focussed on molecular mechanics. It is well worth checking out. As is Zeobuilder which was developed by Toon Verstraelen. They both implement some great features and have strengths in different areas. Of course my dream is to integrate many of these features via Avogadro plugins and have one editor which is capable of being used in a diverse range of applications.

It was also great to speak to Mario Valle who is doing some very interesting work in the area of new visualisation methods and supports a large user base of computational chemistry users. There were of course so many other talks but you can look at the schedule yourself and I think the slides of all the talks should be available in the near future.

I feel sure that many good things will come out of this meeting and hope to be able to attend similar meetings in the future. I would like to thank Jens once again for hosting the meeting and taking care of everything. I hope to see some patches and/or commits from him in the near future

Getting Back to Gentoo and KDE Development

marcus@cryos.net (Marcus D. Hanwell) — 2008-03-09T14:58:00Z

After I got back from the UK in January my wife came to join me. She brought with her my old Acer Ferrari nearly working shy of a new hard drive. I got one from NewEgg at a fairly reasonable price and installed it. The drive arrived with a few bent pins, packaging isn't their strong point obviously, and took twice as long as they said but hey at least I had it. I had been stranded in America without Linux for more than three months and so another few days wasn't going to kill me.

Once I got the drive in it seemed the slot loading CD drive and got itself in a twist and would not load any CDs. An hour later and lots of pieces on the table I had reset it, put it back together (a few times) and got a Gentoo LiveCD in there. So I was in business. I had considered installing kubuntu but I wasn't getting on with it in the VM I had at work and I wanted to get back to Gentoo development too.

Everything looked good. Got an X server up and running. The ATI binary blob was as unstable as ever. Thanks to nerdboy (I think) I got the open source r300 driver working. This is much more stable although missing several OpenGL features we use in Avogadro such as smooth triangle shading, changing vertex colours in drawing operations etc. Other than that it works much better than the binary blob and is far more stable. I like stability in a system.

I have KDE 4 trunk running and am using it as my main desktop right now. Still got some rough edges I haven't had time to figure out just yet such as kmail refusing to save my IMAP accounts, kopete refusing to connect to GTalk and general saving issues most of the core seems pretty solid. I am compiling Thunderbird right now so that I have something. I will hopefully be getting back to Kalzium development pretty soon. If I find the time I would love to help out with KDE in a more general sense as well as helping out with the Gentoo packages.

My desktop is apparently in a customs area somewhere in New York. Delay after delay after delay with the shipping of my household possessions from the UK. So I am still without desktop hoping that an "intensive exam" is not too expensive and does not involve breaking any of my stuff :/ We shall see. Fingers crossed I will actually have furniture and my desktop computer back within a few weeks. That would allow me to be so much more productive having my dual core Gentoo desktop back, creature comforts and all that good stuff.

So keep your fingers crossed for me. Hopefully you will be seeing a lot more commits coming from me in the KDE and Gentoo repositories as well as the Avogadro work I have managed to continue to do on the MacBook Geoff so kindly loaned me whilst I was laptopless (is that a word? It should be!) Not sure that was a sentence now either...

Avogadro 0.6.1 Released

marcus@cryos.net (Marcus D. Hanwell) — 2008-03-09T14:30:00Z

I am pleased to announce that I tagged and released Avogadro 0.6.1 yesterday evening. This is a bug fix release which fixes one pretty large bug that slipped through - the OpenGL context was lost if switching between virtual desktops, multiple views etc rendering the OpenGL window useless unless the application was restarted. As such I would encourage anyone running Avogadro 0.6.0 to upgrade to this new version. It also features several smaller bug fixes and feature enhancements.

The screen shot above shows Avogadro 0.6.1 running in a KDE 4 session. One of the small visual tweaks I made was to add a second light source to our default OpenGL scene which really helps to illuminate the other side of the scene. Thanks go out to Albert for his suggestion of adding another light source. Hopefully there are no really big bugs remaining but Avogadro is still in the beta stages of its development. It is rapidly approaching a stable release though and I am very happy with our progress so far.

We would love to hear what you think of Avogadro. I had one person question why we always have to use the latest and greatest version of OpenBabel and felt I should offer some explanation. Many of the features exposed in Avogadro use functions and structures in OpenBabel. I myself was quite heavily involved in improving OpenBabel's support for Gaussian cube files and the cube format so that we could load and display orbitals for example. As such we often add new features or fix bugs in OpenBabel trunk and so a new release of OpenBabel must be used in order for everything to work.

There are already ebuilds for this latest version in the Gentoo tree. Ubuntu/Debian builds are in the process of being built. We should hopefully have Mac and Windows binaries very soon too.

I am headed to a meeting in the UK where Donald and I will be talking with other scientists about visualisation in chemistry and related areas. We will of course be showing off Avogadro as well as talking with many other people working in this area. I am very much looking forward to it and hope that this will lead to further innovation in the Avogadro project as well as the open source chemistry movement in general. It will of course be great to have a full English breakfast and some real ale too!

Avogadro 0.6 Released

marcus@cryos.net (Marcus D. Hanwell) — 2008-02-29T22:01:45Z

Earlier today we released Avogadro 0.6. This release is quite overdue and we had originally planned to make a release around the new year. This release has many, many new features that have been implemented since the last release.

I am especially proud of the new support for displaying surfaces. This initially led to the surface engine that displays a Van der Waals surface. Tim added the ability to map the electostatic surface potential onto that. I then began work on adding support for displaying orbitals. This led to me getting commit access to OpenBabel and making quite a few commits as I got Gaussian cube loading working. While I was there I couldn't resist improving the Grid classes (still a little more I would like to add in).

Another new feature I am really pleased with is the QGraphicsView based periodic table. I think it works very nicely and I am hoping to add this code to Kalzium. I think the Avogadro library is in great shape for me to begin porting Kalzium to use it. Tim and Geoff have been doing some amazing work in OpenBabel on improving the force fields used for geometry optimisations.

We have also been doing a lot of work to improve Avogadro's performance when rendering large systems. We have added quickRender functions and OpenGL display lists to improve interactivity as well as threading calculations where this made sense. We have a cool little colour widget Geoff coded too that allows colours to be displayed and picked easily, improved default layout and one of my personal favourites - persistent settings in most of Avogadro.

There are certainly some bugs remaining. I would like to get to the bottom of as many as I can and hope to make one or two bug fix releases in the 0.6 branch. We would love to get feedback from people. We have already made a source release and a Mac binary. I will be adding Gentoo ebuilds shortly and we will hopefully get a Windows binary out by early next week.

Hope you enjoy this release. I think we are getting close to a stable API that we can call 1.0 and have most of the core features I was hoping to get into Avogadro. Our framework is extremely modular and extensible and I hope that we will be able to build up a community around Avogadro. It was also great to be able to make this release on 29 February - we have so few of them

Avogadro: New Orbital Support and Gaussian Cube Format

marcus@cryos.net (Marcus D. Hanwell) — 2008-02-19T21:00:00Z

So over the weekend I spent quite a lot of time hacking away at OpenBabel working on the Gaussian cube format support I needed to get working in order to be able to visualise electronic orbitals. The initial support was taken from Molekel but contained no error checking and it was not reading in the example cube file I had.

Thankfully a colleague pointed out a page with details on the Gaussian cube format and I used this as a basis to get it working in OpenBabel with better error checking and a more resilient tokenisation of the cube points. I made my first few commits to OpenBabel over the weekend and got at least simple Gaussian cube files loading that only contain one cube. More will follow I am sure.

After I got the file loading sorted a few more changes to Avogadro got me my first ever orbitals in Avogadro - the benzene HOMO! I believe the cube file came from the JMol test files. Geoff informed me that chemists for some reason see positive as blue and negative as red. That seemed very strange to me and I had initially put it the other way around. Growing up playing with electronics it seemed to me that positive should be red just as the wires in a circuit have a red positive... Other than that orbitals seemed to be working well and I was very pleased.

Next I started to load up some other cube files I had been given as examples. I thought the above image of the electrostatic potential of CH₃Cl looked very nice. When I showed Geoff he also liked it and we wondered if it had been visualised in this way before. I know normally the cube would be mapped onto the Van der Waals surface of a molecule.

Anyway I was pleased and also felt more productive as I now have my laptop back up and running and am developing on Gentoo again. It would be great to hear what other people think of this new support. I am already working on various improvements to the code and getting ready for another release of Avogadro!

Avogadro: Surface Support

marcus@cryos.net (Marcus D. Hanwell) — 2008-02-11T11:50:00Z

I am pleased to announce that we now have working surface support in Avogadro. We originally began working on support for surfaces around November time when Geoff improved the support in OpenBabel for grids and added some initial code to the Avogadro repository. Quite quickly it became evident that this algorithm did not deal well with data sets containing discrete objects that you wanted to polygonise.

I began a search of the available algorithms and came across a very nice guide to the marching cube algorithm with example code. I had a brief look for a C++ implementation and didn't find anything initially, began implementing what was on his page and then came across Zhu3d which is a Qt 4 based program that actually uses an implementation of the algorithms described in the guide and references them. As it is GPL licenced code I imported the relevant class into our repository and began the task of adapting it to fit into our framework.

With some code from Geoff, more pointers from Geoff, some reading and quite a bit of help from Tim (not sure if he has a web page/blog) we got it working. Since then Tim has actually fixed a big bug in OpenBabel that was causing crashes and done more work on the IsoGen class. The screenshots above show the results of our early work where initially we got points and then lines working. I was very pleased at this stage as it adds a feature that had been missing for quite some time to the Avogadro framework.

I then got filled triangles added with transparency support, allowing us to visualise the underlying structure of the molecule and map other parameters onto the surface. Tim then added electrostatic charge mapping to the colour of the surface which I hope you agree is already looking very good.

There is still quite a bit left to do. I need to move the class into libavogadro and out of the engines, integrate it into our Painter API, implement caching and am thinking of using the Qt Concurrent framework to do intelligent multithreading once Qt 4.4 is released as surface generation is quite slow for big molecules. I also need to link the mesh size to the global quality level and see what other optimisations might be possible. Another big one is adding support for visualising molecular orbitals. Geoff has already done a lot of work on the back end in OpenBabel for this and so hopefully it will not take me long to get it added in.

Lots to do but some great progress already. I would love to know what you think. Once we have been able to polish this new feature a little I hope we can get a release out of the door so that more people can test out these new features and let us know what they think, point out bugs and shower us with compliments Until then I hope you enjoy the screenshots!

Note: For some reason the exported graphics shown in this post were not antialiased. They are on screen and I need to look into why they are not when I export graphics in this way.

KDE 4 Release and Party

marcus@cryos.net (Marcus D. Hanwell) — 2008-01-12T11:42:05Z

I am sure you have heard by now about the KDE 4.0.0 release made yesterday. Currently my desktop machine is on a boat in the middle of the Atlantic somewhere, my laptop is broken and I only have access to Apple systems... I was able to check out KDE 4 and test it in a VM and it is looking good.

I haven't been able to help with Gentoo packaging of KDE 4 very much at all which is a shame. I should get my desktop back at the end of the month in my new home in Pittsburgh. I also couldn't attend the release event as I am attending my graduation ceremony on the 17th of January. Hope you guys really enjoy the event and if I could have moved things around to attend I certainly would have.

There is lots of stuff I hope to be able to do for KDE 4.1, and more specifically Kalzium, once I get my desktop back at the end of the month. Avogadro already has lots of improvements including a KGraphicsView based periodic table which I think Carsten is interested in me getting into Kalzium 4.1.

Enjoy the release event and remember to raise a glass to those of us unable to make it for various reasons

XKCD: Responsible Behaviour

marcus@cryos.net (Marcus D. Hanwell) — 2008-01-12T11:00:26Z

I haven't been keeping up with XKCD since my laptop stopped working - never set up my RSS feeds for the few comics I used to regularly read. I thought I would catch up and this one really made me laugh - I guess geek humour is alive and well!

Along with the one linked to from slashdot about the Googlebomb caused by the died in a blogging accident which I may have just added to now I guess.