Marcus D. Hanwell's Blog - FOSS

VTK: New 2D API, Canvas and Charting Features

marcus@cryos.net (Marcus D. Hanwell) — Thu, 24 Dec 2009 14:13:00 -0500

Since joining Kitware in October, one of the first projects I was tasked with is revamping the 2D charting capabilities in VTK and ParaView. At first I was a little daunted as it meant digging through many of the internals of VTK, and breaking an assumption that is made in many parts of VTK - that everything being rendered is 3D.

A large portion of this work is also being driven by the InfoVis features in VTK, along with project Titan that we work on with some really interesting people from Sandia National Labs. The project grew quite a bit from its original scope, and I have now added some new 2D API that uses OpenGL as a backend, with the scope to add further backends in the future. I have been working on optimizing the OpenGL case so that large data sets can be rendered interactively, and small data sets can be rendered with minimal lines of code whilst giving pleasing visual results.

Then when considering user interaction with these 2D elements we decided that a higher level API would be useful, that could contain objects and propagate mouse events to items in the scene. So I set about prototyping a new canvas based API. At this point I had enough new infrastructure that I felt it was about time I got back to my original task of implementing some efficient, well rendered 2D charts in VTK. Once I had my initial prototype in place it was time to expose this in ParaView and see how everything fitted together. As you can see in the screenshot above, things are shaping up very nicely. The new chart is in the bottom right widget, the chart above is the existing chart widget.

I have really enjoyed my first few months at Kitware, and have found my first project both challenging and rewarding. It is great to be working on real problems that have a broader impact, and as I flesh out these features I will try to maintain cross platform, high performance interactive charts. I think I have also added some useful new 2D focused API that can also be rendered over the top of VTK's existing 3D visualizations, opening the door to some very exciting new views on data.

As a physicist I also feel it is interesting the symmetry - Qt adds 3D to a 2D toolkit, and at the same time I am adding 2D to a 3D toolkit. Hope you all have a Merry Christmas, and a Happy New Year. I will be tracking Santa with my son this evening!

Disclaimer: The opinions and musings in this post are mine, and not those of my employer. Any mistakes/inaccuracies are also mine, that said I would love to hear what people think of this new work.

Avogadro 1.0.0 Released!

marcus@cryos.net (Marcus D. Hanwell) — Fri, 23 Oct 2009 18:02:00 -0400

It is with great pleasure that I announce the release of Avogadro 1.0.0. After many years of work we have released what we consider to be a stable Avogadro release on Mole Day, which seems appropriate given the projects's name. There are still some rough edges, but I think this is a good release. With your help we can fix bugs in the release while working on new features in trunk.

Avogadro - Code Swarm from Marcus Hanwell on Vimeo.

What better time to look back to the beginnings of Avogadro. There was a blog post made today by Sourceforge about Avogadro detailing a little of that history. I have also made a code_swarm movie visualizing the history of the Avogadro project. There have been quite some changes in that time both at a project level and a personal level.

I would like to thank Google for sponsoring me for a GSoC project in the summer of 2007. Also Geoff Hutchison for giving me the opportunity to work with him at the University of Pittsburgh on interesting computational and visualization projects. Then there is my new employer, Kitware, who have provided me with an exciting opportunity to push scientific visualization and cross platform development to its limits.

To finish off a great day, my wife has informed me my new espresso machine has arrived! I am going to Camp KDE in January too!

Avogadro Auto Optimization Screencast

marcus@cryos.net (Marcus D. Hanwell) — Fri, 10 Jul 2009 19:44:45 -0400

Geoff showed me a new screencast he created recently. It is made using the latest Avogadro, and is one of the first screencasts with our new and improved user interface. Geoff has also added some audio commentary with notes on the chemical relevance of the auto optimization tool. Check it out and let us know what you think - a new release of Avogadro is coming soon.

I will hopefully find the time to make a few new screencasts soon too. Between my one month old son, day job and waiting on my visa application (does not take any real time - some mental drain) I have not had much spare time to code or blog. Remember that Avogadro was nominated for the SourceForge community choice awards too - click on the link below to vote for us.

Avogadro Nominated for SourceForge Community Choice Awards

marcus@cryos.net (Marcus D. Hanwell) — Tue, 23 Jun 2009 23:23:00 -0400

I am very pleased to announce that Avogadro has been nominated as a finalist in the SourceForge community choice awards this year. We are in the "Best Project for Academia" category, and I would like to encourage you to vote for Avogadro.

This is a real honour for all of us, and I appreciate all of you who nominated Avogadro. We are all pushing very hard on polishing Avogadro, getting ready for our 1.0 release. It would be absolutely amazing to see Avogadro win this award, so please vote for us.

There are also some other really nice projects in there too, such as Lancelot, ClamAV, phpMyAdmin and RepRap. So please take a few moments to place your vote, and tell your friends!

Update: You can vote even without a SourceForge account - just enter your email address and verify your vote.

Avogadro at the APS March Meeting and Q-Chem Workshop

marcus@cryos.net (Marcus D. Hanwell) — Tue, 24 Mar 2009 17:49:00 -0400

So last week was extremely busy. The APS March Meeting was held in Pittsburgh and Q-Chem held a workshop on Q-Chem at the end of the week. I presented a poster on Avogadro (shown below), met lots of interesting people and got lots of new ideas for both research and Avogadro.

As we push towards making a 1.0 release of Avogadro, getting feedback from users in the scientific community is extremely important. As Q-Chem chose to use Avogadro as the builder/visualizer in their workshop I had the opportunity to observe new Avogadro users interact with our application for the first time. I also had the opportunity to help them overcome some initial issues and gained a few new insights.

I was very pleased to meet people at all stages of their career who were very interested in having an open source application that can provide a framework for building and visualizing molecules. I also realized that two of the most sought after features in Avogadro right now are the capability to easily make movies, and a z-matrix editor. People loved the ray-traced images of surfaces, coincidentally I received a request from someone in the press wanting to use an image I put up on my blog last year of ray-traced benzene molecules.

I look forward to hearing from some of the new users we gained in the last week. It is great to see Avogadro receiving more attention. I have started to work on the z-matrix editor and spent the weekend experimenting with movies - more to come soon!

Manifest Hell...The New DLL Hell?

marcus@cryos.net (Marcus D. Hanwell) — Sat, 07 Feb 2009 15:30:00 -0500

I am going to shock you all and admit that I am not a Windows programmer! I do however know quite a bit about cross-platform development and have now learnt the joy that is the manifest in Windows development. It seems that DLL hell is now a thing of the past, all hail manifest hell. A week ago I was not aware of these wonderful little files and this post is an attempt to document what I learned while packaging Avogadro for Windows.

After listening to Bill's talk at Camp KDE I was convinced that it really was a good idea to use CPack to package Avogadro. So when I got back home I spent a day or two getting a Windows development environment set up for Avogadro. We have several dependencies such as Qt and this post documenting a bug in the Visual Studio 9 service pack. So I edited the manifest and lied about the version of the DLLs - it believed me.

Next I found that none of our plugins would load. They are Qt plugins that implement most of the functionality in Avogadro. I found a very long thread on this subject, the crux of which is that the embedded manifests are causing Windows to look for the runtime libraries in the plugin directory. Copying them did indeed work but was not optimal. I found that by adding set(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} /MANIFEST:NO") to our CMakeLists.txt manifests were no longer made for our plugins (which are loadable modules).

So all is quite well with our Windows build now I think. If you would like to try out a new Windows installer, then please download it from here and let me know if it works for you. I have tested it on Windows 2000 and XP virtual machines. This is not the final installer, I need to add some extra data files for OpenBabel and ensure I got the other dependencies right.

If anyone with more Windows experience knows of better solutions please let me know. CPack is absolutely awesome, it seems a shame that the deployment of applications is made so difficult. I know that plugins are not as widely used and so hopefully this post will add to the collective knowledge indexed by Google.

Qt Going LGPL

marcus@cryos.net (Marcus D. Hanwell) — Wed, 14 Jan 2009 14:25:52 -0500

Just to add that I think this is absolutely amazing news - Qt is going to add LGPL licensing as an option. For the few situations I have questioned using the Qt library it has mainly been an issue of license. I think this is huge news and now cannot see any downsides to using Qt when working on C++ GUI applications and libraries.

Thanks - I can't wait to thank some of you in person at Camp KDE!

Git and Automatic ChangeLog Generation

marcus@cryos.net (Marcus D. Hanwell) — Mon, 29 Dec 2008 12:10:58 -0500

After our move to use Git and GitHub to host our repository I got thinking about ChangeLogs. Having a version controlled file, where you manually add details about what the version control system should be recording seems like it should not be necessary. I searched and couldn't find a solution that generates ChangeLogs in the style we prefer, which is a variant of the GNU ChangeLog.

So I wrote a quick Python script to try and accomplish this task. It may not be the prettiest Python code as I have never written more than five or six lines of Python before. ChangeLogs always seemed to be the biggest source of merge conflicts whenever we would work in branches, or just all be working at the same time. This is why I think it is necessary to automatically generate something like this that can be generated with source tarballs.

I called it gitlog2changelog.py and it has all of the basics down already. It may not be the most general script but works pretty well for us. I need to add some extra parsing for file creation/deletion so that we can add the + or - in front of the file names. Is there a general need for this? Are there better scripts out there that I dd not spot?

2008-12-29  Tim Vandermeersch <email@protected>

  ∗ libavogadro/src/pluginmanager.cpp: replace getenv(...) with
  QProcess::systemEnvironment()

  ∗ libavogadro/src/elementtranslate.h: Replace "A_DECL_EXPORT extern ..." with
  "A_EXPORT extern ..."

  ∗ CMakeLists.txt: use /MD compiler flag for MSVC

2008-12-28  Marcus D. Hanwell <email@protected>

  ∗ libavogadro/src/molecule.cpp, libavogadro/src/molecule.h,
  libavogadro/src/python/molecule.cpp: Lots of documentation updates,
  reorganised the functions and grouped in Doxygen tags. Some minor changes
  too, more are needed for const correctness.

  ∗ testfiles/multicubes.cube.gz: Removed from our source as it is the same
  size as all the other files put together. May be we should provide a more
  extensive sample of files in a separate distribution.

  ∗ libavogadro/src/engines/bsdyengine.cpp: Ported to use the new bond position
  functions.

  ∗ libavogadro/src/bond.cpp, libavogadro/src/bond.h: Added functions to
  retrieve bond positions, still need to implement the mid-point function.

  ∗ libavogadro/src/bond.h: Documentation updates.

  ∗ libavogadro/src/python/bond.cpp: Added missing Atom include.

  ∗ libavogadro/src/atom.cpp, libavogadro/src/atom.h: Documentation updates,
  added member function groupings and a destructor.

  ∗ libavogadro/src/atom.cpp, libavogadro/src/bond.cpp, libavogadro/src/bond.h:
  Added some atom accessor functions to the Bond class. This should make using
  bonds easier. Fixed assignment order in Atom constructor.

Avogadro Has Moved to GitHub and Git

marcus@cryos.net (Marcus D. Hanwell) — Wed, 24 Dec 2008 00:15:00 -0500

After some discussion on IRC the Avogadro Project has moved its version control over to GitHub. Our new repository can be found here. Most of us are old Subversion users, and a few of us started out with CVS. I think we are still getting our head around the workflow but feel it is worth the effort for all the advantages the move will bring.

There are some great visualisations from GitHub. I have always loved the speed of Git when compared to other version control I have used. It is also great for adding in bigger changes and not having to halt development in other areas for fear of merge issues. We shall see over the coming months how positive the move was, but I am confident it will be. I have been using Git locally through git svn for over a year now I think.

For those who might wonder, the conversion was not totally automatic. The branches and tags git-svn leaves you with are not Git branches and tags. It basically required me to manually create the branches and tags. So for our repository I ran the following,

mkdir avogadro-git
cd avogadro-git
git svn init https://avogadro.svn.sourceforge.net/svnroot/avogadro --stdlayout
git config svn.authorsfile ../avogadro/authors.txt
git svn fetch

At that point I went and grabbed a coffee, took care of the dog...

git remote add origin git@github.com:cryos/avogadro.git
git push origin master

This got us most of the way there but lacked all of our tags and branches. I found that none of the push options worked as the tags and branches needed to be made into full Git entities first.

git branch -a (show all the branches)
git branch 0.8 0.8
git branch 0.6 0.6
git branch primitive primitive
git tag -f 0.6.0 tags/0.6.0
git tag -f 0.6.1 tags/0.6.1
git push --all origin
git push --tags origin

After all that we have our tags as Git tags and our branches as Git branches. You can browse them and clone the repository. A few of us have been experimenting and everything looks good. Adding current developers as collaborators enables them to push directly to the repository. There are also some web interfaces that allow for pulling from forks.

So if all goes to plan now, there will be no more commits to our old Subversion repository. We have preserved all of our history and I made sure the author metadata was improved. Hopefully this will make our development process more streamlined. We appreciate any and all tips, this looks like a good guide to keeping a fork in sync, pushing and pulling where necessary.

Now back to coding - we want to get a new release out!

New Addition to the Family

marcus@cryos.net (Marcus D. Hanwell) — Wed, 15 Oct 2008 13:42:15 -0400

On Monday we had a new addition to the family arrive - an Asus Eee PC 1000! So far I am pretty impressed with the hardware. The Linux installation was pretty disappointing but I never really intended to keep that anyway. I played around with it on Monday as I built a chroot for the new arrival (pictured below).

I went for the 10" model as I was worried about the size of the keyboard on the smaller ones. I also liked the sound of the SSD drive and it looked like a great little unit. I ended up building a custom stage 4 Gentoo tarball for it on my desktop and installing it here. I have it running the new KDE 4.1.2 ebuilds along with a few KDE 3 apps I can't do without.

All in all it seems to be working pretty well. I only got round to installing Gentoo on it yesterday. So far I have the webcam, wired and wireless networking working. I am using WICD to control networking. I would love to see a Qt4/KDE 4 frontend that integrates better but it is an awesome little app.

This is my first post on it, the screen isn't a bad size and the battery life seems to be good. I am on the road today and my wife acted as an unwilling hand model. I still haven't managed to get the asus_laptop or asus_acpi modules to load and so am missing all those devices. It claims no such device.

I will hopefully be able to post more at some point in the future. I certainly think this is great for being on the road and Skype is working with the webcam. I did use the 2.6.27 kernel whicih made things a lot easier I suspect. I might try building a vanilla kernel soon to see if that lets me insert the asus_laptop module successfully.

For now I have quite a few hours to kill in the BA lounge at JFK!

Stephen Fry and GNU's 25th Birthday

marcus@cryos.net (Marcus D. Hanwell) — Tue, 23 Sep 2008 12:00:24 -0400

This is a little belated as the weekend was hectic but I want to wish GNU a happy 25th birthday too! Stephen Fry made a video to wish GNU a happy 25th birthday which I really enjoyed. I may be biased as an English man and a big fan...

I think he did a great job of describing open source in a very approachable way for those of us who are less geeky. You can also download a higher quality video. I miss QI, and thick-cut back bacon! Neither of which seem to be available over here. Enjoy the video and think of us English people living in a foreign land

ACS Avogadro Talk Slides and Poster

marcus@cryos.net (Marcus D. Hanwell) — Wed, 03 Sep 2008 11:39:00 -0400

I kept meaning to put the slides to my Avogadro talk and the poster I presented at the recent ACS meeting in Philadelphia. Things have been really hectic these last few weeks but here they are. The talk was presented in the chemical eduction section, on Monday 18 August, "Avogadro: An integrated approach to teach computational chemistry modeling, simulation and visualization". The slides were made using LaTeX Beamer and the talk itself was focused on the use of Avogadro when teaching computational chemistry.

I also presented a poster at Sci-Mix on Monday, 18 August, and at the main computational chemistry poster session on Tuesday, 19 August. You may have guessed already but I used LaTeX - this time the A0 poster package. The poster title was "Avogadro: A framework for quantum chemistry simulation and visualization". I really enjoyed the two poster sessions and met lots of interesting people during the sessions.

You can grab copies of the slides or poster by clicking on their titles. It was certainly a very interesting conference, although it was so big it was difficult to choose where to go and what to see at times. Especially as some of the hotels with talks I wanted to attend were thirty minutes apart on foot. It was a great opportunity to tell other scientists about the work we are doing as well as introducing some of the concepts of open source to the wide and varied list of attendees I had the pleasure of meeting.

Chemistry Visualisation and Tools Meeting

marcus@cryos.net (Marcus D. Hanwell) — Thu, 20 Mar 2008 19:20:00 -0400

Last week I was privileged to be invited to speak at a meeting about molecular modelling with a focus on tools, GUIs and visualisation. The meeting was held at the Holiday Inn in Runcorn and the Daresbury Laboratory (England). I wasn't expecting to be back in England quite so soon, having only just returned to Pittsburgh at the end of January.

The meeting was a great opportunity to present some of the latest work I and others in the Avogadro and OpenBabel communities have been doing to create tools that enable the building of molecules and structures, as well as their visualisation. It was also a great chance to hear some very interesting talks by the developers of other building tools and some quantum codes. Donald and I were also invited to Daresbury Laboratory to work with some of the CCP1GUI developers.

I presented my talk on Avogadro on Wednesday morning and have made the slides available here. Donald gave an introduction to Avogadro, some of the history and the architecture before I gave my presentation. We finished by taking questions while I demonstrated the Avogadro application. I think it was extremely productive. We had many more conversations over dinner and drinks later as well as in a workshop setting on Thursday afternoon.

It was great to be able to put a face to a few of the names and discuss current issues more informally in the evening. The talks were all of a very high quality and from a varied list of speakers from other open source projects, some of the free quantum codes as well as commercial products. I have come away from the meeting with a much better appreciation of the needs in the community and I feel that Avogadro is in a great position to fill the apparent void.

I am glad that we were able to get surface and orbital support working in Avogadro before the meeting. Right now we only support Guassian cube files but the implementation is general enough that I will be able to add support for further formats. I really think that if we can get enough people collaborating on a common project everyone can get the tool they need to effectively do their research at a much lower investment than could be achieved by working on many separate projects.

I met Tristan Youngs, the developer of Aten, who had implemented some really nice features in his molecular builder that is much more focussed on molecular mechanics. It is well worth checking out. As is Zeobuilder which was developed by Toon Verstraelen. They both implement some great features and have strengths in different areas. Of course my dream is to integrate many of these features via Avogadro plugins and have one editor which is capable of being used in a diverse range of applications.

It was also great to speak to Mario Valle who is doing some very interesting work in the area of new visualisation methods and supports a large user base of computational chemistry users. There were of course so many other talks but you can look at the schedule yourself and I think the slides of all the talks should be available in the near future.

I feel sure that many good things will come out of this meeting and hope to be able to attend similar meetings in the future. I would like to thank Jens once again for hosting the meeting and taking care of everything. I hope to see some patches and/or commits from him in the near future

Avogadro 0.6.1 Released

marcus@cryos.net (Marcus D. Hanwell) — Sun, 09 Mar 2008 15:30:00 -0400

I am pleased to announce that I tagged and released Avogadro 0.6.1 yesterday evening. This is a bug fix release which fixes one pretty large bug that slipped through - the OpenGL context was lost if switching between virtual desktops, multiple views etc rendering the OpenGL window useless unless the application was restarted. As such I would encourage anyone running Avogadro 0.6.0 to upgrade to this new version. It also features several smaller bug fixes and feature enhancements.

The screen shot above shows Avogadro 0.6.1 running in a KDE 4 session. One of the small visual tweaks I made was to add a second light source to our default OpenGL scene which really helps to illuminate the other side of the scene. Thanks go out to Albert for his suggestion of adding another light source. Hopefully there are no really big bugs remaining but Avogadro is still in the beta stages of its development. It is rapidly approaching a stable release though and I am very happy with our progress so far.

We would love to hear what you think of Avogadro. I had one person question why we always have to use the latest and greatest version of OpenBabel and felt I should offer some explanation. Many of the features exposed in Avogadro use functions and structures in OpenBabel. I myself was quite heavily involved in improving OpenBabel's support for Gaussian cube files and the cube format so that we could load and display orbitals for example. As such we often add new features or fix bugs in OpenBabel trunk and so a new release of OpenBabel must be used in order for everything to work.

There are already ebuilds for this latest version in the Gentoo tree. Ubuntu/Debian builds are in the process of being built. We should hopefully have Mac and Windows binaries very soon too.

I am headed to a meeting in the UK where Donald and I will be talking with other scientists about visualisation in chemistry and related areas. We will of course be showing off Avogadro as well as talking with many other people working in this area. I am very much looking forward to it and hope that this will lead to further innovation in the Avogadro project as well as the open source chemistry movement in general. It will of course be great to have a full English breakfast and some real ale too!

Avogadro 0.6 Released

marcus@cryos.net (Marcus D. Hanwell) — Fri, 29 Feb 2008 22:01:45 -0500

Earlier today we released Avogadro 0.6. This release is quite overdue and we had originally planned to make a release around the new year. This release has many, many new features that have been implemented since the last release.

I am especially proud of the new support for displaying surfaces. This initially led to the surface engine that displays a Van der Waals surface. Tim added the ability to map the electostatic surface potential onto that. I then began work on adding support for displaying orbitals. This led to me getting commit access to OpenBabel and making quite a few commits as I got Gaussian cube loading working. While I was there I couldn't resist improving the Grid classes (still a little more I would like to add in).

Another new feature I am really pleased with is the QGraphicsView based periodic table. I think it works very nicely and I am hoping to add this code to Kalzium. I think the Avogadro library is in great shape for me to begin porting Kalzium to use it. Tim and Geoff have been doing some amazing work in OpenBabel on improving the force fields used for geometry optimisations.

We have also been doing a lot of work to improve Avogadro's performance when rendering large systems. We have added quickRender functions and OpenGL display lists to improve interactivity as well as threading calculations where this made sense. We have a cool little colour widget Geoff coded too that allows colours to be displayed and picked easily, improved default layout and one of my personal favourites - persistent settings in most of Avogadro.

There are certainly some bugs remaining. I would like to get to the bottom of as many as I can and hope to make one or two bug fix releases in the 0.6 branch. We would love to get feedback from people. We have already made a source release and a Mac binary. I will be adding Gentoo ebuilds shortly and we will hopefully get a Windows binary out by early next week.

Hope you enjoy this release. I think we are getting close to a stable API that we can call 1.0 and have most of the core features I was hoping to get into Avogadro. Our framework is extremely modular and extensible and I hope that we will be able to build up a community around Avogadro. It was also great to be able to make this release on 29 February - we have so few of them