Marcus D. Hanwell's Blog

So there have been quite a few people asking what is happening with KDE 4.1 on Gentoo. There are still no ebuilds in the tree for KDE 4.1 and to be totally honest I have not been actively developing the KDE ebuilds until recently (took quite a long break). It is however something I really feel that we should have and so I have been working with a few other developers and interested users on the new ebuilds in an overlay. I think these ebuilds are almost ready and I am very eager to get them into the tree.

So what have we been doing? We have been working on a set of ebuilds that can be used with portage 2.2, there were a few setbacks when it became clear that EAPI 2 was not likely to be allowed in the tree in the very near term, but good noises are being made on the mailing lists. I think one of the big changes I have been working on is bringing KDE back into the normal file system hierarchy. This means that you will not be able to slot multiple minor versions of KDE such as 4.1 and 4.2 together.

The loss of slotting also means that ebuild maintenance will be significantly easier, externally released packages will automatically relink to the latest kdelibs for example. It also means that users will not build up multiple copies of minor KDE versions such as 4.0, 4.1, 4.2. I think for the majority of our userbase (myself included) this situation is undesirable with few benefits for normal use of KDE. So having one slot for KDE 4 and upgrading in the normal fashion seems to be very beneficial.

This has not been universally accepted as a good thing. I personally think our main tree KDE should always be installed in the normal FHS hierarchy as upstream seems to intend it. I see little benefit for most users, and many developers not closely involved in KDE development, to having multiple minor versions installed. Also having external KDE packages such as k3b and amarok installed in /usr but linking to libraries in a KDE prefix has never been optimal.

My vision is for a default in-tree KDE that installs into the normal FHS tree as Gnome, Qt 4, XOrg etc do. Developers, power users and those that like to tweak would still be free to install slotted versions of KDE in KDE prefixes alongside the FHS KDE. They could also remove the FHS KDE and just use slotted versions too. We discussed during KDE 3.5 bumps how the current slotting was not ideal.

I am certainly open to opinions. It would be good to get wider feedback from the community on which direction they would like to go in and why. This will not affect KDE 3.5 slotting with KDE 4 - they will always be slotted. It will make maintenance of external KDE packages simpler as everything will be in the same tree. Overlays with ebuilds in alternate slots and prefixes are of course still easily possible and people could continue using KDE in this way.

Currently this work is taking place in an overlay. I am quite honestly exhausted discussing why this is or is not a good idea. I hope I have made my position clear and why I think it is a good thing for our user base. I really want to get KDE 4.1 into the tree as soon as possible. I do feel that these changes should be pushed into the tree, defaulting to an FHS compliant install and using a KDE prefix if requested.

What are your thoughts (other than get KDE 4.1 in tree now - we are working on that)?

As my regular readers will know I moved over to Pittsburgh, PA in the USA at the end of September last year. Before I left I got some of mine and Louise's family webcams so that we could stay in touch. Then it took ages to sell our house and the shipping nightmare began (which I will try and bring myself to talk about one of these days). Eventually I actually got my desktop computer back (a little worse for wear after the shippers "carefully packaged it").

Going back to my original point, I had been putting off buying a webcam as I wanted it to work well in Linux, and every time I looked into which webcams might be my best bet it seemed that even individual models had multiple chipsets, which may or may not be supported. On Saturday I bit the bullet as Louise had been asking about getting Skype working.

I chose the Logitech Quickcam Pro 9000 (photo above) as it seemed to be part of the new USB video class standard I had been reading about and saw there was a healthy Linux UVC project already. I still anticipated having trouble getting it working on Gentoo but thought I should be able to get it working eventually.

Imagine my surprise when I just typed emerge -av media-video/linux-uvc media-video/luvcview, modprobed the new kernel module, ran luvcview and I could see me staring at the camera! Sometimes I worry this Linux thing might be getting a little too easy Another five minutes and I had the microphone working (just usb audio) and was testing it out in Skype. On Sunday we had our first intercontinental video chat with my Mum and two nephews. Skype also integrates into my KDE 4 desktop without any trouble.

I am pretty new to the webcam thing and had resisted it for a while but it was great to be able to chat and see family and friends back home. I would rather use an open source, cross platform video conferencing application but have yet to find a viable one. Open Wengo looks like it might be that application one day but I couldn't get it to work reliably last time I tried (although I did like the look of it). I also don't seem to be able to find a nice application that will let me capture video messages easily.

I was very pleasantly surprised by my experience and am very happy with the performance of the Linux drivers for the webcam as well as the camera itself. Any Linux webcam tips would of course be gratefully received!

UPDATE: Forgot to mention this great blog post I spotted that helped me decide this webcam was probably a good bet...

On Monday we officially released Avogadro 0.8.0. I think this is a great release with many improvements over 0.6.1. We have been working really hard on improving the stability of Avogadro and I hope this shows in this release. There have also been some very nice improvements in speed and lots of new features.

The screen shot above shows an imported Gaussian formatted checkpoint file displaying the HOMO molecular orbital. The import of output from quantum codes and subsequent calculation of MOs is at a very early stage but you should take a look at it in the extensions menu if you are interested. It currently only calculates s, p and d orbitals and can only parse formatted checkpoint files from Gaussian. The goal is to add at least f orbitals and many more quantum codes.

I don't think I ever talked about the new periodic table QGraphicsView that I am very proud of. It allows you to select elements that are not listed in the combobox, shows the colour they will have on screen and is about as compact as I can make it. I have already packaged Avogadro 0.8.0 for Gentoo, we have binaries (Windows, Mac) and source available for download and should hopefully have packages available for Debian and Ubuntu very soon.

There were a few features that didn't make it in and a few bug fixes we found just after release. Once we have ironed out a few more bugs and received more feedback we will make a bug fix release. I was amazed to see that Avogadro is currently shooting up the Categories: Avogadro, Chemistry, Gentoo, GSoC, KDE | 4 Comments

After I got back from the UK in January my wife came to join me. She brought with her my old Acer Ferrari nearly working shy of a new hard drive. I got one from NewEgg at a fairly reasonable price and installed it. The drive arrived with a few bent pins, packaging isn't their strong point obviously, and took twice as long as they said but hey at least I had it. I had been stranded in America without Linux for more than three months and so another few days wasn't going to kill me.

Once I got the drive in it seemed the slot loading CD drive and got itself in a twist and would not load any CDs. An hour later and lots of pieces on the table I had reset it, put it back together (a few times) and got a Gentoo LiveCD in there. So I was in business. I had considered installing kubuntu but I wasn't getting on with it in the VM I had at work and I wanted to get back to Gentoo development too.

Everything looked good. Got an X server up and running. The ATI binary blob was as unstable as ever. Thanks to nerdboy (I think) I got the open source r300 driver working. This is much more stable although missing several OpenGL features we use in Avogadro such as smooth triangle shading, changing vertex colours in drawing operations etc. Other than that it works much better than the binary blob and is far more stable. I like stability in a system.

I have KDE 4 trunk running and am using it as my main desktop right now. Still got some rough edges I haven't had time to figure out just yet such as kmail refusing to save my IMAP accounts, kopete refusing to connect to GTalk and general saving issues most of the core seems pretty solid. I am compiling Thunderbird right now so that I have something. I will hopefully be getting back to Kalzium development pretty soon. If I find the time I would love to help out with KDE in a more general sense as well as helping out with the Gentoo packages.

My desktop is apparently in a customs area somewhere in New York. Delay after delay after delay with the shipping of my household possessions from the UK. So I am still without desktop hoping that an "intensive exam" is not too expensive and does not involve breaking any of my stuff :/ We shall see. Fingers crossed I will actually have furniture and my desktop computer back within a few weeks. That would allow me to be so much more productive having my dual core Gentoo desktop back, creature comforts and all that good stuff.

So keep your fingers crossed for me. Hopefully you will be seeing a lot more commits coming from me in the KDE and Gentoo repositories as well as the Avogadro work I have managed to continue to do on the MacBook Geoff so kindly loaned me whilst I was laptopless (is that a word? It should be!) Not sure that was a sentence now either...

I am pleased to announce that I tagged and released Avogadro 0.6.1 yesterday evening. This is a bug fix release which fixes one pretty large bug that slipped through - the OpenGL context was lost if switching between virtual desktops, multiple views etc rendering the OpenGL window useless unless the application was restarted. As such I would encourage anyone running Avogadro 0.6.0 to upgrade to this new version. It also features several smaller bug fixes and feature enhancements.

The screen shot above shows Avogadro 0.6.1 running in a KDE 4 session. One of the small visual tweaks I made was to add a second light source to our default OpenGL scene which really helps to illuminate the other side of the scene. Thanks go out to Albert for his suggestion of adding another light source. Hopefully there are no really big bugs remaining but Avogadro is still in the beta stages of its development. It is rapidly approaching a stable release though and I am very happy with our progress so far.

We would love to hear what you think of Avogadro. I had one person question why we always have to use the latest and greatest version of OpenBabel and felt I should offer some explanation. Many of the features exposed in Avogadro use functions and structures in OpenBabel. I myself was quite heavily involved in improving OpenBabel's support for Gaussian cube files and the cube format so that we could load and display orbitals for example. As such we often add new features or fix bugs in OpenBabel trunk and so a new release of OpenBabel must be used in order for everything to work.

There are already ebuilds for this latest version in the Gentoo tree. Ubuntu/Debian builds are in the process of being built. We should hopefully have Mac and Windows binaries very soon too.

I am headed to a meeting in the UK where Donald and I will be talking with other scientists about visualisation in chemistry and related areas. We will of course be showing off Avogadro as well as talking with many other people working in this area. I am very much looking forward to it and hope that this will lead to further innovation in the Avogadro project as well as the open source chemistry movement in general. It will of course be great to have a full English breakfast and some real ale too!

I am sure you have heard by now about the KDE 4.0.0 release made yesterday. Currently my desktop machine is on a boat in the middle of the Atlantic somewhere, my laptop is broken and I only have access to Apple systems... I was able to check out KDE 4 and test it in a VM and it is looking good.

I haven't been able to help with Gentoo packaging of KDE 4 very much at all which is a shame. I should get my desktop back at the end of the month in my new home in Pittsburgh. I also couldn't attend the release event as I am attending my graduation ceremony on the 17th of January. Hope you guys really enjoy the event and if I could have moved things around to attend I certainly would have.

There is lots of stuff I hope to be able to do for KDE 4.1, and more specifically Kalzium, once I get my desktop back at the end of the month. Avogadro already has lots of improvements including a KGraphicsView based periodic table which I think Carsten is interested in me getting into Kalzium 4.1.

Enjoy the release event and remember to raise a glass to those of us unable to make it for various reasons

Since moving out to Pittsburgh I have been using my Acer Ferrari laptop at home for development, keeping in touch with people back home and trying to figure out some of the weirder rules out here such as driving licenses in this state. Last night I booted up when I got home and after logging in everything went a little funky - got I/O errors when typing ls for example.

I tried rebooting but it wouldn't even get to the GRUB prompt. So I got out an old Gentoo LiveCD which I had luckily left in my laptop bag. That got so far then failed to mount the drive. After a few reboots and a little wine (to calm my nerves ) I managed to access the drive. I thought I would try copying some stuff across to my external hard drive and ended up getting more I/O errors and some memory errors for good measure. Half the time when I booted from the LiveCD the kernel panicked when attempting to mount the hard drive partitions.

So today I am a very unhappy English man in Pittsburgh. Due to me having spent far more than I budgeted for moving out here, and then more again on flying my dog out, I don't foresee being able to replace it any time soon. This means my development activity will be limited. Not sure if compiling part of a KDE 4 checkout might have been what finished it off. Looks like some kind of motherboard issue. The trackpad stopped working months ago, now I am getting random memory and I/O errors and the CPU has been running hotter and hotter recently. It served me for just over two years which isn't great but I worked it pretty hard and it travelled a fair part of the globe with me too.

Still can't help being cheered up a little as I will be picking Dax, my overgrown German Shepherd dog up from Dulles Washington International Airport tomorrow (not Dallas as it sounded like on the message they left me which made me very unhappy when I first heard).

Update: My very nice and generous new boss has very kindly offered to let me use the group laptop in the interim. It is a MacBook Pro though using something called Mac OS X Leopard. I can keep hacking on Avogadro with it and I even know how to take screen shots with it now too! It is also capable of checking email and browsing the web so my Internet connection in the apartment won't go to waste. I am considerably happier now but will miss my KDE 4 and Linux fix

Today we released Avogadro 0.2 which features many improvements since the 0.1 release. It has lots of the new features I worked on during my Google Summer of Code project along with some great work by other contributors such as the bond centric manipulation tool coded by Shahzad Ali, Ross Braithwaite and James Bunt.

Geoffrey Hutchison announced the release earlier today, on what I have been informed is mole day due to the date format used in the US - 10/23 corresponding to the 10^23 of Avogadro's number. So today seemed like an even more fortuitous time to make a release of Avogadro!

Today also marked my first commit to the Gentoo repository in quite some time to add this release to Gentoo's ebuild repository. If you have been waiting for me to add/update packages and I haven't gotten around to it due to life being so busy now would be a good time to poke me. I will also hopefully be completing the ribbon support I have already talked about and that should make it into a release soon but wasn't ready for this release.

We would love to receive feedback on the latest release of Avogadro. These new features will also make their way into Kalzium in KDE 4.1.

On Saturday I attended the Gentoo UK 2007 conference held at UCL in London. I would like to thank James and Peter for the effort they put into organising it. I gave a short talk on my work on KDE and some of the other work I do on scientific applications and libraries as part of the scientific herd. You can grab a copy of my slides here if you would like to take a read. I did just use them as a framework to talk around though...

I enjoyed the Gentoo UK conference. It had quite an informal tone and was a great chance to meet a few other Gentoo developers along with budding developers, users and other interested parties. I wasn't able to make the last couple of meetings down in London so it was nice to make this one, although I could only make it down for the day. I took quite a few photos that I have put up here. Both sides of my talk received interest from different parties. I am most excited about my current work, Avogadro/Kalzium 3D molecular editor, which combines my two passions.

I attended my first aKademy conference this year in Glasgow. On Sunday I gave a talk on Gentoo & KDE, you can get the slides here and download a video of my talk (warning - 107.6 MB). You can also see all the other talks and videos/slides on the programme page.

I think the talk went well and it was certainly useful for me to meet lots of KDE developers. I was there for the whole conference and had the opportunity to learn a lot from some really great talks. There was no internet connection for the first day and a half which prevented me from bloggging and checking my mail but did let me give my full attention to the talks.

I think my talk was well received and got some interesting questions. I also had some very interesting discussions after the talk and look forward to getting some work done on the KDE 4 ebuilds. I am still around for the rest of the week playing both the Kalzium GSoC student and the Gentoo developer.

I hope that my talk was of interest to those who attended it or watch it on the internet. My greatest hope was that my talk would help to improve the way we work with KDE and I feel I have achieved that goal. The next few months will tell just how successful I really was. I will be giving another talk at the Gentoo UK conference on 14 July on KDE and scientific applications.

My blog is aggregated on three planets now, Gentoo, KDE and SoC. So I guess my readership will be at its peak. There have been some hurdles leading up to my Google Summer of Code (TM) project but I am very pleased to have made it and intend to make the very best of this amazing opportunity to work on what is a really exciting project for me.

I will be working on the 3D molecular editor project which was an idea proposed by Carten Niehaus on the KDE GSoC 2007 ideas page. I have to admit that when I read it I was very excited. I had read some stuff on the Planet KDE (which I have read for ages and am really proud to be added to now) about the new 3D molecular viewer for Kalzium.

I was hoping to be able to find an area I could help out with that when I discovered the idea that looked like it had been written for me! I then discovered the Avogadro project and met some of the project's developers. As I was putting my application together I helped with a few small patches and ported the molecule navigation code across to Avogadro from Kalzium. I also provided a patch for a Van der Waals sphere view of the molecule as I have always liked that particular method of visualising molecules.

Some of my patches were accepted and integrated. Eventually I heard back about my GSoC application - I was very pleased to learn it had been accepted. Unfortunately for personal reasons I spent the next few weeks offline (after letting my new mentor know) and didn't get to celebrate straight away.

Once I was back online I got commit access for the KDE subversion repository, my commit name is "hanwell" as they don't like you to use nicks. I also got commit access to the Avogadro repository where my commit name is "cryosuk" due to my usual "cryos" already being taken on SourceForge. So now I am committing under multiple aliases in different projects

Benoit Jacob is my mentor along with Carten Niehaus, Donald E. Curtis and Geoff Hutchison who have all been very welcoming. They already contributed so much to the project I am just getting involved in. Open Babel and the Avogadro library underpin much of the 3D molecular editor and I have already made quite a few commits to the Avogadro and Kalzium subversion repositories.

There was an application freeze planned at the start of June for KDE 4.0, so I started my project early in order to get some of the great new features of Avogadro into KDE 4.0. Some of the highlights of this work were porting the navigate tool, adding the manipulate tool, various improvements to the label engine as well as some small enhancements to the engine API. We had the goal of ensuring all kalzium 3D viewer code was ported to the Avogadro library before we began porting Kalzium to use the Avogadro library to display molecules.

We accomplished that a few weeks ago and then began porting Kalzium to use the Avogadro library as its 3D visualisation backend. Along the way the Avogadro project made its 0.1 release, I added an ebuild for Gentoo and also made a few posts about the initial work. The feature freeze has been delayed but we got everything we wanted to done before the original deadline which I am pleased about.

Today marks the first official day of my project. I have already implemented several features I had planned including the port to the Avogadro library and ensuring all 3D Kalzium code was ported to the Avogadro library. I am really looking forward to working on this project over the summer and creating an awesome 3D molecular visualisation application. I am lucky to be working with such a friendly and motivated team of developers and will be sure to make regular posts about what I am up to. I welcome feedback on my work and have placed a screen shot of Avogadro (first picture) and the 3D molecular viewer in Kalzium (second picture).

I started my Google Summer of Code (TM) work a little early due to the upcoming feature freeze for KDE 4.0. Yesterday the Avogadro 0.1.0 beta was released and I have of course added an ebuild for it to Gentoo. Avogadro is at an early stage in development but it is already packed with features and I am really pleased that I have been given the opportunity to work on this project for the summer.

Above is a screen shot of the Avogadro release viewing a 4-methylbenzenethiol molecule (the ones that encapsulate the gold nanoparticles I work with on my PhD project). I have done a lot of work on the navigation and manipulation tools as well as adding the Van der Waals sphere engine. I have been working for the past few weeks on getting the missing features needed to port kalzium to libavogadro into the library.

So I have already made quite a few commits to Avogadro, and yesterday I made my first commit to the KDE SVN repository. I am suffering from split personalities though On sourceforge I am cryosuk (due to cryos already being taken) and on KDE SVN I am hanwell (it is KDE policy to use surnames rather than nicknames). So now I have three nicknames/account names in use...

I have been doing some initial work porting kalzium to use libavogadro too. Right now my time is really limited for a number of reasons though and so I have posted my limited progress to the kalzium list. Hopefully I will be able to do a little more this weekend. We need to have it working before the feature freeze in June. I am confident we will be able to do that. More porting is certainly required but I do not think that much more time is needed before we get it rendering again.

We would of course to hear any feedback you have on Avogadro. We also have lots of new features planned, many of which I will be implementing over the summer as my project gets into full swing and I can work on it full time. Time for me to get back to my thesis - hope you like the Avogadro release!

In the past few weeks I have been doing lots of data analysis and graph plotting. I use some graphical tools such as Grace, Veusz and QtiPlot but when doing lots of plots for a big document I wanted something a little more scriptable.

ePiX seemed to fit this bill and when I contacted the author about a few problems I had getting it to work he was very responsive. He is a mathematician whereas I am a physicist and have to handle lots of data in general. So I found the data file reading quite fragile. It does however have some great advantages such as being able to write LaTeX equations right into the graphs and figures.

The image shown above shows a plot of the absorption profile of a gold nanoparticle suspension. In the recent ePiX 1.0.24 release I added a tokeniser to the file reading functions of the data_file class in order to make data file reading more robust and I was able to read all my data files in using a simple loop and plot them as shown.

I have also just finished a data pruning function for the same class, but I am not sure if there is a better way to implement it. It uses brute force right now to iterate through the data and erase unwanted points. It does work well but I am not sure if there is a better way to accomplish its goal using some of the STL algorithms. It does need to delete the whole row though.

  void data_file::prune(double min, double max, unsigned int col)
  {
    // Erase rows where the data is outside of the specified range
    std::vector::iterator> iter(m_data.size());
    for (unsigned int i = 0; i < m_data.size(); i++)
      iter.at(i) = m_data.at(i).begin();

    while (iter.at(0) != m_data.at(0).end())
    {
      if ( *iter.at(col-1) < min || *iter.at(col-1) > max )
      {
        for (unsigned int j = 0; j < m_data.size(); j++)
          m_data.at(j).erase(iter.at(j));
      }
      else
      {
        for (unsigned int j = 0; j < m_data.size(); j++)
          iter.at(j)++;
      }
    }
  }

This pretty much covers the extra bits I needed for data analysis. A nice legend function would be good as drawing legends isn't as automated as I might like. I would welcome any feedback on the prune function as I think this would make a good addition to ePiX.

Yesterday and today I went along to an Open Street Map event organised in Sheffield. When I get time I will talk about that more in another post. As I got back yesterday afternoon I got talking to Carsten about it on IRC and he introduced me to Torsten Rahn who developed Marble.

I have already been testing out some of the stuff in KDE trunk but had not yet had chance to try out Marble. It is still at an early stage of development but is already very nice to use. I have added Marble 0.3 to portage as it can be built using just Qt 4. It is a great 3D globe that doesn't use OpenGL and so actually works very well on my laptop that doesn't have 3D accelerated drivers.

As Anant Narayanan had already done a lot of work coordinating the update of Gentoo developer locations I asked him about getting a list in KML format which Marble supports. Above is a screen shot showing most of Europe and the developers located there. You can search for your favourite developer and even measure the distance between them.

It is only keyworded ~amd64 so far. If you are on another architecture you can always test it and ask your friendly architecture team to keyword it if it works well for you. Torsten has lots of stuff on his todo list for Marble. He will also be giving a talk on it at aKademy this year. One of the things I am particularly interested in is Open Street Map support. I am also looking forward to tile downloading in the next release.

I am pleased to announce that my talk submission on 'Gentoo & KDE' has been accepted. I am really looking forward to attending this year's aKademy which is being held in Glasgow. I am camping nearby and will certainly make it along to everything on the weekend.

I hope we will have other Gentoo users in attendance, I have learnt in the last few days that we have quite a few KDE developers who use Gentoo and even hang out in #gentoo-kde on Freenode. That channel seems to have more people in there everyday. I have always enjoyed meeting developers and users in real life after hours spent chatting online but have not had chance to make it to many conferences in the last year.

As part of my research in the last week or so I have been checking stuff out in KDE 4 and trying them out. I have to say the okular is already looking awesome. It would have been so good to have had this available throughout my PhD work but I think it will be well worth the wait. Kalzium is also looking fantastic with its 3D molecular editor. I am hoping to help make it even more awesome this summer too. Hopefully more on that later...

Some other stuff I haven't managed to get working just yet but it is in very heavy development. I only wish I had more time to help out now but finishing my thesis is currently top of my priority list There are new versions of digiKam and K3b in portage that are looking great but personally I can't wait for KDE4!