mickeyl August 22nd, 2005

BitBake & OpenEmbedded / OE-based Linux distributions / Opie & GPE

Introduction

Salut! It’s time for another newsletter. A lot of exciting things have been happening in the last months and once again - the greatest things are still forthcoming.

BitBake

BitBake is a highly configurable flexible dependency parser, and task execution tool. It can be downloaded from Berlios.de.

Since the last newsletter, a lot of work happened around BitBake. The most exciting development was lead by Überhacker Holger ‘Zecke’ Freyther, who reimplemented the BitBake internal data storage model gaining us a reduced memory load of about an order of magnitude. BitBake now seldomly needs more than 100MB of RAM - compared to using more than 1GB previously. Cheers, Zecke!

Michael ‘Mickey’ Lauer (yes, that’s me) added an interactive mode to BitBake - the BitBake Shell. I found that when I work on OpenEmbedded, I often have a sequence of characteristic commands, i.e. create a new package, rebuild it, edit it when it fails to build, build it again, call showdata, build it again, set some variables, etc. There is nothing you can’t do with a good set of aliases and bash functions, however, when you’re rebuilding a providee (as opposed to a bbfile), or you just want to build yet another provider after you built a different one, the parsing time can be somewhat annoying. The BitBake shell improves many of these issues since it decouples parsing from building and gives you interactive control over its functions.

Here is a link to an example session to get you an idea of the things you can do with the shell. The most recent additions is a glob syntax for providers (i.e. you can say build *opie*. You can start the interactive mode by issuing bitbake -i. Oh, did I tell you that almost all commands support tab completion? :)

The work on improving the BitBake (re-)parsing time is going one. Holger Freyther already integrated a part of Marc Singer’s work on a C-parser to improve parsing time by using a generated Python parser.

OpenEmbedded

OpenEmbedded is our award-winning(*) set of recipes and metadata to build Linux distributions for embedded devices with the BitBake build system.

OpenEmbedded is on a roll. The number of changesets per week is constantly rising and thanks to the new Bugzilla based bugtracker, we get patches and new .bb files from more and more contributors. Nowadays, we have an average of 20 developers regularly contributing to OpenEmbedded - thanks to all of you. The current number of BitBake receipes in OE is 2600 and these recipes create over 6000 packages - which is a quite nice amount of libraries and applications for your distribution feeds.

Support for OpenSlug, the distribution for the LinkSys NSLU2 has been improving greatly - the nslu2 team is telling us that a new 2.x release is imminent. We also received some patches for the LinkSys WRT54G(S), although more testing from people with serial cables would be necessary until we can release something.

One word to the nslu2 developer group - we are pretty glad that they chose OpenEmbedded as the base for OpenSlug since they are much less PDA centric than most of us are. Thanks to those people, a growing number of packages in the OpenEmbedded repository become also useful on devices like routers, set-top-boxes, and other multifunctional gadgets running Linux.

Florian Boor and Koen Kooi led the integration of the Maemo suite into OpenEmbedded. Maemo is the open source framework for the software found in the forthcoming Nokia 770 internet tablet and thanks to the work of Florian and Koen we will soon be able to build an OpenEmbedded based distributions for this device. I had the chance to play with a Nokia 770 on LinuxTag 2005 … and I already ordered one :)

In the beginning of August, Mickey added teTeX and pine, both which are among the more complex packages because they come with build systems that are not exactly sane^h^h^h^happropriate for cross compiling. Hence, this months bitbake file is Pine 4.63.

Since 1st of July, the free BitKeeper license is no longer. In the last weeks of June, we successfully migrated our repository over to the Monotone distributed source control management system. We are very happy that monotone exists, because it provides us with nearly the same workflow we had when we were using BitKeeper. In a few areas, monotone performs significantly worse than BitKeeper, i.e. the initial pull time is pretty high - however the monotone folks are willing to improve these things and until someone presents an alternative that performs faster while retaining our established workflow, there’s no point in complaining about monotone. See openembedded.org and monotone.vanille.de for more information.

(*) I’m glad to announce that OpenEmbedded won the first prize in the annual TuxMobil GNU/Linux Award 2005. This award honors free software projects that improve the Linux experience on mobile devices. Thanks to all who nominated us, and to the jury of course! The SL-6000 we won will be owned by the project - that means a developer can request it for some months to develop for it.

OpenZaurus

OpenZaurus is our award-winning (*) alternative Linux distribution for the Sharp Zaurus family of PDA devices.

OpenZaurus 3.5.4 is just around the corner. Since 3.5.3 we incorporated a lot of improvements. The most notable ones are as follows:

• Kernel 2.6.13 on C7×0 - Kernel 2.6.13 is faster and has less latency on the Zaurus than we’ve seen in any kernel before. In particular this has been realized by improvements in the function that copies memory from user to kernel. Originated by Russel King, but actually made to work by OpenZaurus kernel hacker Richard Purdie. Richard is becoming an authority in the linux-arm-kernel world thanks to the bugs he spots and fixes in the bleeding edge kernels. Speaking about bleeding edge, the pcmcia, udev, and hotplug subsystems are getting revamped in the 2.6 series and although we’re trying hard to catch up subtle bugs in said subsystems are kind of expected.
• We now overload the kernel keymap on all Zaurus models except the 5500 and the 5600. This fixes the previously non-working extra keys found on the clamshell models and the SL6000. It also makes further improvements pretty easy, since /etc/keymap.map is just a text file. Note that you need to run /etc/init.d/keymap to make the kernel aware of the new keymap. Some GUI environments may also need restarting after you change the keymap. Eventually we will also have localized keymaps for the SL5500 models that were shipped to some european countries - without having to use gross hacks like the zkeyapplet in Opie. Of course this also applies to USB and Bluetooh keyboards using the standard HIDP (or kbdd, but let’s not talk about taiwanese BT keyboards using rfcomm…) which should work in the Linux console as well as in Opie and GPE out of the box now.
• A lot of work has been involved to get the Bluetooth subsystem fully working on 2.6. I’m proud to report that at least my card (a noname CSR CF card) gets recognized and attached to hci0 out of the box - it even survives a (hopefully unlimited number of) suspend/resume cycle(s).

Support for the newly released Zaurus model SL-C3100 (Borzoi) has just been added to the OpenEmbedded repository - in order to release an image, we now need people with this hardware to do some tests or people donating so that our developers themselves can test on the hardware.

We celebrate the 3rd birthday of the *cough* excellent 2.4.18-embedix Linux kernel. Cheers! Some of our Zaurus models running OpenZaurus 3.5.3 with 2.4.18 were plagued by a couple of hard to reproduce suspend/resume problems. Special thanks go to Hector Martin who - in a really altruistic and brave attempt to look at Lineo code - fixed the infamous collie kernel bugs in just a couple of minutes, namely the suspend-on-right-arrow-hold and the deadlock-on-resume. Quoting Hector: “Now I will never ever look at this code again.”

Work on Kernel 2.6 for the other Zaurus models is still continuing. Dirk Opfer is making progress with 2.6 on the Tosa (SL-6000) -which even boots right into Opie with a 2.6 kernel. BigAl recently started to work on 2.6 for the Spitz (SL-C3000). and Richard Purdie has been bought a Spitz device by his boss. This means we’ll soon have more Zaurus devices in 2.6 mainline, however we still need more help in this area, so if you want to hack on drivers or want to start a port for the new Akita and Borzoi, then please contact us. NEWSFLASH: Richard is currently tackling power management and is rewriting all the Sharp drivers for C7×0 and CXK into a common driver. He has some problems with voltage readings though and is requesting help. So… join the OZ kernel team NOW! :)

One of our mid term goals has always been to add the fantastic Enlightenment Window Manager to OpenZaurus. In the last three years, the team around E-Mastermind Rasterman has written the foundation libraries for something that is truly ahead of its time. To quote Rasterman: “In developing Enlightenment DR17, it was made clear that we needed an entirely new set of libraries and tools. Raster had a bold vision of what was possible and where he wanted the next release to go, starting with Imlib2 and EVAS, and eventually growing into new libraries largely based on or around EVAS. It became clear that the usefulness of these libraries and tools went far beyond the DR17 release itself, just as Imlib did in DR16. Thus the collective library back-end of DR17 was given the independent title: the Enlightenment Foundation Libraries, or EFL for short.” After a recent update of the Enlightenment Foundation Libraries in OpenEmbedded, Justin Patrin took over the maintenance of all E-related packages in the OpenEmbedded metadata repository and worked on the Entrance Login Manager and the Enlightenment Window Manager for his C3000 (see below). If this works out as we expect, we might ship Enlightenment as an additional X11-based image for the next OpenZaurus release!

Good news on the hardware donation front - Matthias ‘CoreDump’ Hentges got an Akita (SL-C1000) which is the first clamshell model for CoreDump. We now will get his fine AltBoot stuff and polished OZ-based releases also for this model. Btw., if you like the software we are involved with, then why not donate as well? See the list of our developers. Note that the OpenZaurus project as a whole has no account to donate money to, however the core developers have individual accounts. Don’t be afraid to frequently use those :)

The Opie people have always looked begrudgingly over to the GPE folks who rendered text with antialiased TrueType fonts. Thanks to pioneer work (and recurrent pestering) done by Marcin ‘Hrw’ Juszkiewicz, we finally switched over to using TrueType fonts in the Opie-flavour of OpenZaurus. This gives a much nicer appeareance, especially on the high resolution models. It also means we have some space problems on the 16MB models, but with OpenZaurus adopting the Hentges ‘AltBoot’ Style in further releases this will be a non-issue.

(*) Last but not least, I’m glad to announce that OpenZaurus won the second prize in the annual TuxMobil GNU/Linux Award 2005. This award honors free software projects that improve the Linux experience on mobile devices. Thanks to all who nominated us, and to the jury of course! The SL-C750 we won will be owned by the project - that means a developer can request it for some months to develop for it.

OpenSlug

OpenSlug is an alternative Linux Distribution for the LinkSys NSLU2 device created with OpenEmbedded and BitBake.

Last month, the OpenSlug team donated a LinkSys NSLU2 device to me - thanks very much! Originally, this is a 133MHz (or 266, if you kill R83 :)) ARM USB-Storage to Ethernet bridge, however thanks to the OpenSlug team this device gets much more versatile and the possibilities to are just limited by what you can do with two USB 2.0 Host plugs and one 100Mbit Ethernet plug on a Linux capable device.

The first thing I did with it was installing a beta release of OpenSlug and overall I’m pretty satisfied with how this worked out. The installation itself was a breeze - since I had the original LinkSys firmware on the device, I could flash OpenSlug via the LinkSys web interface. Upgrading from another firmware is not much harder either. After 5 minutes of flashing, my NSLU2 rebooted straight into OpenSlug. After 10 minutes more (using the excellent ‘turnup’ script), it booted from one of my USB sticks and I installed python, a web server, samba, and more. Stay tuned for some more detailed reports.

Opie

Opie is a graphical environment for PDAs, WebPads and other (embedded) devices. It contains a large number of applications in the categories PIM, Networking, Graphics, Settings, etc. Specific device supported has been integrated for all available Zaurus models, most IPAQs, the SIMpad, and the YopY.

Work on the stable branch 1.2.x continues and fortunately the amount of bugfixes outwages the amount of new features to debut in 1.2.1 - which is scheduled to be tagged later this month (August 2005). This means we’re commited to keeping 1.2.x really stable. One of the focused areas for 1.2.1 was the OResource initiative lead by Dan ‘Drw’ Williams who converted all of the Opie apps to using the OResource framework. Opie 1.2.1 will be the best release ever for the high resolution models because of the OResource subsystem painting large icons where necessary. We also got some patches to improve OBEX Push/Pull via Bluetooth from Michael Haynie which have been incorporated into the tree - thanks, Michael!

No one less than your faithful author contributed a PCMCIA framework for opiecore which lead to the development of the new pcmciaapplet. Compared to the old cardmonapplet, the pcmciaapplet has two important advantages: First, we no longer rely on spawning an external process (cardctl) and parsing its output but call the respective kernel functions directly - which improves the performance and the reliability of pcmciapplet. Second, I added a configuration framework, where you can specify actions that should be performed when a certain card has been inserted. This configuration is per-card. Last but not least you have the possibility to bind unknown cards to a driver already known to the system.

The future of Opie beyond what’s happening in the 1.2.x tree is a bit vague at this point. TrollTech has started to port Qtopia 2.x to Qt/Embedded 4.x and the rumours mill is telling that they’re supposed to finish this by the end of 2005. However, since - again - they’re developing this hidden behind their curtains, there is little interest among the Opie core developers to do the same in parallel or to wait until they have finished and then port our libraries and applications over to their framework. In the meantime, Bero has started to port Opie to Qt4/X11 which may serve as a better startpoint for future Opie developments - some of us are thinking Qt4/X11 based on the kdrive xserver is the way to go. This would enable our users to run Opie applications at the same time as other GPE and other X-based applications.

The latest ChangeLog can be seen here. As usual we also want to encourage you to join our team to work with wonderful people on a wonderful project. Opie welcomes contributions in every aspects. You can translate, document, work on our wiki based website or send in other patches for the tasks you could tackle. We are looking forward to your contribution and hope to meet you soon on our mailing lists or on IRC.

GPE

GPE is a graphical environment for PDAs, webpad and other embedded devices.

The past few months have been a busy time for the GPE team. A lot of bugs and small annoyances were fixed and most applications were tweaked for VGA screens. The GPE PIM puite was ported to the maemo platform as well as the GPE-mini-browser. GPE-mini-browsers is a small (~1MB) browser based on the gtk-webcore browser engine. It’s fast, small and under active development.

In the productivity department we now have evince, gqview, abiword, gnumeric, LyX and the gimp in OE and they are looking pretty good on a VGA screen:

Google has awarded Summer of Code positions to handhelds.org and three of them are GPE related:

• Mads Chr. Olesen in working on adapting the gaim UI for smaller screens
• Martin Felis is working on an OpenSync plugin for GPE
• Kirill Kononenko is working on C# and GTK# for ARM devices

GPE is slowly getting into mainstream distros too, Moray Allan has been making debian packages for some time now and Rene Wagner has submitted ebuilds to the gentoo people, so you can easily install and run GPE on your desktop:

Florian Boor and Nils Faerber are working on a development manual that will contain all the information and examples you’ll need to start developing on GPE! Better integration with other devices is coming along nicely, especially after the opie people decided to use kernel keymaps for zaurus devices instead of ugly QT/e mods. Things like autorotation and suspend when swiveling the screen work now. At the moment we are preparing for GPE 2.7, take a look at the GPE Roadmap to see what’s in store.

Ångström

Ångström is a new user- and developerfriendly OE-based distribution for Linux-based PDAs and WebPads which is currently in planning state.

The first Ångström release will support the HP iPAQ, Sharp Zaurus, and SIEMENS SIMpad family of devices. Here is a quick glance at the most important aims and policies for the upcoming Ångström distribution.

• Aims
  • Be userfriendly, have clear instructions and good pointers to docs, images should work out of the box for common tasks.
  • Have a central - structured - point of information. Ideally we would have all Ångström specific documentation on the Ångström site and general stuff in the hh.org wiki.
  • Be developerfriendly. Use nslu2-linux style makefiles, make every release bundled with an SDK + sourcepackages.
  • Have a low contribution threshold. Make it easy to get patches accepted, welcome involvement, be communicative.
  • Be compatible. Make package based upgrades and downgrades Just Work(tm).
• Policies
  • Don’t put all eggs in the OE basket -> allow native/scratchbox builds of stuff that isn’t in OE into the feeds.
  • Develop distro specific stuff in branches to gain some freedom and stability approaching releases, merge back successes after a release.
  • Almost religiously use tinderbox/buildbot/bugzilla/mantis/etc and maybe ubuntu’s malone if it’s ready.
  • Have Machine Mentors -> only release an image if has been tested on the device by two ‘machine mentors’. We will put up a list of vacant positions asap
  • Improve communication with other distros (familiar, pdaX, intimate, etc) about issues, bugfixes, and interoperability.

We already set up a mailing list for Ångström development @ handhelds.org. Visit Angstrom-Dev to see the archives and to subscribe or unsubscribe. There will be a multitude of places to become active in the Ångström community, so just join us and participate - make the distribution do what you want it to do.

For some of you, the most important question is ‘When?’ - I’m sorry, but there is no answer except ‘When it’s ready’. We have already fairly well working set of distributions out there and we are in no hurry. We rather do it right this time.

Epilogue

I hope you enjoyed reading the 7th edition of the OOO newsletter. Feel free to provide feedback or send in questions. Please don’t send support inquiries for any of the aforementioned projects to me personally - they will just be ignored since mailing lists exist for a reason. Questions of general interest will be answered in the next newsletter.

Thanks, Michael ‘Mickey’ Lauer.

• misc