mickeyl April 30th, 2005

BitBake / OpenEmbedded / OpenZaurus / Opie / GPE

Introduction

Salve! Time flies. Once again, half a year has passed and I think it’s definitly time for another newsletter. A lot of exciting things have been happening in the last six months and once again - great things are forthcoming.

BitBake

BitBake is a highly configurable flexible dependency parser and task execution tool. It can be downloaded from Berlios.de.

Since we released BitBake 1.0, it has received hardly any changes except some minor bug fixes. It has proven very stable and so far we are very happy with what you can do with BitBake - as are a lot of people who recently started using it. While the overall reception of BitBake is very positive, we frequently get two important complaints.

One complaint addresses the parsing time which takes very long. Parsing time is linear to the number of packages being parsed, this means you can improve it by telling BitBake to parse less files. The best way to do that is by setting an appropriate BBMASK in conf/local.conf. BBMASK is a regular expression, so, e.g. to mask out everything ‘FOO’ related you would write something like BBMASK = ( FOO-* ).

The second complaint is about overall memory usage. BitBake is a memory hog - admitted. This is due to the way the metadata structure has been implemented. In version 1.x, all the metadata is held in memory, which means that memory usage is also linear to the number of packages being parsed - hence the aforementioned BBMASK strategy will improve that as well.

There are several plans to improve the BitBake performance. Chris ‘Kergoth’ Larson is working on BitBake-NG, a reimplementation of the BitBake core from scratch. This is a long term project which is likely to result in a slightly incompatible metadata format. Other projects are short term and less intrusive to the metadata format, e.g. there’s the bbcow patch which uses a copy-on-write technique to lower memory usage. Holger Freyther is working on a sqlite based data storage backend which would greatly reduce memory usage. Last but not least there is a preliminary .bb parser written in C using ‘lemon’. This is also a very interesting project which I’d like to see pursuited, since we could develop a Python extension based on that and then use this parser as a drop-in replacement for our current one.

OpenEmbedded

OpenEmbedded is a set of receipes and metadata to build Linux distributions for embedded devices with the BitBake build system.

As always, first some current numbers: Since the last newsletter has been published, the OpenEmbedded metadata repository has received 3600 ChangeSets, that makes 600 ChangeSets per month and an average of 20 ChangeSets per day. Around now ten people are frequently committing ChangeSets and a couple of people are submitting patches to the OpenEmbedded mailing list - the archives of this list are hosted at gmane.org. The current number of BitBake receipes in OE is 2320 and these receipes create over 3500 packages - which is a quite nice amount of libraries and applications for your distribution feeds.

The latest successful releases based on OpenEmbedded were OpenZaurus 3.5.3 and Familiar Linux 0.8.2. It looks like the next batch of distributions will come out after the summer holidays. See also below for some important news concerning the future of these distributions!

Previous editions of this newsletter always featured nice and small BitBake files to demonstrate how easy it is to get new packages added to OpenEmbedded. This month I’ll show you that it’s also relatively simple to add something more complicated like a C runtime library. glibc is one of the toughest challenges for the aspiring cross compiling adept :). Hence, this months featured BitBake file is glibc_2.3.3+cvs20050420.bb.

To talk about some bad news… due to circumstances beyond our control, BitMover has said it will no longer give out future versions of the BitKeeper source control management tool for free. You may know that BitMover hosts the OpenEmbedded metadata repository and we use BitKeeper to develop OE in a distributed way. There hasn’t been a decision made about what system to use in the future. Neither do we have to abandon Bitkeeper immediately. We will most likely continue to use it as long as it works and in the meantime look out for a free software alternative. Please don’t suggest us using CVS or SVN since they are fundamentally different from the way BitKeeper worked. Once you get used to a distributed SCM you don’t want to go back to centralized one.

On the PR front - I visited this years FOSDEM and not only did I met a couple of nice people and drank a few nice Leffe Brune, but I also gave a presentation in the Embedded Developers Room about how to use BitBake and OpenEmbedded to build embedded Linux distributions from scratch. Unfortunately there are no audio or video recordings from my talk, however the slides can be downloaded from my site.

Let me conclude this section with a call to all individuals and groups working on custom Linux distributions: OpenEmbedded has been designed to be platform and distribution agnostic and to support adding new targets very quickly. If you want to leverage the power of BitBake and OpenEmbedded for your device, then contact us and join a wonderful group of knowledgeable people and let’s work towards a bright future of free (embedded) Linux distributions!

OpenZaurus

OpenZaurus is our alternative Linux distribution for the Sharp Zaurus family of PDA devices.

We have released OpenZaurus 3.5.3 in the first week of April featuring Opie 1.2.0, GPE 2.6, and Bootstrap images. Supported devices for this release were the SL-5×00, SL-6000, SL-C7×0, and SL-C8×0. Support for the new models SL-C3000 and SL-C1000 is coming along well. While you are reading this newsletter, we already released an OZ 3.5.3 snapshot with Opie for those new devices. Special thanks go to OZ developers Jose Bernado Silva and Jonathan ‘Noodles’ McDowell who were leading the Akita (C1000) and Spitz (C3000) porting efforts.

For the Opie image, OpenZaurus 3.5.3 debuts Kernel 2.6.11 on the C7×0 and C8×0 models. This is a great step since it eliminates the last bits of nasty Sharp kernel code in our free software distribution. What’s even more exciting is that most of the board support and core drivers have been submitted to mainstream and were accepted. This was only possible because of the labour of our kernel hacker extraordinaire Richard ‘RP’ Purdie. Kernel 2.6.11 is working much better on the C7×0 than 2.4.18-embedix ever did. All hardware except the Sharp Camera Module (for which Sharp never released drivers and which is of too low quality to justify any work in writing drivers) works with kernel 2.6. Yes, that means also SD/MMC will work on C7×0 with Kernel 2.6. Sharp used the standard PXA interface on these models - contrare to the infamous Locomo chip used on SL-5500 and SL5000 (collie) for which we most likely never will have SD/MMC. We released the kernel 2.6.11 for Opie because we felt it is ready for public consumption, although some minor things are unfinished yet. Richard maintains his own page for the 2.6 status on C7×0. Stay tuned for an updated GPE image using 2.6 which will also be released this month.

Dirk Opfer recently started working on 2.6 support for the Sharp SL-6000 board and made great progress. We’re looking forward to shipping a later version of OpenZaurus for the SL-6000 with kernel 2.6. The same goes for 2.6 support on the SL-5×00 models, which is being led by John Lenz. If you want to help developing for these models, visit Johns page or contact our handheld kernel developers via the kernel-discuss mailing list.

C1000 C3000

Now that OpenZaurus also works on the SL-C1000 and SL-C3000 which are using the same messy and broken 2.4-embedix kernel as the older models (in fact they are using a much more retarded one since they apparently took 2.4.18-rmk7-pxa3-embedix-sl6000 and applied all sorts of machine unspecific patches to it including diffs to bring it to 2.4.20), we are desperately looking for someone motivated to dive into 2.6 support for these models. Thanks to a lot of common code which already has been submitted to mainline by Dirk, Richard, and John, it should be much easier to support these new models than it was for the previous models.

On the donation front I’m glad to report that Marcin ‘hrw’ Juszkiewicz has received a SL-C760 and Michael ‘Mickey’ Lauer has received a SL-C3000. The latter being donated by SIEMENS Germany in return to having successfully used BitBake and OpenEmbedded to complete a project. Both devices will be used to improve the OpenZaurus experience on these models. Many thanks to the respective donators! Unfortunately although the OpenZaurus team dedicates a lot of their spare time to developing open source software and delivers a lot of work we are getting much less donations than other free software teams. So, if you are using the software we are involved in, consider donating to the developers. Note that the OpenZaurus project as a whole has no account to donate money to, however the core developers have individual accounts. Please make use of those :)

Opie

Opie is a graphical environment for PDAs, WebPads and other (embedded) devices. It contains a large number of applications in the categories PIM, Networking, Graphics, Settings, etc. Specific device supported has been integrated for all available Zaurus models, most IPAQs, the SIMpad, and the YopY.

After having released 9 versions of the unstable tree, we finally released Opie 1.2.0 on March, 25th. Since the last stable release Opie 1.0.3, there have been over 90 bug fixes to increase stability and improve the user’s experience. We incorporated a number of new applications, in particular

• PIM data conversion - converts contacts, calendar and to-do list between XML & SQLite formats,
• New Opie E-mail client - supports SMTP, POP and on-line IMAP,
• Opie-Eye - advanced image viewer,
• Package Manager - the next generation application package installer,
• Wellenreiter II - a network discovery tool,
• Opie-Camera - a tool for capturing images using the Sharp CF camera accessory,
• Gutenbrowser - an electronic book viewer supporting the Gutenberg Project’s vast collection of free texts,
• Dagger - a viewer for electronic texts using the Sword library,

Work on the stable tree continues with bug fixes and various improvements to look and feel. Dan ‘drw’ Williams did some great work with Qt/Embedded and Opie patches that improve the scaling of icons for hires devices. Alberto ‘Skyhusker’ Garcia fixed some bugs in libopienet and system time. Opie Bluetooth and Networking is currently under redesign and we are also working on icon themes for later releases. In general, the plan is to release Opie 1.2.1 when the next OpenZaurus release is due and subsequently more 1.2.x (stable) releases.

It is currently under discussion among the developers if there will be another unstable tree again, and if so, what to focus on - or whether to start from scratch with a different GUI Toolkit and some fresh ideas. If you’re interested to follow or even participate the discussion, see this thread’s archive @ gmane.

The latest ChangeLog can be seen here. As usual we also want to encourage you to join our team to work with wonderful people on a wonderful project. Opie welcomes contributions in every aspects. You can translate, document, work on our wiki based website or send in other patches for the tasks you could tackle. We look forward for your contribution and hope to meet you soon on our mailing lists or on IRC.

GPE

A lot of work happened on GPE regarding both user interface issues but also “under the hood”. New features have appeared in the following applications

• GPE-todo
• GPE-sketchbook
• GPE-conf

GPE-sketchbook in particular gained an sqlite backend and GPE-conf features improved keyboard configurations. Finally, GPE has a new picture gallery to show-off all these new features!

Imendio have been working on replacing the use of bonobo/ORBit with dbus in two oft-used libraries, gnome-vfs and gconf - see his project website. This greatly reduces the required dependencies for a lot of Gnome applications, making it an easier job to port said applications to the GPE framework without removing functionality. This also reduces the amount of dependencies of gpe-filemanager, making it a more viable possibility that it be included in the base-image for devices with a low amount of flash memory in the future.

A concerted effort is being made by GPE developers to ease the process of contribution. This includes fully documenting all the GPE-specific libraries and adapting current projects to use autotools for configuration. Building a GPE app will become as simple as “./configure && make && make install”. A new CVS time-line can be viewed here as well as a dependency graph.

Thanks to Richard Purdie and his on-going work on the 2.6 kernel, many bugs have been fixed and improvements made for the Zaurus clamshell machines that affected the use of GPE.

GPE gained a new multimedia playback application, in the form of gxine. Work has been undertaken to improve its interface and operation, in respect to small-screen devices. This work will hopefully be included upstream eventually, but patches will be applied in OpenEmbedded to give enhanced functionality now.

Work is also planned to improve the interfaces of more GTK applications in the future, such as Gaim and XChat. GPE also has a new gpe-helpviewer application, to present the user with an easy to use and familiar interface for getting help. The helpviewer is still not in its definitive form, but it is improving nicely. It is using gtkhtml to do basic rendering, so that help pages can be easily created and maintained. The most important changes up ahead are harmonizing the gpe-help calls to use the helpviewer and chapter support. Also a different html rendering core might be used to minimize flash usage further.

The Future of OpenZaurus, Familiar Linux, and OpenSIMpad

In 2003, the OpenZaurus team founded the OpenEmbedded initiative to develop a set of distribution and platform agnostic tools that could be used to create software for (embedded) Linux systems. In 2004, the handhelds.org team joined the OpenEmbedded team to work on integrating the iPAQ and Familiar Linux specifics into OpenEmbedded - so that future Familiar Linux releases could be built up from scratch using the BitBake tool and the OpenEmbedded metadata. This joint venture lasted since then and it was very successful. We are now in the position to do frequent releases and to cooperate by the means of the OpenEmbedded metadata repository. Among all the Linux distributions built using BitBake and OpenEmbedded, three of the (among the Handheld, PDA and WebPad community) widely known ones are

• Familiar Linux,
• OpenZaurus,
• OpenSIMpad.

Back in the days, when we started to work together as the OpenEmbedded team, we found it would be better to keep these distributions under their known “brand names” while we were completing the migration from the non-OE releases to the OE based releases. After a few successful releases and the integration of the new linux-eabi, forthcoming distribution releases will be binary compatible with each other (and with debian-arm, btw.).

This situation gives us the possibility to do the next logical step and complete the merger by starting a new spin off distribution, that will be dedicated to the devices that previously were targetted by Familiar Linux, OpenZaurus, and OpenSIMpad. This will have many advantages, including unified compatible feeds, less duplicated work, more evenly distributed maintenance workload, and much more. We are commited to making this new project evolve as a small, fast, stable, and professional Linux distribution for all kinds of devices - including handhelds, webpads, settop-boxes, PDAs, POS, and perhaps also mobile phones. In light of this new distribution, OpenZaurus and OpenSIMpad will phase out. Familiar Linux will be continued to developed, however with a different workflow and a slightly changed focus.

Although we have already agreed to perform the merger, a lot of details are under discussion, e.g. the name for the new distribution, the timeframe, logo, and many more topics. At least for the question of the naming, we need your input. Please visit http://handhelds.org/moin/moin.cgi/NewDistroNames and contribute to our brainstorming.

Epilogue

I hope you enjoyed reading the 6th edition of the OpenZaurus/OpenEmbedded/Opie/and friends… newsletters. Feel free to provide feedback or send in questions. Please don’t send support inquiries for any of the aforementioned projects to me personally - they will just be ignored since mailing lists exist for a reason. Questions of general interest will be answered in the next newsletter.

Thanks, Michael ‘Mickey’ Lauer.

• misc