Kalari at last

It took me much less time than I expected to put the new layout of the Wesnoth-UMC-Dev website together. Observe.

Okay, that's basically because most of the design was already made long time ago, in the form of the site's earlier incarnation, codenamed “Soradoc”, which looked rather busy and useless with the sidebar and other design elements. The new design, “Kalari”, removes the sidebar, clears the site banner a bit, and blends the site with Wesnoth.org as far as appearance is concerned. It's not the same design, but it's similar — that should be a good thing considering the purpose of Wesnoth-UMC-Dev.

That site also had a Blosxom-based blog, but I removed it since nobody was making actual use of the space.

The greatest thing about all this is that most of the PHP, “Poison Ivy” was finished in 1 night, while the rest took me just a few additional hours. Now that Poison Ivy is completed, I can reuse its code for the next incarnation of this very website blog.

It's all for teaching some web design and programming basics to myself, really.

Mozilla Firefox 3.5

Long, long ago, I talked about several issues I had with Mozilla Failfox Firefox 3.0 and openSUSE 10.3 for the AMD64/EM64T architecture.

Ever since then, I have learned several things:

  1. Debian's Iceweasel fork doesn't seem to be much ahead of mainline Firefox in terms of bugfixes, as far as I can see. This might be not true for security fixes and such; I admit I haven't done any actual research on this and I'm basing this statement on my user experience.
  2. The Download Day was a trap.
  3. Other people who I have talked to regarding Firefox's stability on Linux claim that is never/rarely crashes, but all of them use x86 kernels and userspace.
  4. Iceweasel 3.0 taints the Debian GNU/Linux 5.0 Lenny distribution on the AMD64/EM64T architecture, with no differences in either of my laptops. This Linux distribution is remarkably stable otherwise, and lived up to my expectatives since I originally switched to it when it was the Testing distribution — this is, comparing it to the released openSUSE 10.3.
  5. Off-line browsing is truly, horribly underestimated, to the point that one of the major web browsers does not support it at all; probably in favor of simplicity and ease of use, and “permanently connected people”. But, STILL... 😕
  6. It's not a good idea to leave a chainsaw and a newspaper near the reach of a cow.

I recently switched to Debian Squeeze, which is still under development (e.g. Testing) as of this writing. Originally, I just got a newer revision of Iceweasel 3.0 with the set. Some weeks ago, I got upgraded to 3.5.

As I mentioned in my previous post in this series, the status bar does still glitch a lot — no, wait — the status bar glitches even more than in 3.0. Scrolling is less laggy but only with smooth scrolling disabled, although I am not exactly using a well-supported video configuration at the moment and I probably should not complain about performance issues with any 2D application unless I'm willing to use the unaccelerated X.org VESA driver for benchmarking or shut up.

The Live Bookmarks feature stopped working after the upgrade until I went and manually reloaded every single Atom/RSS feed I had linked in a neat folder in the bookmarks toolbar. It took me a while to realize that nobody posting anything near Christmas was a bad sign — I didn't miss much anyway, since my feed sources aren't really chatty. Yes, I know I'd be better using an actual feeds reader, but I'm just that lazy, which is also why I don't use Opera as much as I want.

However, this version of Firefox is much, much more stable than 3.0 — as far as this AMD64/EM64T architecture user is concerned, that is. Firefox just got better, really. But it's still rather odd because I've heard comments on IRC of people claiming that it got more unstable instead. Hmm... Well, maybe Windows or x86 Linux users are less lucky this time?

Firefox 3.5 also supports the CSS text-shadow property, which was introduced in the CSS level 2 specification, removed in revision 1 (CSS 2.1), and seems to have been picked up again for CSS 3. No version of Internet Explorer before and including 8.0 supports this (although ISTR that they support a shadow filter using a custom extension to CSS that didn't even follow the specification for naming vendor-specific properties), and current Opera, Safari and Chrome support this property well. That means that I must make more use of it in this site's stylesheets from now on. 😉

Creativity drops

At this point, I should know better than tempting fate in a blog post:

[...] Some days before Xmas, my creativity returned from its long, chaotic journey and my Wesnoth add-on, After the Storm (sequel to Invasion from the Unknown has seen steady progress and two new releases were published in less than two weeks. Keep in mind that this add-on had not seen any public releases for almost a year. [...]

Two days afterwards, my creativity disappeared like a drop of water in the sea, again — which means that After the Storm has seen relatively no progress since then. I hope I get better next week, because this work needs to be completed as soon as it's possible.

Is it over already?

Finally, it's January! The New Year celebrations are mostly over and fading away, and people all around the world are going back to regular business and everything should be back to normal in a few days.

I used to be fond of the Christmas and New Year celebrations as a child as I could spend time with my family and eat delicious food. That is not the case anymore, because, even if I still live with my parents, there's no longer a sense of family here and we only want to throw sharp stuff at each other. There's not much enthusiasm by the end of the year anymore, and phrases such as “Merry Christmas” and “Happy New Year” (in Spanish, though) are truly unheard of in this house. Recent disagreements amongst us indicate that this is not going to be a good year for anyone. To add insult to injury, one of our cats died in a rather tragic and violent fashion on December 22th — it's a tradition here that one or more pets must always die in December. While we have many of them, the first ones to die are those whom we are most attached to.

To mark the actual start of 2010 (as far as the Gregorian calendar is involved, of course), there was a black-out on the area about 6 minutes 7 seconds past midnight, which left us with no Internet or tap water until around 1:50 AM. What a great way to start the first day of the year.

But there's still some hope at the moment. Some days before Xmas, my creativity returned from its long, chaotic journey and my Wesnoth add-on, After the Storm (sequel to Invasion from the Unknown has seen steady progress and two new releases were published in less than two weeks. Keep in mind that this add-on had not seen any public releases for almost a year.

After the last released version of AtS (0.2.1) including 5 of 12 planned scenarios in Episode I, there has been more progress in the Wesnoth-UMC-Dev repository. Just yesterday, I finished the two-part cutscene that is the sixth scenario of Episode I, one of the most important points of the plot's development, in which two forest elves finally make contact with the desert/Quenoth elves.

I won't be able to release AtS 0.2.2 or 0.3.0 until scenario 7 and the next cutscene (appropriately named “Resolutions”) are finished, since I'd be teasing the players otherwise. However, those who are really interested on it can always check AtS out from the repository's trunk into their <wesnoth preferences dir>/data/add-ons dir and play using the latest development version of Wesnoth:

svn co https://wesnoth-umc-dev.svn.sourceforge.net/svnroot/wesnoth-umc-dev/trunk/After_the_Storm

It's really exciting to work with several plot elements from quartex's Under the Burning Suns in new, creative manners — kind of like Fanfic production taken to a new level using the power of the GNU General Public License (version 2 or later!). Nevertheless, I am fairly sure he deliberately left much details unresolved in the original campaign, and that he'd fry us (Espreon, AI0867 and me) alive if he found out what we are doing with his story.

One week before Xmas, the Wesnoth.org forums saw another upgrade on which Turuk and I worked hard and quickly to improve forum usability by not only upgrading the codebase to phpBB 3.0.6, but also tweaking the templates, adding modifications and a couple of new forum styles to take advantage of the new features implemented by the phpBB devs in this iteration of their software. The main points were highlighted in this forum post (originally a Global Announcement).

This year should also bring us a new stable series (1.8) of the Battle for Wesnoth game itself. There are currently some problems delaying the first Release Candidate and getting us flooded with generic beta releases, but the developers in charge of them will (hopefully!) find a solution so we can get 1.8 released and trunk “thawed” soon, to work on new features and allow new code contributors to join the project. As for me, I can't wait for the new stable series — development series players seem to be scarce and the new versions of IftU and AtS are receiving little feedback on the forums because of this! I suppose Multiplayer content authors are similarly eager to get more fresh meat to play their add-ons.

I also recently talked about how I can finally suspend my laptop to RAM using Linux, and run some basic OpenGL-based software without crashing or destroying anything. That's something I didn't expect to be able to do in the near future, so the Mesa, libdrm and X.org radeon driver developers have my thanks for improving the Linux experience of those unfortunate enough to own onboard ATI graphics controllers!

In summary, as usual, a new year brings good and bad news. I guess it's up to us to take what's good and fix what's wrong. So, anyway (although I guess it's pretty much unnecessary at this point): happy New Year and have fun!

Google Chrome, Mozilla Firefox 3.5

Some days ago, I was told that Google had released a beta version of their Chrome browser for the GNU/Linux operating systems, all in middle of a discussion about how much Failbox sucked — er, I mean Firefox.

So, I downloaded a Debian package, installed, ran the application, and it crashed on my face right after trying to show the first-run page. That's not a very good start; not even Firefox does that. Then I try to run it again, it worked, okay.

Google Chrome seems to be rather nice, and reminds me of Apple's Safari in appearance. It's fairly quick to start compared to Firefox/Iceweasel 3.0 and 3.5 (which I now got from the Debian testing distribution), and its simplicity feels generally comfortable. I could quickly find an extension for hiding ads and browse some web pages afterwards.

It's pretty nice for testing web pages and even has some neat web development features. However, I'm probably not using it for regular browsing since it lacks an off-line browsing mode option as far as I can see. I have very good reasons for requiring that feature that Opera, Firefox and even Internet Explorer provide. Also, it doesn't respect some fontconfig settings in Debian Squeeze for some reason.

Besides that, I got switched to Iceweasel 3.5 as a consequence of running aptitude full-upgrade blindly. It does appear to be more stable than Iceweasel and Mozilla Firefox 3.0, although I should wait some more time before assuming this to be true; the 3.0 bugs on amd64 Linux were pretty random and sometimes I could run the browser for days without crashing, and in other occasions it would crash after running for a few seconds, while trying to contact a web server for the first time in the session.

There are no visible UI changes in 3.5 as far as I'm concerned (besides the tab bar, and some menu icons which got some hue changes), and some glitches remain — for example, scrolling pages is less laggy now, but only if smooth scrolling is disabled; and the status bar still glitches for the current web site if one clicks on a single link a lot. However, I've heard rumors that many security vulnerabilities were fixed in this version, so I guess it's a reasonable trade-off; not that I'd visit suspicious websites, or worse, run a web browser on Windows.

So, for now, I still use Opera when I need reliability, and Firefox when I forget that I'm supposed to use Opera.

Annoying spambot behavior

While checking the Wesnoth.org forums ensuring that everything is working properly, I occasionally find spambots who have managed to make it past our hacky anti-bot registration measures, probably with help from humans. Most of these spambots go into a hit-and-run attack leaving one spammy post in some random place of the forums — sometimes in existing threads, sometimes starting new threads themselves.

Two nights ago (November 30th, 4:45 AM), some guy came into our Off-topic section asking this:

Post subject: Linux

How long does it take to become a Linux Administrator? Also is 1 year of linux study enough to get a job as System linux administrator? What exactly do I need to learn to achieve that goal?

It was his first post, and there were no signature or links whatsoever. He joined the forums two days before that (November 28th, 8:08 AM). You may note that the message above is not in proper English, but there are many underage or foreign users in the board. Nonetheless, some regulars and I replied to this guy's message in good faith.

Spam screenshot

On the next day, December 1st 8:36 AM, “he” edits his message adding spammy links after a fake signature separator similar to the phpBB 3 subsilver2 template's, except a bit longer. The post spent approximately 14 hours as search-engine bait because no other Wesnoth developers noticed the cheat (or read the Off-Topic forum, I guess) and I was away for the whole day until I went in and found this surprise after a third look at the thread.

Fascinating. 😐

In one occasion I had one of these drones join the forums and make their first spam post after 6 weeks (and shortly get kicked out of the house). I guess it must have been collecting posts and feeding them to a learning engine in order to enhance future spam posts by fitting them into the board's context for the time being. Not that its own attempt at posting succeeded.

The problem is that they sometimes do manage to produce reasonable posts — as long as you use a very flexible definition of “reasonable”, which is more or less required around the Wesnoth.org forums these days especially since there are real people who join the board and never post anything constructive or interesting and roam just the Off-Topic forum asking weird, unimportant questions and answering other weird, unimportant questions with bogus, misleading or uninformed answers or opinions (no, not pointing at anyone in particular here). On the other hand, it's still possible that those weird guys or gals will eventually drop their masks and reveal their true identities as spambots. Or maybe someone else will remove it for them.

libpng's complexity

libpng is the reference implementation of the PNG file format, which has become quite popular during the last decade thanks to the benefits of lossless compression coupled with multiple storage formats — including indexed pixmaps, RGB pixmaps with optional alpha channel, or 256 grayscale colors, adding Adam7 interlacing as an option to allow displaying incomplete drafts of images before reading all the pixmap data in the stream. It also allows for some metadata such as text comments and internal timestamps, or even unknown chunks of (meta)data, allowing it to become a container format of sorts, most infamously abused by Mozilla Firefox to implement animated PNGs through a slightly modified version of the library — because lossless RGB pixmaps with alpha transparency are so cool compared to indexed pixmaps with binary opacity...

As cool as this sounds, libpng by itself is a trap for the unexperienced and/or impatient programmer, of which I fall into the latter category.

The API allows for many different applications, resulting in a convoluted interface and some poor or missing documentation for it. Reading the manual and examples included with it does not help a lot; when skimming through the header one can find some undocumented interfaces. It's a hell of flexibility that makes me feel lost in middle of a big forest without a map.

I foolishly wrote Wesnoth-TC without knowing any of this beforehand, and ended up creating a monster known as version 1.0, which is a mess of quickly crafted ugly code that breaks under big-endian platforms and leaves huge memory leaks when running on anything. I could have used a different library but I didn't have more documentation than libpng's at the time. Now that I'm preparing to release Wesnoth-TC 1.5.0 and have learned to keep away from this library, I'm turning my attention to wrapper libraries that should be hopefully much easier to work with if I just want to read and write ARGB streams from/to PNG files.

But Wesnoth-TC 1.5.0 will still use libpng, although I'll rewrite the code that uses it. Why? Because this time, it's personal.

LGMs, dragons and TuxOnIce

So, last Friday I finally got around to upgrade from Debian Lenny (Stable) to Squeeze (Testing), but not without facing some problems.

network-manager got restarted too early while I still needed to download more packages to finish the upgrade, I got stuck for 20 minutes trying to convince cnetworkmanager to make the wifi adapter work again until I figured out I just needed to restart D-Bus, the Linux kernel package v2.6.30-2 in Squeeze has some patches that rendered the wifi adapter completely useless after restarting, had to quickly rebuild a custom kernel to solve that, and work with Squeeze's udevd and get rid of the annoying default console bell (BEEP! — in a library, thrice), and then had to build another kernel enabling PAT support because the new X.org needs it for some reason or radeonhd gets unusably slow — phew.

It was much better than I expected and the migration from KDE 3.5.9/10 to 4.3.1 was relatively seamless; Opera 10.0 and VirtualBox 3.0.10 for Debian lenny still work fine after the upgrade, and KDE 4 didn't try to eat my /home dir, nor did I try to prevent that after all.

After a few hours of using the new desktop environment I started to like it. IMHO, rumors of KDE 4's suckiness are being greatly exaggerated now that it's at 4.3.x.

Of course not everything is perfect, and I hit a very bad bug in Dolphin, KDE 4's lightweight file manager, which turns it into a time-bomb of sorts, crashing roughly after 1 minute of use. However, that bug doesn't affect good old Konqueror and that gave me an opportunity to go "whoaaah" and "ahhhh" and "ohhhh" at the various user interface changes in this version; for example, the previews of directories are much better than on the KDE 3.5 version of Konqui — now we have actual previews embedded on the directory icons, rather than little icon overlays. IMHO such feature belongs in the realm of eye candy, but there's still a fair possibility of revealing your pr0n stash if you navigate the parent directory with this enabled. 😉

KDE 4.3's performance here is far better than what I experienced with a amd64 openSUSE-based LiveCD image using KDE 4.2.x. I couldn't know if this is because it's Debian, or because it's KDE 4.3, or because my custom kernel config rocks. 😛 Regardless, the laptop's battery still lasts almost the same as before upgrading to Squeeze — roughly 45 minutes. Of course, although I did enable some eye-candy to give radeonhd's XRender acceleration a decent work-out, I made sure that the new power management applet will turn it off when running on batteries. This is all very handy and beautiful...

But Squeeze lacks an important component for me, uswsusp, which provided some reliable work-arounds to get my laptop working fine both before and after suspending to disk (S4 power state). It is currently excluded from Squeeze due to some rather astonishing bugs, which means I had to seek other solution for suspending my pretty Linux system to disk...

And the solution in my case is the TuxOnIce patch, which surprisingly manages to bring the screen, keyboard and touchpad back to life — a feat that the kernel's built-in S4 support won't perform here. With this patch applied on top of an otherwise vanilla Linux kernel I now have a completely usable Squeeze system that supports all the hardware and software features I need.

Goodbye Lenny!

No, I don't actually know anybody named Lenny except the current Debian GNU/Linux stable distribution.

I switched from openSUSE 10.3 to Debian Lenny when it was the testing distribution back in... November 2008 IIRC. I also installed it on my HP Pavilion dv5-1132la laptop — which I received for Christmas the same year and still use as of this writing — but I continued tracking Lenny instead of the new testing distribution when Lenny became the new Stable on February 2009.

Defying all laws of Common Sense™ (again!), I'm now going to try to switch “back” to Debian testing — which is, this time, Squeeze.

I admit that I feel a lot of curiosity for trying KDE 4.3, but I also want to see how well are some other software packages doing: uswsusp, freetype, the GIMP, Iceweasel, VLC media player, and Boost, among others. I'm currently using a few packages that I built from their Sid sources with Lenny:

  • GIMP 2.6.6
  • Gutenprint 5.2.3 (story)
  • irssi 0.8.14

I'm also already using a few software packages or versions from their upstream providers:

  • Linux 2.6.31.6
  • DOSBox 0.73
  • qemu 0.11.0
  • VirtualBox 3.0.10 (story for both this and qemu)
  • QtCurve 0.68.1
  • radeonhd module for X.org - constantly updated from the upstream git repository's HEAD
  • kvirc 4.0.0 RC1 + svn (r3496)

I love how well Debian has worked for me till now — despite running across some really, really bad issues (my fault) compromising the installed system's integrity, I've never needed to reinstall the distribution and I had forgotten how often Windows users need to reinstall everything when a single piece breaks. Although I'm running a kernel version greater than the one Debian Lenny ships with I've had mostly no problems except a very random bug with Audacious that crashes it when playing MP3 audio files.

So why would I switch to Squeeze if I seem to be perfectly safe and comfortable with my current system configuration?

  • I want to check any new and shiny stuff included with newer versions of the Boost library, which is an essential dependency for the Listra and Thoria, tools that I'm writing for the Wesnoth-UMC-Dev project.
  • I want to try KDE 4.3 in a live environment.
  • The VLC media player version included in Lenny is a bit “outdated”, the Squeeze/Sid versions pull dependencies not available in Lenny, and the upstream package is also a bit difficult to compile since apt-get build-dep won't help me here.
  • Firefox/Iceweasel 3.5 is hopefully more stable than Firefox/Iceweasel 3.0 (it's not a problem specific to Debian's fork, really) so it could be an opportunity to recover my faith in Mozilla...
  • I want to try KDE 4.3 in a live environment.
  • The version of scons which ships with Lenny has some mildly annoying bugs.
  • Did I mention I want to try KDE 4.3?

I don't really expect to have much trouble upgrading, but I've backed up my /etc and /srv trees, some stuff from /var, and not to mount the partition for /home on the first reboot with Squeeze; I'll be using a minimal /home tree from the rootfs partition first. It helps that I'm using separate partitions for /var, /usr, /usr/local, /opt and /home since day zero. Maybe I'm being slightly paranoid — why would KDE 4.3 kill my home dir anyway? 😛

The smallest bugs are the nastiest

Preparing the release of a fixed demo revision for my Wesnoth add-on After the Storm, which is the immediate sequel to Invasion from the Unknown, I ran into a rather nasty bug in one of the scripts I made for easy creation of .tar.bz2 packages for the Wesnoth-UMC-Dev Project, build-external-archive.sh.

The script in question is one of my earliest experiments with bash scripting, and as such suffers from the quirks of a novice programmer trying to get work done fast. It's some sort of crapload of hacks that just ‘‘works’’, somehow. I have an odd habit of tempting fate and using my earliest programming experiments in production environments — and this includes Shikadibot and Soradoc, which is the PHP-based layout code generator used both here and at the Wesnoth-UMC-Dev website.

The only reasons for not killing it are the following features offered to the selected few who know how to use it:

  • It reliably produces a .tar.bz2 archive for add-on distributions on regular websites — including SF.net — provided a few simple arguments.
  • It roughly* conforms to the Wesnoth .ign file syntax, although it doesn't provide the same defaults as the game engine.
  • It knows how to cleanly handle SVN check-outs.
  • There's built-in, automatic support for XDelta patch generation, provided the xdelta tool is installed and in PATH.
  • Provides MD5 and SHA1 digests of produced packages and writes them to .MD5 and .SHA1 files for distribution.
  • No suitable replacement has been provided yet.

(* Espreon pointed out on the #wesnoth-umc-dev channel at chat.freenode.net, that `build-external-archive.sh` may match .ign patterns only in the add-on's root directory, and not in its subdirectories. This, of course, a bug.)

build-external-archive.sh has got a few inoffensive bugs in the past, perhaps the greatest of which was a set of dependencies upon Linux distribution-specific conventions (why, hello there, openSUSE!). However, yesterday I hit the nastiest, yet unseen bug.

Since I usually use actual SVN tag check-outs for creating the packages, or get someone else to do it for me, I had not checked the interactions of this script with an input directory that is actually a symbolic link pointing to a directory containing files or paths matched by a _server.ign file. This tool needs to copy the input directory to a separate temporary directory in order to physically remove the files (actually copies) that match the aforementioned .ign patterns, and then proceed to run tar to wrap the modified temp directory into a nifty bzip2 tarball.

And how was it copying the input directory to /tmp when the input is not a SVN check-out? Without going into much detail, it is similar to this:

cp -a ${input_directory} /tmp/${output_file.fakeroot}.dir/${addon_name}

The -a switch to GNU coreutils' cp is equivalent to -dpR, which roughly means ‘‘be recursive, preserve timestamps, access masks and never dereference symlinks and preserve them!’’ So you can start to imagine why this would become a huge problem when the $input_directory is a symlink.

Such a small overlooked mistake ended up destroying the git-svn tree contained within my add-on's directory, which was indeed specified as a symlink in ~/.wesnoth-1.7/data/add-ons/After_the_Storm pointing to ~/src/wesnoth-umc-dev/git/trunk/After_the_Storm. The symlink itself was copied to the temporary directory and the removal of unwanted files took place on the original location instead of a temporary tree. This gave me that disturbing feeling of having a bucket with cold water dropped on my head whenever I accidentally remove files I wasn't supposed to remove. Thankfully, I had committed all my changes to the upstream SVN repository and only had to regenerate a git-svn tree to continue working after making the release using a fallback SVN check-out.

The correct pseudo-code, forcing symlinks to be dereferenced and timestamps, file modes and ownerships to be preserved, is actually:

cp -LRp ${input_directory} /tmp/${output_file.fakeroot}.dir/${addon_name}

As you can see, it's perfectly easy to screw up your own files with a technically small mistake in a command line. However, in this case some idiot (me!) didn't read the manual correctly and didn't notice that the -a switch had additional effects, and that's a big logical mistake. Fortunately, only Espreon, AI0867 and I rely on this script, as far as I know.

This tool is going to be eventually replaced by umcdist (codenamed Blackwater), which is written in manageable Perl, provides the same functionality plus archive signing with OpenPGP, and would be done in a few days if I actually concentrated on it — don't take me wrong, it exists and there's code written for it, but it still needs to be completed. Whereas build-external-archive.sh is a deliberately undocumented IftU-specific tool that was later inherited by the wesnoth-umc-dev project, umcdist is targeted to the project as a whole, it will have proper online and built-in documentation and it'll be readable by more than one person to allow easier bug-fixing and maintenance.