After almost two days of wandering around the streets of the city, my emissaries located a place to buy a new AC adapter for my much beloved HP Pavilion “ATI hellspawn” dv5-1132la, and I have thusly regained access to my development environment for Wesnoth and related projects.
48 hours of using a laptop with a broken display, short-lived (8 minutes) battery, unusable touchpad buttons, different keyboard layout and outdated user config can be very frustrating, but it was a good exercise nevertheless. It's better to have a broken spare laptop running Linux (Debian Lenny before Stable) than no spare laptop or no Linux laptop at all. 😉
My laptop's AC adapter has finally died after passing out in three opportunities. I've already sent my emissaries across the city to find a spare for this thing.
For now, I'm using my old, broken Acer laptop and I won't be available for most of the time until I can use my HP Pavilion dv5-1132la again.
When I got my current HP laptop on December 2008, it was pretty nice to compile Wesnoth from scratch with 3 or 4 parallel compiler instances in less than 15 minutes, even when making -O3 builds.
Nowadays, it can take an hour to make an -O3 build with 1 single compiler instance, and parallel compiling is out of the question. In comparison, Wesnoth 1.0 can be compiled in 3 minutes and a half with 1 single instance, and much less time with -j 4.
Exactly what is wrong with this? Let's take a look at the relevant system stats and compilation settings:
CPU: AMD Athlon X2 Dual-Core QL-62 (2 GHz each core)
-O3 obviously involves extra CPU load at compile time because of the extra optimizations compared to, say, the default -O2. Nonetheless, I know well that it used not to take this long to build Wesnoth at the start of 2009. Wesnoth's code grew a lot over the course of the year due to the introduction of the new AI framework, the new lobby and more GUI2 development. Particularly, if I run scons with -j 2 or greater nowadays, I can run out of free RAM (making Linux page everything out to swap memory) during the compilation of the AI framework.
The most likely reason for this problem is the common use of C++ templates in that code. This feature is used a lot in GUI2 as well, but the code generated by template instantiation on every object file appears to be smaller.
While building stuff from scratch should be relatively uncommon for a developer like me who also has ccache installed and enabled, very minor changes in GUI2 or the AI from other developers currently trigger a near-total recompilation of Wesnoth. I have complained a lot about this in many opportunities, but it seems that I've hit a hard wall as a consequence of Wesnoth's evolution.
An -O3 executable of the main game target can be as large as 14 MiB. An -O0 debug build with symbols nears 400 MiB, and the directory with the resultant object files (including the executable) reaches 1.4 GiB of size.
Right now I'm considering adding more RAM for other reasons — namely, running Windows XP SP3 on VirtualBox can easily starve the host if certain apps like Iceweasel or Kate are running on the latter. I've yet to see if this will help me compile Wesnoth faster. It won't magically reduce the frequency of full rebuilds, though, and that's really discouraging me from hacking the mainline engine unless it's a one-shoot patch like the one involved in the fix for bug #15902.
The alternative is getting a quad-core machine or a distcc host. The first is least likely to happen. 😐
As I mentioned in my Wesnoth Evolution: 0.1 article, I contacted David White (a.k.a. Dave/Sirp) to ask him a few questions about Wesnoth's early history — more specifically, about codename “strategy” and Wesnoth 0.1.
The Battle for Wesnoth has been around for some years already and its ever increasing content and artwork quality has attracted many code developers, sprite and portrait artists and musicians over time. I started playing Wesnoth on version 0.9.6 which was distributed with the SUSE Linux 10.0 operating system; I later switched to version 1.0.2 and closely followed the 1.1 release cycle.
Now that version 1.8 has been released and approx. 7 years have passed since the very first release, I felt curiosity to see exactly what has changed and what hasn't changed since the very first Wesnoth version published by David White (a.k.a. Sirp) that isn't even available in the downloads page at SourceForge.net. Yep, that's right, this is a review of Wesnoth 0.1 against current versions of Wesnoth such as 1.8 and 1.6, or even intermediate modern versions such as 1.0.
For historical reference I'm using both an actual Wesnoth 0.1 build for Windows I prepared using the Mingw32 cross-compiler environment for Linux (running on WINE), and the Battle for Wesnoth Project's SVN trunk changelog, which starts at version 0.2.1. The project's combined CVS-SVN history, on the other hand, starts on September 15th 2003, 15 days before the first CVS release tag, version 0.4.8. The first version announced since the beginning of the Wesnoth forums (which predate Wesnoth.org itself) is 0.3.7.
After being in Development Hell for...I forgot how many months, Wesnoth-TC, the advanced Team-Colorizer version 1.5.0 is finished and released.
This version introduces a few changes in the output files' naming scheme, and adds support for palette switches a la ~PAL() image function from Wesnoth's engine (which I implemented, by the way). The old Makefile is gone and there's a normal autotools build system in place, which seems to work well on Linux systems, as well as Mac OS X and even Mingw32-based cross compiling environments!
Thanks to this improvement and a small help from loonycyborg, Wesnoth-TC also has a experimental “official” Win32 build this time, which I have successfully tested on Windows XP SP3, and Debian Squeeze using WINE. It was cross-compiled using the Mingw32 environment packages from Debian. There are no important functionality differences between the Win32 build and normal Unix-based builds. It remains a command-line application as it's supposed to be.
Windows (Win32) binary:wesnoth-tc-1.5.0-win32.zip (373.3 KiB)
(SHA1: 311adf05b3855bf74eb782fe020992e61aac80eb)
In an ideal world, the availability of a CLI Win32 binary would raise interest on this tool and someone would volunteer for working with me on a GUI front-end. That's very unlikely to happen in the real world, so I'm researching Qt4 and messing around with some ideas for a cross-platform front-end based on penguin's Mac OS X applet.
Instructions
Both distributions come with a README file, and the source code distro includes a INSTALL file with detailed instructions on configuring, compiling and installing wesnoth-tc. The Win32 binary distribution doesn't require any installation besides unpacking it into an appropriate directory — which you may optionally add to PATH.
EDIT: some people may have problems compiling under certain platforms or configurations because the configure script in this version enables strict mode by default, treating compiler warnings as errors. To solve this, pass the --disable-strict option to configure. Git master already solves this problem, plus a particular warning reported on Ubuntu 9.10.
Behind the scenes
As I mentioned before, I tried using gd for PNG manipulation but ultimately discovered that its limitations weren't nice for wesnoth-tc's particular case. I rewrote the PNG reading and writing code again afterwards, using the SDL_image library as a model instead.
The Win32 build doesn't have support for libpng exception handling (via longjmp/setjmp) because the compiler complains about it for some reason:
warning: variable ‘height’ might be clobbered by ‘longjmp’ or ‘vfork’
This shouldn't be a big deal since libpng won't complain unless someone feeds a corrupted or invalid PNG file to it. (EDIT: figured out the cause with some help, but haven't fixed it in Git master yet.) The other functionality change is the lack of support for checking file access permissions considering effective user and group ids, etc.
Supposedly, this release should work properly on PowerPC and other big-endian platforms. However, I don't have such a machine and therefore can't test.
My own software projects tend to be very much like pets to me. I take care of them, carry them with me anywhere if it's possible, I feel horribly sad when something bad happens to them and, even when I go mad at them for something, in a few hours we are together like a neat happy family again.
Invasion from the Unknown has been theWesnoth add-on project of mine since around September 10th 2007 and it has evolved throughout time and endured 3 mainline development cycles introducing drastic game engine changes, receiving little automated help from the likes of wmllint. Instead, it has been kept on shape by me and a few people who have helped with the huge maintenance burden that this epic-length campaign is.
With an initial goal of 30 playable scenarios, 29 of them were made at first and I later shrank the campaign to approx. 26 stages per suggestions from various people including ESR, and split it into two halves of roughly 13 scenarios each. IftU outlasted two other projects of mine which finally rotted among my hard disk backups for good — one after 5 years of development and little progress, and the other after three months of development and decent progress. It also directly or indirectly created or inspired the following Wesnoth-related projects:
For a few years since ESR established the new mainline conventions in Wesnoth for tagging and adding new contributed music tracks, I've been in charge of helping with the process of importing tracks since ESR doesn't seem to be available for this anymore, and our “Lords” of Music never get familiarized with SVN to do it themselves. Normally I'm asked to do this whenever I'm available, but for some reason, another developer did it this time on March 13th — that is, two days ago — and I didn't find it out until today.
This minor thing reminded me of a substantial problem with campaign music in Wesnoth in regards to originality, which I've been perceiving for a few months already.
Since I'm fed up with libpng's complexity (which borders on obscurity), I asked for recommendations on libraries for reading, editing and writing PNG files in #defocus, freenode's social channel. Someone mentioned gd, which I mostly knew for being a dependency of PHP-based software that deals with graphics, so I decided to give it a try.
It turns out to be incredibly easy to learn and use — I quickly started the gd_writer branch for the Wesnoth Team Colorizer tool and nuked hundreds of convoluted lines of libpng calls to replace them with short, concise and clean code.
However, I then noticed something in the (thorough and easy to read) documentation that spelled trouble for my usage:
gd retains only 8 bits of resolution for each of the red, green and blue channels, and only 7 bits of resolution for the alpha channel. The former restriction affects only a handful of very rare 48-bit color and 16-bit grayscale PNG images. The second restriction affects all semitransparent PNG images, but the difference is essentially invisible to the eye. 7 bits of alpha channel resolution is, in practice, quite a lot.
No justification is provided in the documentation as far as I could find. However, the main header file (gd.h, which is perfectly readable unlike libpng's png.h) has this precious missing bit explaining the reasoning here:
If 'truecolor' [in a gdImage struct] is set true, the image is truecolor; pixels are represented by integers, which must be 32 bits wide or more.
True colors are repsented (sic) as follows:
ARGB
Where 'A' (alpha channel) occupies only the LOWER 7 BITS of the MSB. This very small loss of alpha channel resolution allows gd 2.x to keep backwards compatibility by allowing signed integers to be used to represent colors, and negative numbers to represent special cases, just as in gd 1.x.
Ah, good old backwards compatibility, always biting the programmer in the ass after a while — Windows 9x being a major commercial example of backwards compatibility gone mad.
While I agree that the difference might not be noticeable for the human eye, this still means that wesnoth-tc/gd produces output that differs from the original images in more than just team colors. It'd be often silly to use team-colored semi-transparent in Wesnoth unit sprites, but team color and palette switches can, thanks to the flexibility of the image path functions mechanism, be applied on virtually any kind of image that SDL_image can read into a surface pixmap, including things such as transparent haloes for visual effects.
Experimenting with wesnoth-tc/gd, a linear sequence of pixels with these alpha channel values:
255, 254, 253, 252, 251, 250, 249, 258 [...]
...turns into this in the application's output:
255, 255, 253, 253, 251, 251, 249, 249 [...]
What happens here is that values below 0x7F — which is 127 in base 10 and 1111111 in base 2, and as you can see, the maximum positive integer that can be represented with 7 bits — can only be even, and values above are odd. There's no 0x7F.
I could just ignore this issue and merge the gd_writer branch, but no, I think I'll just try yet another library. It's a pity because this experiment had created the possibility of making a web interface for recoloring/team-coloring Wesnoth artwork, but I believe this kind of loss of information isn't any good for our purposes, even if it should be virtually unnoticeable at the standard 72x72 sprite scale. At least unit shadows shouldn't be affected by this limitation since their alpha value is supposed to be 153, which would remain unchanged under this scheme.
And so, Wesnoth-TC 1.5 stays in Development Hell for now, and the search for a simple enough libpng wrapper continues.