Is it over already?

Finally, it's January! The New Year celebrations are mostly over and fading away, and people all around the world are going back to regular business and everything should be back to normal in a few days.

I used to be fond of the Christmas and New Year celebrations as a child as I could spend time with my family and eat delicious food. That is not the case anymore, because, even if I still live with my parents, there's no longer a sense of family here and we only want to throw sharp stuff at each other. There's not much enthusiasm by the end of the year anymore, and phrases such as “Merry Christmas” and “Happy New Year” (in Spanish, though) are truly unheard of in this house. Recent disagreements amongst us indicate that this is not going to be a good year for anyone. To add insult to injury, one of our cats died in a rather tragic and violent fashion on December 22th — it's a tradition here that one or more pets must always die in December. While we have many of them, the first ones to die are those whom we are most attached to.

To mark the actual start of 2010 (as far as the Gregorian calendar is involved, of course), there was a black-out on the area about 6 minutes 7 seconds past midnight, which left us with no Internet or tap water until around 1:50 AM. What a great way to start the first day of the year.

But there's still some hope at the moment. Some days before Xmas, my creativity returned from its long, chaotic journey and my Wesnoth add-on, After the Storm (sequel to Invasion from the Unknown has seen steady progress and two new releases were published in less than two weeks. Keep in mind that this add-on had not seen any public releases for almost a year.

After the last released version of AtS (0.2.1) including 5 of 12 planned scenarios in Episode I, there has been more progress in the Wesnoth-UMC-Dev repository. Just yesterday, I finished the two-part cutscene that is the sixth scenario of Episode I, one of the most important points of the plot's development, in which two forest elves finally make contact with the desert/Quenoth elves.

I won't be able to release AtS 0.2.2 or 0.3.0 until scenario 7 and the next cutscene (appropriately named “Resolutions”) are finished, since I'd be teasing the players otherwise. However, those who are really interested on it can always check AtS out from the repository's trunk into their <wesnoth preferences dir>/data/add-ons dir and play using the latest development version of Wesnoth:

svn co https://wesnoth-umc-dev.svn.sourceforge.net/svnroot/wesnoth-umc-dev/trunk/After_the_Storm

It's really exciting to work with several plot elements from quartex's Under the Burning Suns in new, creative manners — kind of like Fanfic production taken to a new level using the power of the GNU General Public License (version 2 or later!). Nevertheless, I am fairly sure he deliberately left much details unresolved in the original campaign, and that he'd fry us (Espreon, AI0867 and me) alive if he found out what we are doing with his story.

One week before Xmas, the Wesnoth.org forums saw another upgrade on which Turuk and I worked hard and quickly to improve forum usability by not only upgrading the codebase to phpBB 3.0.6, but also tweaking the templates, adding modifications and a couple of new forum styles to take advantage of the new features implemented by the phpBB devs in this iteration of their software. The main points were highlighted in this forum post (originally a Global Announcement).

This year should also bring us a new stable series (1.8) of the Battle for Wesnoth game itself. There are currently some problems delaying the first Release Candidate and getting us flooded with generic beta releases, but the developers in charge of them will (hopefully!) find a solution so we can get 1.8 released and trunk “thawed” soon, to work on new features and allow new code contributors to join the project. As for me, I can't wait for the new stable series — development series players seem to be scarce and the new versions of IftU and AtS are receiving little feedback on the forums because of this! I suppose Multiplayer content authors are similarly eager to get more fresh meat to play their add-ons.

I also recently talked about how I can finally suspend my laptop to RAM using Linux, and run some basic OpenGL-based software without crashing or destroying anything. That's something I didn't expect to be able to do in the near future, so the Mesa, libdrm and X.org radeon driver developers have my thanks for improving the Linux experience of those unfortunate enough to own onboard ATI graphics controllers!

In summary, as usual, a new year brings good and bad news. I guess it's up to us to take what's good and fix what's wrong. So, anyway (although I guess it's pretty much unnecessary at this point): happy New Year and have fun!

The smallest bugs are the nastiest

Preparing the release of a fixed demo revision for my Wesnoth add-on After the Storm, which is the immediate sequel to Invasion from the Unknown, I ran into a rather nasty bug in one of the scripts I made for easy creation of .tar.bz2 packages for the Wesnoth-UMC-Dev Project, build-external-archive.sh.

The script in question is one of my earliest experiments with bash scripting, and as such suffers from the quirks of a novice programmer trying to get work done fast. It's some sort of crapload of hacks that just ‘‘works’’, somehow. I have an odd habit of tempting fate and using my earliest programming experiments in production environments — and this includes Shikadibot and Soradoc, which is the PHP-based layout code generator used both here and at the Wesnoth-UMC-Dev website.

The only reasons for not killing it are the following features offered to the selected few who know how to use it:

  • It reliably produces a .tar.bz2 archive for add-on distributions on regular websites — including SF.net — provided a few simple arguments.
  • It roughly* conforms to the Wesnoth .ign file syntax, although it doesn't provide the same defaults as the game engine.
  • It knows how to cleanly handle SVN check-outs.
  • There's built-in, automatic support for XDelta patch generation, provided the xdelta tool is installed and in PATH.
  • Provides MD5 and SHA1 digests of produced packages and writes them to .MD5 and .SHA1 files for distribution.
  • No suitable replacement has been provided yet.

(* Espreon pointed out on the #wesnoth-umc-dev channel at chat.freenode.net, that `build-external-archive.sh` may match .ign patterns only in the add-on's root directory, and not in its subdirectories. This, of course, a bug.)

build-external-archive.sh has got a few inoffensive bugs in the past, perhaps the greatest of which was a set of dependencies upon Linux distribution-specific conventions (why, hello there, openSUSE!). However, yesterday I hit the nastiest, yet unseen bug.

Since I usually use actual SVN tag check-outs for creating the packages, or get someone else to do it for me, I had not checked the interactions of this script with an input directory that is actually a symbolic link pointing to a directory containing files or paths matched by a _server.ign file. This tool needs to copy the input directory to a separate temporary directory in order to physically remove the files (actually copies) that match the aforementioned .ign patterns, and then proceed to run tar to wrap the modified temp directory into a nifty bzip2 tarball.

And how was it copying the input directory to /tmp when the input is not a SVN check-out? Without going into much detail, it is similar to this:

cp -a ${input_directory} /tmp/${output_file.fakeroot}.dir/${addon_name}

The -a switch to GNU coreutils' cp is equivalent to -dpR, which roughly means ‘‘be recursive, preserve timestamps, access masks and never dereference symlinks and preserve them!’’ So you can start to imagine why this would become a huge problem when the $input_directory is a symlink.

Such a small overlooked mistake ended up destroying the git-svn tree contained within my add-on's directory, which was indeed specified as a symlink in ~/.wesnoth-1.7/data/add-ons/After_the_Storm pointing to ~/src/wesnoth-umc-dev/git/trunk/After_the_Storm. The symlink itself was copied to the temporary directory and the removal of unwanted files took place on the original location instead of a temporary tree. This gave me that disturbing feeling of having a bucket with cold water dropped on my head whenever I accidentally remove files I wasn't supposed to remove. Thankfully, I had committed all my changes to the upstream SVN repository and only had to regenerate a git-svn tree to continue working after making the release using a fallback SVN check-out.

The correct pseudo-code, forcing symlinks to be dereferenced and timestamps, file modes and ownerships to be preserved, is actually:

cp -LRp ${input_directory} /tmp/${output_file.fakeroot}.dir/${addon_name}

As you can see, it's perfectly easy to screw up your own files with a technically small mistake in a command line. However, in this case some idiot (me!) didn't read the manual correctly and didn't notice that the -a switch had additional effects, and that's a big logical mistake. Fortunately, only Espreon, AI0867 and I rely on this script, as far as I know.

This tool is going to be eventually replaced by umcdist (codenamed Blackwater), which is written in manageable Perl, provides the same functionality plus archive signing with OpenPGP, and would be done in a few days if I actually concentrated on it — don't take me wrong, it exists and there's code written for it, but it still needs to be completed. Whereas build-external-archive.sh is a deliberately undocumented IftU-specific tool that was later inherited by the wesnoth-umc-dev project, umcdist is targeted to the project as a whole, it will have proper online and built-in documentation and it'll be readable by more than one person to allow easier bug-fixing and maintenance.

Half-assed commits

During my work on the Coordinated Wesnoth User-Made Content Development Project (which we dub "wesnoth-umc-dev" for short), I came up with an interesting concept related to Subversion's standard workflow. Half-assed commits are revision commits to the Subversion repository that are not completed due to the subversion client (or server!) process dying unexpectedly, usually due to anything but a SIGTERM.

The obvious symptom of a half-assed commit in your local file system is a bunch of 'L' flags in the svn st command output. These can be removed with svn cleanup. So, most half-assed commits are harmless to you. However, according to the (holy) Subversion Book, it may leave garbage, half-assed transactions in the repository. These are not viewable to anyone but the repository admin of course, and should not harm anyone provided the filesystem on which it resides does not run out of space.

😐 Last afternoon I ran into a more harmful and painful sort of half-assed commit. I renamed some files in my working copy, invoked svn ci, and my crappy Wireless LAN connection burped just when it was about to update the working copy with the changes introduced to the repository:

Transmitting file data ...svn: Commit failed (details follow):
svn: MERGE request failed on '/svnroot/wesnoth-umc-dev/trunk/Invasion_from_the_Unknown'
svn: MERGE of '/svnroot/wesnoth-umc-dev/trunk/Invasion_from_the_Unknown': Could not read status line: Connection reset by peer
(https://wesnoth-umc-dev.svn.sourceforge.net)
svn: Your commit message was left in a temporary file:
svn: '/home/shadowm/src/wesnoth-umc-dev/trunk/Invasion_from_the_Unknown/svn-commit.2.tmp'

Unsurprisingly, I was left with my files in an awful state that caused local conflicts with the repository. That is, next svn update failed because the commit above was successful for the server, leaving the renamed files in the repository. SVN just didn't like that at my end, because I had those renamed files already in the working copy as result of the svn move result I just (half-ass) commited.

Thanks for nothing SVN! Seriously, the protocol should have the server request for a final confirmation from the client to check-in the transaction after its changes have been merged in the client's working copy. Or the inverse: have the client react in a smarter fashion to these situations that people like me often run into.