Google Chrome and a conspiracy theory

I just found out that Chromium (browser) has been in Debian experimental and Sid for a while.

I'm currently tracking Squeeze and pulling some packages from Experimental, in particular Iceweasel 3.6, which feels much more stable to me than its counterpart in Testing, version 3.5 — which will probably have to remain in the upcoming Stable release as explained by one of the package maintainers.

(Granted, I'm a fool who doesn't care about security because I don't visit unknown odd sites at all. If it weren't for this, you'd say I should not be pulling packages from Experimental, but I am, fully understanding the risks!)

Despite I can see other packages from Experimental in my package manager, including a localization package for Chromium, I can't see Chromium itself, which is really odd. I have Google Chrome installed and I pull it from Google's repository because…because it added itself to apt's sources after I installed it for trying it out last year — which unfortunately reeks of Internet Explorer's old “integration” thing that started with IE 4, frankly. I mean, why didn't it even ask me about adding the source? Is it modifying other parts of my system's configuration without my consent? What the hell, Google?

Rant aside, this is a strange coincidence, which could be related to a mirroring issue in any case, but I don't rule out the possibility that Chrome is somehow banning Chromium from my package manager. Alternatively my laptop might be possessed by some evil spirit that wants me to leave Debian's free-as-in-freedom packages for evil “Big Brother” software suites. Uncanny?

*(For the Google lovers and haters in the audience: I'm perfectly fine with using Google stuff, mind you. My main email account is from Gmail, my preferred search engine is Google's, I also use Google Maps, Google Earth, and this memory/method call profiling suite of sorts that Sirp recommended to me. I also use Google Translate and reCAPTCHA. So, no, I'm not really bothered by Google Chrome's additions, but I'm really mildly pissed off at their decision to change my package manager's sources without asking me through debconf or something.)

ACPI DSDT patching again

My old patched ACPI DSDT didn't work well after adding 2 GB of RAM and caused a lot of “interesting” problems, so I built a kernel without a custom DSDT to be able to work safely without hitting a “bad page.”

Last night I fetched the system's DSDT again, patched and recompiled it, removing the rules that disallow non-Vista systems to get certain info from the thermal zone. So now I have a working 2.6.33.5 kernel that can access the ACPI thermal zone again.

shadowm@bluecore:~$ taintedness.pl
Tainted: G A
A (ACPI) - ACPI DSDT overridden.
shadowm@bluecore:~$

With the usual side-effect of tainting the kernel.

More RAM at last! (or, the side-effects of patching the ACPI DSDT)

Today I bought and installed another 2 GB of RAM from my HP Pavilion dv5-1132la laptop while on an errand to buy a new laptop for someone else — it ended up being a HP Pavilion dv4-something in case you are wondering.

This means that I can not only run Windows 2000, Windows XP, Windows 98, OpenSolaris 2009.06 and Debian Lenny on VirtualBox all at the same time now, but I'll also be able to compile Wesnoth faster now that I can take advantage of the dual-core AMD processor without running out of RAM and getting excess swapping to disk during the build!

It didn't work quite well at first, though. Problems occurred when I started enough processes to consume over 2 GB of RAM:

BUG: Bad page state in process VirtualBox pfn:6febe
page:ffffea000187b990 flags:4000000000800000 count:0 mapcount:0 mapping:(null) index:0
Pid: 4323, comm: VirtualBox Tainted: G A 2.6.33.4-bluecore263-preempt-suspend2 #1
Call Trace:
[<ffffffff81086a48>] ? bad_page+0x102/0x115
[<ffffffff810882a2>] ? get_page_from_freelist+0x3b0/0x517
[<ffffffff810884f6>] ? __alloc_pages_nodemask+0xed/0x588
[<ffffffff810884f6>] ? __alloc_pages_nodemask+0xed/0x588
[<ffffffff810a0b2a>] ? __vmalloc_area_node+0xea/0x10a
[<ffffffffa021d514>] ? rtR0MemObjLinuxAllocPages+0xd8/0x1cc [vboxdrv]
[<ffffffffa021d630>] ? rtR0MemObjLinuxAllocPhysSub2+0x28/0xde [vboxdrv]
[<ffffffffa022a7dd>] ? g_abExecMemory+0x1ddd/0x180000 [vboxdrv]
[<ffffffffa022ad78>] ? g_abExecMemory+0x2378/0x180000 [vboxdrv]
[<ffffffffa022d2b0>] ? g_abExecMemory+0x48b0/0x180000 [vboxdrv]
[<ffffffff8102cc1c>] ? cpuacct_charge+0x54/0x76
[<ffffffffa023b2f4>] ? g_abExecMemory+0x128f4/0x180000 [vboxdrv]
[<ffffffffa023d28f>] ? g_abExecMemory+0x1488f/0x180000 [vboxdrv]
[<ffffffffa023d7bb>] ? g_abExecMemory+0x14dbb/0x180000 [vboxdrv]
[<ffffffffa02316da>] ? g_abExecMemory+0x8cda/0x180000 [vboxdrv]
[<ffffffffa0218ded>] ? supdrvIOCtl+0x1241/0x20f8 [vboxdrv]
[<ffffffffa021ce22>] ? rtR0MemAlloc+0x90/0xb4 [vboxdrv]
[<ffffffffa02152a7>] ? VBoxDrvLinuxIOCtl+0x114/0x18e [vboxdrv]
[<ffffffff81028440>] ? pick_next_task_fair+0xac/0x112
[<ffffffff810ba422>] ? vfs_ioctl+0x23/0x93
[<ffffffff810ba92d>] ? do_vfs_ioctl+0x429/0x46d
[<ffffffff810aeff3>] ? fget_light+0xc3/0xe8
[<ffffffff810ba9ad>] ? sys_ioctl+0x3c/0x5c
[<ffffffff81001f2b>] ? system_call_fastpath+0x16/0x1b

While at first I thought it was a problem with the new RAM module itself (either that or damage on the old module, which is occupying the formerly free slot) I quickly suspected of the Linux kernel configuration instead because of several ACPI-related errors in the boot log.

reserve_ram_pages_type failed 0x6febe000-0x6febf000, track 0x10, req 0x10
ioremap reserve_memtype failed -16
ACPI Error: Could not map memory at 000000006FEBEE70, size 7 (20091214/exregion-180)
ACPI Exception: AE_NO_MEMORY, Returned by Handler for [SystemMemory] (20091214/evregion-475)
ACPI Error (psparse-0537): Method parse/execution failed [\_SB_.PCI0.TOM_] (Node ffff88013f8605d0), AE_NO_MEMORY
ACPI Error (psparse-0537): Method parse/execution failed [\_SB_.PCI0._CRS] (Node ffff88013f860430), AE_NO_MEMORY
ACPI Error (uteval-0250): Method execution failed [\_SB_.PCI0._CRS] (Node ffff88013f860430), AE_NO_MEMORY

After running some memory test utilities and getting no problem reports I recompiled another kernel (blessed be ccache, by the way!) without my patched DSDT which I needed for getting thermal zone readings with Linux. I might have mentioned before that this laptop has a special rule in the ACPI DSDT to not allow any operating system other than Windows Vista (even the particular version) to read the temperature status.

As I suspected, all the boot errors went away after rebooting to an unpatched/untainted kernel, and right now I'm using 3678 MB of 3707 MB (damned graphics controller) without hitting a “bad page.”

I'm not sure why the patched DSDT caused this little mess, but I'll check what happens if I repatch it now, using the current “original” DSDT.

On Wesnoth's grow rate

When I got my current HP laptop on December 2008, it was pretty nice to compile Wesnoth from scratch with 3 or 4 parallel compiler instances in less than 15 minutes, even when making -O3 builds.

Nowadays, it can take an hour to make an -O3 build with 1 single compiler instance, and parallel compiling is out of the question. In comparison, Wesnoth 1.0 can be compiled in 3 minutes and a half with 1 single instance, and much less time with -j 4.

Exactly what is wrong with this? Let's take a look at the relevant system stats and compilation settings:

  • CPU: AMD Athlon X2 Dual-Core QL-62 (2 GHz each core)
  • RAM: 2 GiB minus 256 MiB (onboard graphics)
  • OS: Debian GNU/Linux “Squeeze” (amd64) (GCC 4.4.2)
  • Kernel: Linux bluecore 2.6.33.2-bluecore261-preempt-suspend2-audit #1 SMP PREEMPT Mon Apr 12 21:38:39 CLT 2010 x86_64 GNU/Linux (Tux-On-Ice patch, optimizations for AMD K8, timer freq. 1000 hz., NO_HZ, PREEMPT)
  • Filesystem where .ccache and Wesnoth's source are: ext3 (sda6)
  • Filesystem for /tmp: XFS (sda9)<
  • CXXFLAGS: -pipe -mtune=native -march=native -O3 -Wl,--gc-sections,--relax
  • Build system: SCons (ccache=True, fast=True)

-O3 obviously involves extra CPU load at compile time because of the extra optimizations compared to, say, the default -O2. Nonetheless, I know well that it used not to take this long to build Wesnoth at the start of 2009. Wesnoth's code grew a lot over the course of the year due to the introduction of the new AI framework, the new lobby and more GUI2 development. Particularly, if I run scons with -j 2 or greater nowadays, I can run out of free RAM (making Linux page everything out to swap memory) during the compilation of the AI framework.

The most likely reason for this problem is the common use of C++ templates in that code. This feature is used a lot in GUI2 as well, but the code generated by template instantiation on every object file appears to be smaller.

While building stuff from scratch should be relatively uncommon for a developer like me who also has ccache installed and enabled, very minor changes in GUI2 or the AI from other developers currently trigger a near-total recompilation of Wesnoth. I have complained a lot about this in many opportunities, but it seems that I've hit a hard wall as a consequence of Wesnoth's evolution.

An -O3 executable of the main game target can be as large as 14 MiB. An -O0 debug build with symbols nears 400 MiB, and the directory with the resultant object files (including the executable) reaches 1.4 GiB of size.

Right now I'm considering adding more RAM for other reasons — namely, running Windows XP SP3 on VirtualBox can easily starve the host if certain apps like Iceweasel or Kate are running on the latter. I've yet to see if this will help me compile Wesnoth faster. It won't magically reduce the frequency of full rebuilds, though, and that's really discouraging me from hacking the mainline engine unless it's a one-shoot patch like the one involved in the fix for bug #15902.

The alternative is getting a quad-core machine or a distcc host. The first is least likely to happen. 😐

Wesnoth Evolution: An interview with Dave

As I mentioned in my Wesnoth Evolution: 0.1 article, I contacted David White (a.k.a. Dave/Sirp) to ask him a few questions about Wesnoth's early history — more specifically, about codename “strategy” and Wesnoth 0.1.

This is not an all-inclusive interview. You can find more information on Wesnoth's history in Wesnoth.org if you know how and what to search. Wesnoth Philosophy on the wiki (which includes The History of, and Philosophy behind Wesnoth) may be a good starting point.

Continue reading “Wesnoth Evolution: An interview with Dave

Wesnoth Evolution: 0.1

The Battle for Wesnoth has been around for some years already and its ever increasing content and artwork quality has attracted many code developers, sprite and portrait artists and musicians over time. I started playing Wesnoth on version 0.9.6 which was distributed with the SUSE Linux 10.0 operating system; I later switched to version 1.0.2 and closely followed the 1.1 release cycle.

Now that version 1.8 has been released and approx. 7 years have passed since the very first release, I felt curiosity to see exactly what has changed and what hasn't changed since the very first Wesnoth version published by David White (a.k.a. Sirp) that isn't even available in the downloads page at SourceForge.net. Yep, that's right, this is a review of Wesnoth 0.1 against current versions of Wesnoth such as 1.8 and 1.6, or even intermediate modern versions such as 1.0.

For historical reference I'm using both an actual Wesnoth 0.1 build for Windows I prepared using the Mingw32 cross-compiler environment for Linux (running on WINE), and the Battle for Wesnoth Project's SVN trunk changelog, which starts at version 0.2.1. The project's combined CVS-SVN history, on the other hand, starts on September 15th 2003, 15 days before the first CVS release tag, version 0.4.8. The first version announced since the beginning of the Wesnoth forums (which predate Wesnoth.org itself) is 0.3.7.

Continue reading “Wesnoth Evolution: 0.1

Wesnoth-UMC-Dev reaches another milestone!

Being just two years old, the Wesnoth-UMC-Dev fulfills its mission again and beetlenaut's Dead Water has just entered mainline (about 45 minutes ago)!

Since its inception on March 14th 2008, Wesnoth-UMC-Dev has hosted and served as a staging area for the following mainline campaigns:

  • Legend of Wesmere (entered mainline on October 7th 2008, 02:28 UTC)
  • Delfador's Memoirs (entered mainline on April 10th 2009, 02:08 UTC)
  • Dead Water (entered mainline on April 6th 2010, 17:02 UTC)

Wesnoth-TC 1.5.0

After being in Development Hell for...I forgot how many months, Wesnoth-TC, the advanced Team-Colorizer version 1.5.0 is finished and released.

This version introduces a few changes in the output files' naming scheme, and adds support for palette switches a la ~PAL() image function from Wesnoth's engine (which I implemented, by the way). The old Makefile is gone and there's a normal autotools build system in place, which seems to work well on Linux systems, as well as Mac OS X and even Mingw32-based cross compiling environments!

Thanks to this improvement and a small help from loonycyborg, Wesnoth-TC also has a experimental “official” Win32 build this time, which I have successfully tested on Windows XP SP3, and Debian Squeeze using WINE. It was cross-compiled using the Mingw32 environment packages from Debian. There are no important functionality differences between the Win32 build and normal Unix-based builds. It remains a command-line application as it's supposed to be.

In an ideal world, the availability of a CLI Win32 binary would raise interest on this tool and someone would volunteer for working with me on a GUI front-end. That's very unlikely to happen in the real world, so I'm researching Qt4 and messing around with some ideas for a cross-platform front-end based on penguin's Mac OS X applet.

Both distributions come with a README file, and the source code distro includes a INSTALL file with detailed instructions on configuring, compiling and installing wesnoth-tc. The Win32 binary distribution doesn't require any installation besides unpacking it into an appropriate directory — which you may optionally add to PATH.

EDIT: some people may have problems compiling under certain platforms or configurations because the configure script in this version enables strict mode by default, treating compiler warnings as errors. To solve this, pass the --disable-strict option to configure. Git master already solves this problem, plus a particular warning reported on Ubuntu 9.10.

As I mentioned before, I tried using gd for PNG manipulation but ultimately discovered that its limitations weren't nice for wesnoth-tc's particular case. I rewrote the PNG reading and writing code again afterwards, using the SDL_image library as a model instead.

The Win32 build doesn't have support for libpng exception handling (via longjmp/setjmp) because the compiler complains about it for some reason:

warning: variable ‘height’ might be clobbered by ‘longjmp’ or ‘vfork’

This shouldn't be a big deal since libpng won't complain unless someone feeds a corrupted or invalid PNG file to it. (EDIT: figured out the cause with some help, but haven't fixed it in Git master yet.) The other functionality change is the lack of support for checking file access permissions considering effective user and group ids, etc.

Supposedly, this release should work properly on PowerPC and other big-endian platforms. However, I don't have such a machine and therefore can't test.

It's hard to bid farewell

My own software projects tend to be very much like pets to me. I take care of them, carry them with me anywhere if it's possible, I feel horribly sad when something bad happens to them and, even when I go mad at them for something, in a few hours we are together like a neat happy family again.

Invasion from the Unknown has been the Wesnoth add-on project of mine since around September 10th 2007 and it has evolved throughout time and endured 3 mainline development cycles introducing drastic game engine changes, receiving little automated help from the likes of wmllint. Instead, it has been kept on shape by me and a few people who have helped with the huge maintenance burden that this epic-length campaign is.

With an initial goal of 30 playable scenarios, 29 of them were made at first and I later shrank the campaign to approx. 26 stages per suggestions from various people including ESR, and split it into two halves of roughly 13 scenarios each. IftU outlasted two other projects of mine which finally rotted among my hard disk backups for good — one after 5 years of development and little progress, and the other after three months of development and decent progress. It also directly or indirectly created or inspired the following Wesnoth-related projects:

Continue reading “It's hard to bid farewell

Original music in Wesnoth content

For a few years since ESR established the new mainline conventions in Wesnoth for tagging and adding new contributed music tracks, I've been in charge of helping with the process of importing tracks since ESR doesn't seem to be available for this anymore, and our “Lords” of Music never get familiarized with SVN to do it themselves. Normally I'm asked to do this whenever I'm available, but for some reason, another developer did it this time on March 13th — that is, two days ago — and I didn't find it out until today.

This minor thing reminded me of a substantial problem with campaign music in Wesnoth in regards to originality, which I've been perceiving for a few months already.

Continue reading “Original music in Wesnoth content