On Wesnoth's grow rate

When I got my current HP laptop on December 2008, it was pretty nice to compile Wesnoth from scratch with 3 or 4 parallel compiler instances in less than 15 minutes, even when making -O3 builds.

Nowadays, it can take an hour to make an -O3 build with 1 single compiler instance, and parallel compiling is out of the question. In comparison, Wesnoth 1.0 can be compiled in 3 minutes and a half with 1 single instance, and much less time with -j 4.

Exactly what is wrong with this? Let's take a look at the relevant system stats and compilation settings:

  • CPU: AMD Athlon X2 Dual-Core QL-62 (2 GHz each core)
  • RAM: 2 GiB minus 256 MiB (onboard graphics)
  • OS: Debian GNU/Linux “Squeeze” (amd64) (GCC 4.4.2)
  • Kernel: Linux bluecore 2.6.33.2-bluecore261-preempt-suspend2-audit #1 SMP PREEMPT Mon Apr 12 21:38:39 CLT 2010 x86_64 GNU/Linux (Tux-On-Ice patch, optimizations for AMD K8, timer freq. 1000 hz., NO_HZ, PREEMPT)
  • Filesystem where .ccache and Wesnoth's source are: ext3 (sda6)
  • Filesystem for /tmp: XFS (sda9)<
  • CXXFLAGS: -pipe -mtune=native -march=native -O3 -Wl,--gc-sections,--relax
  • Build system: SCons (ccache=True, fast=True)

-O3 obviously involves extra CPU load at compile time because of the extra optimizations compared to, say, the default -O2. Nonetheless, I know well that it used not to take this long to build Wesnoth at the start of 2009. Wesnoth's code grew a lot over the course of the year due to the introduction of the new AI framework, the new lobby and more GUI2 development. Particularly, if I run scons with -j 2 or greater nowadays, I can run out of free RAM (making Linux page everything out to swap memory) during the compilation of the AI framework.

The most likely reason for this problem is the common use of C++ templates in that code. This feature is used a lot in GUI2 as well, but the code generated by template instantiation on every object file appears to be smaller.

While building stuff from scratch should be relatively uncommon for a developer like me who also has ccache installed and enabled, very minor changes in GUI2 or the AI from other developers currently trigger a near-total recompilation of Wesnoth. I have complained a lot about this in many opportunities, but it seems that I've hit a hard wall as a consequence of Wesnoth's evolution.

An -O3 executable of the main game target can be as large as 14 MiB. An -O0 debug build with symbols nears 400 MiB, and the directory with the resultant object files (including the executable) reaches 1.4 GiB of size.

Right now I'm considering adding more RAM for other reasons — namely, running Windows XP SP3 on VirtualBox can easily starve the host if certain apps like Iceweasel or Kate are running on the latter. I've yet to see if this will help me compile Wesnoth faster. It won't magically reduce the frequency of full rebuilds, though, and that's really discouraging me from hacking the mainline engine unless it's a one-shoot patch like the one involved in the fix for bug #15902.

The alternative is getting a quad-core machine or a distcc host. The first is least likely to happen. 😐

Hardware, Miscellaneous, Software, Wesnoth