How to quickly navigate an unfamiliar makefile

The other day, I was working with an unfamiliar build and I needed to get familiar with it in a hurry. In this case, I was dealing with a makefile generated by the Perl utility h2xs, but the trick I’ll show you here works any time you need to find your way around a new build system, whether it’s something you just downloaded or an internal project you just transferred to.

What I wanted to do was add a few object files to the link command. Here’s the build log, with the link command highlighted:

gcc -c  -I. -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -g   -DVERSION=\"0.01\" -DXS_VERSION=\"0.01\" -fPIC "-I/usr/lib/perl/5.10/CORE"   mylib.c
rm -f blib/arch/auto/mylib/
gcc  -shared -O2 -g -L/usr/local/lib mylib.o   -o blib/arch/auto/mylib/   \
chmod 755 blib/arch/auto/mylib/

Should be easy, right? I just needed to find that command in the makefile and make my changes. Wrong. Read on to see how annotation helped solve this problem.

Read the rest of this entry »

What is SparkBuild?

At the 2009 Electric Cloud Customer Summit we introduced SparkBuild, a free gmake- and NMAKE-compatible build tool. SparkBuild is now in public beta, and several people have asked us for some more explanation: what is SparkBuild and why should I care? I thought I’d take a crack at answering those questions, hopefully without sounding too “marketingy”. Here goes.

SparkBuild is actually a package containing two components: SparkBuild emake and SparkBuild Insight. As the names imply, these components are derived from the corresponding pieces of ElectricAccelerator and ElectricInsight. That means that they offer some of the same benefits that our commercial product does, but for free, of course. Why did we do this? I’ll be completely honest with you: we’re hoping that people will use SparkBuild, share SparkBuild and talk about SparkBuild, ultimately raising awareness of Electric Cloud and our commerical products. Beyond that though, I’m personally excited about SparkBuild because I want to see these technologies that I’ve worked on for so long get used by as many people as possible.

I think you’ll be interested in SparkBuild because it offers some of the same benefits that our commercial tools do (annotation and build analysis), and even some that aren’t yet part of Accelerator (subbuilds). Read on to learn more about what these features provide and how you can use them.
Read the rest of this entry »

Subbuilds: build avoidance done right

I’ve heard it said that the best programmer is a lazy programmer. I’ve always taken that to mean that the best programmers avoid unnecessary work, by working smarter and not harder; and that they focus on building only those features that are really required now, not allowing speculative work to distract them.

I wouldn’t presume to call myself a great programmer, but I definitely hate doing unnecessary work. That’s why the concept of build avoidance is so intriguing. If you’ve spent any time on the build speed problem, you’ve probably come across this term. Unfortunately it’s been conflated with the single technique implemented by tools like ccache and ClearCase winkins. I say “unfortunate” for two reasons: first, those tools don’t really work all that well, at least not for individual developers; and second, the technique they employ is not really build avoidance at all, but rather object reuse. But by co-opting the term build avoidance and associating it with such lackluster results, many people have become dismissive of build avoidance.

Subbuilds are a more literal, and more effective, approach to build avoidance: reduce build time by building only the stuff required for your active component. Don’t waste time building the stuff that’s not related to what you’re working on now. It seems so obvious I’m almost embarrassed to be explaining it. But the payoff is anything but embarrassing. On my project, after making changes to one of the prerequisites libraries for the application I’m working on, a regular incremental takes 10 minutes; a subbuild incremental takes just 77 seconds:

Standard incremental:
Subbuild incremental:

Not bad! Read on for more about how subbuilds work and how you can get SparkBuild, a free gmake- and NMAKE-compatible build tool, so you can try subbuilds yourself.
Read the rest of this entry »