The last word on SCons performance

My previous look at SCons performance compared SCons and gmake on a variety of build scenarios — full, incremental, and clean. A few people suggested that I try the tips given on the SCons ‘GoFastButton’ wiki page, which are said to significantly improve SCons performance (at the cost of some accuracy, of course). Naturally, I felt that I had to do one last follow-up exploring this avenue. And since that meant I would already be running a bunch of builds, I figured I’d try out SCons’ parallel build features too. My findings follow.

Can SCons “GoFast”?

You can read all about the setup in the previous post, so I’ll just jump straight to the results. According to the “GoFastButton” recommendations, –max-drift=1 –implicit-deps-unchanged will “run your build as fast as possible”, so that’s what I used. In all cases, I did an initial, untimed from-scratch build first, to generate the initial dependency graph, then I ran a second timed build. After that timed run, I ran a clean build and finally ran the full build again, this time with -j 2, to evaluate the impact of SCons parallel build features.

Contrary to my expectations, the GoFast settings had relatively little impact over much of the test range — only about 5-10% faster than without those flags. Only the very largest build showed any significant impact, with a 25% improvement. Unfortunately that impressive result is more likely because SCons uses less memory with GoFast settings enabled. If you recall from the previous tests, with 50,000 source files, SCons’ memory footprint was a hefty 2,023 MB — enough to force my test machine to start swapping. With the GoFast settings, SCons used “only” 1,838 MB — still an awful lot of memory, but just smaller enough to avoid thrashing the system, with the end result being a substantially improved build time.

Building in parallel had a more substantial impact — reducing build times about 30% on the largest build (compared to a serial SCons build with GoFast settings enabled). That’s not as good as I had hoped for (on a large, relatively “flat” build such as this, I expected the build to parallelize very well), but it’s not terrible. Here are the complete results:

SCons full build performance, click for full size

So, GoFast seems to be a bust on full builds. It’s definitely better than vanilla SCons, but still nowhere near as fast as gmake.

Things look a little better on “one-touch” incremental builds though, where GoFast settings cut build times by about 30% across the board:

SCons incremental build performance, click for full size

Ironically, the most impressive results are on clean builds (scons -c). GoFast settings cut build times by about 40% at the low end of the test range, and by more than 50% at the high end of the test range:

SCons clean build performance, click for full size

To my amazement, SCons with GoFast settings actually beats gmake on clean builds. My guess is that this is probably because SCons handles file deletion in-process, while gmake must invoke a separate process (rm).

That’s all folks!

That’s it for my analysis of SCons performance. Thanks to everybody who offered ideas for improving my benchmarks! I hope you found this series of posts interesting.

9 Responses to “The last word on SCons performance”

  1. Johan Holmberg Says:

    In your earlier post about SCons you talked about publishing your generator script for the benchmark. How is that going?

  2. Parallelize it Says:

    Try make -j or make -j 4

    -j [jobs], –jobs[=jobs]
    Specifies the number of jobs (commands) to run simultaneously. If there is more than
    one -j option, the last one is effective. If the -j option is given without an argu‐
    ment, make will not limit the number of jobs that can run simultaneously.

  3. Eric Melski Says:

    @Johan: yep, just took a little longer to get the OK from management. I put the script up here. The script just generates a build tree in a subdir of your working directory called “sconsbld”. I invoked it as “./ -d N -l 2 -f 500”, where N varied from 3 to 99, indicating the number of subdirectories of “sconsbld”.

  4. Eric Melski Says:

    @Parallelize it: I had run the gmake tests with -j 2 also, but I didn’t include that data because I thought it would be a distraction in this article. Anyway, it cut the gmake build times between 35% and 40%, depending on the size of the build. Here’s the graph:

  5. Tweets that mention The last word on SCons performance « The Electric Cloud Blog -- Says:

    […] This post was mentioned on Twitter by Tech news (BOT) and Proggit Articles, Eric Melski. Eric Melski said: The last word on SCons performance: […]

  6. Matt Doar Says:

    Interesting work.

  7. Eric Melski Says:

    @Matt: Thanks for commenting!

  8. Top posts for September 2010 « Says:

    […] The last word on SCons performance: 13% of page views […]

  9. Make syntax is the worst… except for all the alternatives « Says:

    […] series of comparisons between SCons and GNU make sparked a lot of discussion, not just about SCons and gmake, but about many other build tools. That […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: