grahamc/hydra2 - Change YEXD7CBKPTMRCYQFNJFWXJUFOAKXNCVVYYJW3PHXQ5CX53Y5CUGQC

Fix findBuildDependencyInQueue

Previously this function didn't actually have a lot of effect. If a build A had a dependency B, Hydra would start B first. But on the next scan through the queue, it would start A anyway, because of the "busy => 0" restriction.

Now the queue runner won't start a build if a dependency is already running. (This is not necessarily optimal, since the build may have other dependencies that don't correspond to a build in the queue but could run. One day we'll start all Hydra builds in parallel…)

Also, for performance, use computeFSClosure instead of "nix-store -qR". And don't bother with topological sorting because it didn't have an effect anyway since the database returns dependencies in arbitrary order.

Created by Eelco Dolstra on May 3, 2013

YEXD7CBKPTMRCYQFNJFWXJUFOAKXNCVVYYJW3PHXQ5CX53Y5CUGQC

Dependencies

In channels

main

Change contents

Insertion in src/script/hydra-queue-runner at line 51 [3.7823]

[3.8398]

[4.10]

# Given a build, return an arbitrary queued build on which this build
# depends; or undef if no such build exists.

Replacement in src/script/hydra-queue-runner at line 55 [3.7823]

B:BD[4.65] → [4.65:163]

B:BD[4.163] → [5.0:1]

B:BD[5.1] → [6.2:39]

∅:D[5.23] → [4.168:200]

∅:D[6.39] → [4.168:200]

B:BD[4.168] → [4.168:200]

B:BD[4.200] → [6.40:205]

B:BD[6.205] → [2.10771:10907]

∅:D[7.7033] → [6.345:367]

∅:D[2.10907] → [6.345:367]

B:BD[6.345] → [6.345:367]

    my $drvpath = $build->drvpath;
    my @paths = reverse(split '\n', `nix-store -qR $drvpath`);
    my $depBuild;
    my @drvs = ();
    foreach my $path (@paths) {
        push @drvs, $path if $path =~ /\.drv$/ && $path ne $drvpath;
    }
    return unless scalar @drvs > 0;
    ($depBuild) = $db->resultset('Builds')->search(
        { drvpath => [ @drvs ], finished => 0, busy => 0, enabled => 1, disabled => 0 },
        { join => ['project'], rows => 1 } ) ;
    return $depBuild;

[4.65]

[4.565]

    my @deps = grep { /\.drv$/ && $_ ne $build->drvpath } computeFSClosure(0, 0, $build->drvpath);
    return unless scalar @deps > 0;
    return $db->resultset('Builds')->search(
        { drvpath => [ @deps ], finished => 0, enabled => 1, disabled => 0 },
        { join => ['project'], rows => 1 })->single;

Replacement in src/script/hydra-queue-runner at line 111 [3.7823]

B:BD[8.728] → [5.119:186]

B:BD[5.186] → [6.370:427]

                my $depbuild = findBuildDependencyInQueue($build);
                $build = $depbuild if defined $depbuild;

[8.728]

[4.644]

                # Find a dependency of $build that has no queued
                # dependencies itself.  This isn't strictly necessary,
                # but it ensures that Nix builds are done as part of
                # their corresponding Hydra builds, rather than as a
                # dependency of some other Hydra build.
                while (my $dep = findBuildDependencyInQueue($build)) {
                    $build = $dep;
                }
                next if $build->busy;