what are you thinking about the following features

[Joe Marcus Clarke]

On Sun, 2010-01-10 at 22:46 +0100, olli hauer wrote:
> Joe Marcus Clarke wrote:
> > On Tue, 2010-01-05 at 17:54 +0100, olli hauer wrote:
> >> Hi,
> >>
> >> what are you thinking about the following features.
> >>
> >> For example you inject a big amount of ports to the build queue
> >> because you don't want to fiddle out all the depending ports or
> >> need the port even not only as dependency (list for ports build
> >> on a remote machine with 'pkg_info -qoa')
> >>
> >> To make it more readable I will use addPort for the following statement.
> >> ./tc addBuildPortsQueueEntry -b ...
> >>
> >>
> >> addPort archivers/php5-zlib
> >> ...
> >> addPort databases/php5-pgsql
> >> addPort databases/postgresql84-client
> >> addPort databases/postgresql84-server
> >> ...
> >> addPort graphics/php5-gd
> >> addPort lang/php5
> >> ...
> >> addPort ports-mgmt/tinderbox
> >> ...
> >> addPort www/php5-session
> >>
> >>
> >> Now php5 is a dependency for php5-zlib but this port will build php5
> >> before, now tinderbox knows about both it build first php5 than
> >> php5-zlib and twenty ports later again php5.
> > 
> > I don't understand what you're asking.  Tinderbox uses make(1), so it
> > will obey's make's rules on when to rebuild a target.  That is, if
> > dependency A of port X is updated, then X must be rebuilt.  If
> > dependency A has not been changed, and X has not been changed, then
> > neither A or X will be rebuilt.
> > 
> >> Since tinderbox knows this is in the same build it could update the
> >> status off php5 to the last BuildDate and the status to 'SUCCESS' to
> >> prevent a second build.
> 
> given someone add the following ports to the queue in this order without
> the tinderd parameter -norebuild (after port php5 was updated)
> 
> 1. ports-mgmt/tinderbox
> 2. lang/php5
> 
> Now tinderbox builds first all all decencies for tinderbox, so php5 is
> rebuild (new port version)
> If the port tinderbox is finished it starts over and builds the port
> php5 again.
> 
> An improvement can be to check for every port which was build successful
> as dependency if the port is additional in the queue and then set the
> state to success to prevent a second build.

That's what the -norebuild option is for.

> 
> > The one exception to the rule above is if a port is explicitly requested
> > to be built in Tinderbox (i.e. on the tinderbuild command line).  In
> > that case, Tinderbox will remove the existing package, and force a
> > rebuild.  If you want to bypass this behavior, pass -norebuild to
> > tinderbuild.
> 
> 
> Thanks for the hint, I tried a rebuild with the modified tc_command.pl
> for ~300 ports and noticed one thing.
> 
> For every port in the queue the following command was executed, even the 
> port was not rebuild.
> 
>   tar -C $space/tinderbox/7.2-FreeBSD -xpf 
> $space/tinderbox/jails/7.2/7.2.tar

Every time tinderbuild is invoked, it will re-extract the Jail contents
to the Build directory.  This cannot be stopped.  This is done to ensure
that each Build is clean.

> 
> [19:12:34 /mirror/tinderbox/scripts]
> # ps -f | grep tinderbox
>   tar -C /mirror/tinderbox/7.2-FreeBSD -xpf 
> /mirror/tinderbox/jails/7.2/7.2.tar (bsdtar)
>   /bin/sh /mirror/tinderbox/scripts/tc tinderbuild -b 7.2-FreeBSD 
> -nullfs -onceonly -norebuild mail/p5-Mail-Tools
>   /bin/sh -T /mirror/tinderbox/scripts/tinderd -nullfs -onceonly -norebuild
> 
> [19:13:21 /mirror/tinderbox/scripts]
> # ps -f | grep tinderbox
>   tar -C /mirror/tinderbox/7.2-FreeBSD -xpf 
> /mirror/tinderbox/jails/7.2/7.2.tar (bsdtar)
>   /bin/sh /mirror/tinderbox/scripts/tc tinderbuild -b 7.2-FreeBSD 
> -nullfs -onceonly -norebuild devel/p5-PatchReade
>   /bin/sh -T /mirror/tinderbox/scripts/tinderd -nullfs -onceonly -norebuild
> 
> [19:14:27 /mirror/tinderbox/scripts]
> # ps -f | grep tinderbox
>   tar -C /mirror/tinderbox/7.2-FreeBSD -xpf 
> /mirror/tinderbox/jails/7.2/7.2.tar (bsdtar)
>   /bin/sh /mirror/tinderbox/scripts/tc tinderbuild -b 7.2-FreeBSD 
> -nullfs -onceonly -norebuild graphics/p5-GD-Grap
>   /bin/sh -T /mirror/tinderbox/scripts/tinderd -nullfs -onceonly -norebuild
> 
> 
> Is there a parameter to stop extracting the jail for every port?

For all ports within a ports build run, the Jail will only be
re-extracted if an error occurs.  Each separate invocation of
tinderbuild will re-extract the Jail.

> 
> > 
> >> Additional a small function like ./tc rebuildAllPorts which injects all
> >> finished ports back to the queue will become interesting since there are
> >> maybe now ports which have not updated but have an old decencies to
> >> php5-5.2.11 instead php5-5.2.12.
> > 
> > Try this patch.  It changes addBuildPortsQueueEntry so that the -d
> > argument is optional.  If not specified, all ports for a given Build
> > will be inserted into the queue.
> > 
> > http://www.marcuscom.com/downloads/tc_command.pl
> 
> Thanks, this works super. Exactly what is was looking for.

I committed this to HEAD.

Joe

-- 
PGP Key : http://www.marcuscom.com/pgp.asc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: This is a digitally signed message part
URL: <http://marcuscom.com/pipermail/tinderbox-list/attachments/20100110/cf88cc35/attachment.bin>