[Exherbo-dev] Replacing RESTRICT

Elias Pipping pipping at lavabit.com
Sat Aug 6 21:28:59 BST 2011

We currently use RESTRICT from gentoo on exherbo.

Possible values are userpriv, test, and strip.

Let's talk about the 'test' value for a bit.

By default we run a src_test phase that just tries to run make
check/test (that's determined using `make -n`). There's also room for
a src_test_expensive phase that is undefined by default.

There are a couple of problems with this approach. One advantage of
being able to say RESTRICT=test instead of having to define src_test()
{ :; } is that paludis will know there is no test suite; a distinction
that should be made. The use of this is narrowed, however, by the fact
that a package for which the default src_test() is used looks to
paludis like a package with a test suite, regardless of whether an
actual test suite is run of `make -n check/test` just returns failure.

Using RESTRICT=test for packages that have no test suite is also not
desirable; A package could add a test suite later and nobody would
ever find out.

Furthermore, we should distinguish clearly between a package that does
not have a test suite and a package whose test suite is broken. The
latter could prompt a QA client to issue a warning.

So these are basic cases that we would like to distinguish:

 * there is a test suite and it works
 * there is a test suite but it is broken
   (for us; be that our fault or upstream's)
 * there is no test suite

We should do this for both tests and expensive tests separately.

Relevant discussion on IRC:

17:28 <pipping> also, 'RESTRICT="test" src_test_expensive() { emake
                check; }' or something along this lines is done in a
                couple of packages; it's sort of abusive..
17:30 <zlin> it's currently the correct way to do it
17:31 <pipping> well. I find 'src_test() { :; }; src_test_expensive()
                { [..]; }' little more verbose but clear
17:32 <pipping> RESTRICT="test" means "do not run tests because
                something is broken" to me
17:32 <zlin> that shows a recommended_tests option that doesn't do
      	     anything though
17:32 <pipping> Right.
17:33 <pipping> But we also show recommended_tests for all kinds of
                packages that don't have a test suite
17:33 <pipping> so why do we care there?
17:35 <zlin> I think we should replace RESTRICT with better mechanisms
             where we track this stuff properly
17:36 <zlin> and actually state whether it's disabled because shit is
      	     broken or because stuff is expensive or because trying to
      	     run them is utterly pointless
17:36 <zlin> and make the shit is broken case annoying
17:36 <zlin> and where we can disable userpriv for tests and just
17:52 <pipping> sounds good
17:53 <pipping> I'd like it if a package clearly stated whether it has
                tests or not, too.
17:53 <pipping> A package is unlikely to just grow a test suite or
                lose one
17:54 <pipping> so that's leave us with something like
17:54 <pipping> or unset TESTS
17:56 <zlin> there are three types of broken
17:56 <zlin> well, at least two
17:56 <pipping> broken our-fault / broken-upstream-sucks?
17:57 <zlin> one where upstream just doesn't give a fuck and the test
      	     suite just doesn't make sense
17:58 <zlin> whether we want to special case things that require X
      	     from that is debatable
17:58 <zlin> and one where it might be fixable
17:58 <pipping> I guess we can always make that finer grained
18:01 <zlin> the important distinction is between should it be
      	     annoying or is it just pointless
18:03 <zlin> also if we go for tracking none I really don't want us to
      	     rely on people noticing the addition of tests f or them
	     to ever get run
18:04 <zlin> I wouldn't mind if the test phase gets run anyway and
      	     gets really annoying if it sees test targets though
18:05 <ciaranm> require tests [ mode=verify-no-tests ]
18:06 <ciaranm> probably shouldn't be an exlib...
18:08 <zlin> why did we never add a test phase that runs as root?
18:08 <ciaranm> because tests are a mess thanks to being inherited
      		from gentoo and gentoo never handling them properly
18:10 <zlin> RESTRICT=strip can be replaced by dont_strip $path in
      	     src_install, right?
18:10 <ciaranm> if we had dont_strip, yes
18:11 <ciaranm> actually
18:11 <ciaranm> binaries
18:11 <ciaranm> can't remember how we handle stripping for binaries
18:12 <pipping> zlin: is it likely that a package will just add a test
18:12 <pipping> zlin: I'm only aware of one such case (emacs 24 will
      		add a test suite whereas 23 did not have one)
18:16 <zlin> because it happens rarely people are likely to miss it
18:18 <pipping> agreed. but what can we do about that?
18:18 <zlin> what I just said
18:19 <pipping> okay, the annoying test-target-seer
18:19 <pipping> fine with me.
18:20 <zlin> not sure if other build systems support looking for a
      	     default test target
18:23 <ciaranm> make doesn't really support it either
18:24 <ciaranm> the hack we have now sometimes does horrible stuff
18:24 <ciaranm> for example for paludis
18:24 <pipping> it relies on make -n, right?
18:24 <ciaranm> yeah, which goes splat if the target has dependencies
18:25 <ciaranm> maybe we should just make every exheres specify a
      		bunch of properties
18:26 <ciaranm> CONTROL="tests [[ mode=broken privs=root ]]"
18:28 <zlin> privs=root should not apply to every phase
18:29 <ciaranm> right, hence tests [[ ]]
18:29 <ciaranm> or just test, if we want it to map to phase names
18:29 <ciaranm> test and test_expensive makes things a bit icky there
18:29 <ciaranm> maybe test_expensive should go
18:30 <ciaranm> CONTROL="test [[ recommended=true expensive=true ]]"
      		src_test() { if expecting_tests ...
18:30 <zlin> do we have any non-tests things that we want to control?
18:31 <ciaranm> strip, merge
18:31 <ciaranm> replacing userpriv in general
18:32 <zlin> we currently have exactly two packages with
      	     RESTRICT=userpriv and one of them even had its test
      	     disabled two years ago :p
18:32 <zlin> which leaves just openssh
18:33 <zlin> but yeah sounds sensible enough to me
18:33 <ciaranm> i just had an idea
18:33 <ciaranm> which i haven't thought through
18:34 <ciaranm> CONTROL="compile [[ parallel=false ]]"
18:34 <zlin> we have ten packages with RESTRICT=strip
18:34 <ciaranm> which forces EXJOBS=1 for a particular phase


Elias Pipping

More information about the Exherbo-dev mailing list