Comment 5 for bug 631785

Revision history for this message
Vincent Ladeuil (vila) wrote : Re: [Bug 631785] Re: Tests should pass using installer bzr versions and be run as part of the release process

>>>>> Alexander Belchenko <email address hidden> writes:

<snip/>

    >> We need a way to get the code to test and we want to run selftest
    >> (the actual installer doesn't embed enough of the test suite for
    >> that if I remember correctly). So we need bzr and its associated
    >> test tools anyway.

    > If we talking about bzr.exe then everything except python.exe
    > should be there.

And bzr.exe will update itself ? I had bad experiences with such setups
for tests envs.

I'd strongly prefer working a good isolation when running the tests,
that's the point anyway: focus on the system under test instead of
tweaking the whole system.

    >> Of course we should isolate the actual tests to ensure we don't
    >> grab any hidden dependencies though.
    >>
    >> A dedicated test suite for the installers sounds like a good plan
    >> anyway.

    > Just skip tests if we run bzr.exe -- it's easy to create Feature. The
    > check for bzr.exe is very simple:

    > getattr(sys, "frozen", None) is not None

Sure. But adding a check to *all* existing tests ?

I've thought about that for other needs, and invariably the answer is:
creating a dedicated test suite (which could include exising tests) is
way easier than forcing new constraints on all existing tests.

And when I say dedicated test suite, it could be as simple as running:

  selftest -s bt.test_installer

And these tests could well use the feature you mentioned above to skip
in the normal case.