refactored autopilot tests are fail prone for wrong environment setup

The "install" step is needed to maintain the distinction between mocks we need and mocks we don't, during autopilot testing. If you can come up with a solution, please do - but to me the "install" step just brings us closer to the real environment.

What we maybe need to do instead is to allow "make autopilot unity8.shell...." to launch autopilot accordingly? Maybe the "install" step should be part of ./build ?

This also has the path hardcoded to x86... Tests don't work on the phone. Besides "make install" is not available when building on the phone. So right now we can't run a local build of tests on the phone.

And we got fooled again by running installed tests instead of local ones for half an hour. Putting this to confirmed. If there is really no way around the make install it should happen automagically, either by allowing make autopilot to filter for a single test case or calling make install from within the __init__.py.

And the fallback to the installed one really needs to go if there is the slightest indication that a local (non-installed) test suite is ran.

Hi,

Autopilot already prints which test suite it's loading. I'm not sure what more you'd like autopilot to do in this case?

If there's some more information we can log, I'm happy to do it, I'm just not sure this is really an autopilot bug.

Marking as incomplete for now, bug happy to revisit later.

Hi Thomi,

no, this is more about the unity8 test suite, not autopilot itself.

Hi again Thomi,

actually, what I meant is that running a local unity8 autopilot test suite actually loads tests from the local directory, but falls back to loading the unity8 binary which is installed in the system if something went wrong with firing up the locally compiled binary.

So we could add some debug message in autopilot itself that prints which binary it starts up, not only where it loads the test suite from. But as I said, this is really something that is wrong in the unity8 test suite code and actually shouldn't ever happen.

On Mon, Sep 30, 2013 at 9:05 PM, Michael Zanetti <
<email address hidden>> wrote:

> So we could add some debug message in autopilot itself that prints which
> binary it starts up, not only where it loads the test suite from. But as
> I said, this is really something that is wrong in the unity8 test suite
> code and actually shouldn't ever happen.
>

Hi Michael,

We actually already do this - there's a log line inside autopilot that
prints exactly what command line we use to launch the application,
including the full path. I think that's what you're after?

Cheers,

--
Thomi Richards
<email address hidden>

I believe it should fall back to the installed one anyway - for you to be able to run an uninstalled suite on installed unity8. As discussed, though, we could improve the visibility of which is being ran.

Maybe even a prompt if finds a builddir but not the binary?

I am pretty sure that veebers is not working on this bug.