Browser Chrome tests extended!
Yesterday i've pushed the last parts of my small improvements to browser chrome tests harness.
Some time ago Dao pushed bug 521282, ensuring that we focus the browser window before any test run. I've extended that concept with bug 528469. Now, before each test (and also before the first test) we check that we have a consistent number of browser windows, and ensure that the window where the test will run is focused.
This means that if your browser chrome test will leave windows open, we will detect them and notify that as an error of your test. We will then close the unexpected windows, avoiding failures in next tests.
With bug 485269 and bug 529860 i've instead brought to browser chrome tests a couple features we already had in xpcshell tests.
You can now have an header file, called head.js, in the tests folder. This file will be injected into your test's scope, so that you can collect utils and helpers into it, for all the tests in that folder.
You can also register clean up functions, that are guaranteed to be run after your test finishes, regardless the fact it succeeds, fails or timeouts. You just have to call registerCleanupFunction(myCleanupFunc), and you can register as many functions as you will. You can also call it in the head.js file, so it will be run for any test in that folder.
This way you can always guarantee clean up of the environment, that's pretty important for browser chrome tests, since they don't run in new context, so if you don't clean up browser before finishing your test, it is most likely that next tests could fail.
Finally i've provided a way to enlarge timeout for those random failing tests that are timing out due to taking too much time. Notice this is not intended as a solution, you should really investigate why your test is taking so much time, and evaluate if that's due to a bad test design, or to a real performance issue. This is instead intended to try reducing orangeness on mozilla-central.
To use the new method you just have to call requestLongerTimeout(integerFactor), where integerFactor is an integer used to multiply the default timeout value (30 seconds). So if you pass in a factor of 2, your test will have a 60 seconds timeout (30s*2).
I've also updated MDC page about Browser Chrome Tests, for your reference.
I hope these changes will allow people to write better tests and reduce orangeness as well.
Happy browser chrome tests to everyone!
