Popular posts

I am the wrong guy to answer this question.

What I do is - If IE must be targetted (running on my Mac), dev.windows.com/en-us/microsoft-edge/tools/vms/mac then using browsersync.io/ for manual tests. I don't run automated tests in the browser. It's too slow and error prone.

vorlonjs.com from Microsoft is (maybe) better than BrowserSync but it has a steep learning curve IMO because of its huge feature list and configuration options. I like vorlon.js but I still do use BrowserSync - old habits don't die easily ;-)

TL;DR

Generally speaking, automating tests in the browser is very complicated​ as I feel. The setup has a long list of dependencies where each dependency can be a single point of failure and can cause corruption of the test result.

My solution to avoid regression:

• create a browser specific HTML JS and CSS files and only put testing code when features broke in that one browser which is targetted by the test file only
• before deploying a new online version, spin up some virtual machines with specific browsers and open the corresponding HTML (I think this is the part where Vorlon.js will be great some day).

On bug reports by users:

• I send them a link to test HTML site and call them the take a screenshot of the unit test results. Just to confirm it's not a regression and makes the user feel he his not dealing with old issues. By doing so, I have only the test result of a browser, an assertion library and my test cases. Not many single points of failures which can corrupt the test results. No messing around with Polyfills or Shims to make tests pass on browsers which do not have a known issue but to satisfy the automated browser test solution.

I spend hours and days to get around VM+Selenium+WebDriver issues, just to find out running the test natively in a browser does not cause errors. But when called from Selenium+Webdriver (On VM or if possible in a native browser) tests fail in individual test environments. It impacts trustworthiness of test results.