Support Center

Analysing the test results

Last Updated: Apr 04, 2013 12:00PM CEST

In the Test Runs tab of Project view you can see your 10 most recent test runs and an indicator of what percentage of test cases have been successfully run across all devices. By clicking a test run you can open a detailed view of all the results generated during that test.

The following is a short video outlining the Test Results Page.  More information can be viewed below.
 

Test Run view

The detailed test run view allows you to see and compare the results in a very flexible way using the filters box on the left. You can filter the devices by Android version, screen resolution, and many more properties. Filtering works globally for all tabs available in Test Run view which means that you can freely switch tabs and your filters remain active.

There are three tabs is Test Run view:

Info

 

The info tab shows the following detailed results for each test run:

  1. Performance statistics – a timeline divided into main tasks of the test process. Viewing details for every single device.
  2. Test results by device – in this view you can see a bar chart reviewing the number of passed and failed tests for each device in the cluster.
  3. Test case execution times by device – this bar chart shows a test execution time for each test case per device. This view is very useful to find any performance related issue as well as to monitor your application's performance on different kinds of Android devices.
  4. Detailed test results – in this view you can inspect detailed results for each test case. The table lists the results for each test case by device, complete with test case execution times. In case of a failed test this view provides a link to complete exception message detailing how the fail happened.
Logs

 

The Logs tab provides a complete Logcat outputs for each device from the entire test run. This info is very valuable when you are debugging and trying to indicate reasons for a specific test case failure. At the top of the view there's also a button to download the whole Logcat output as a zip file for further analysis.

Screenshots

 

This tab provides you with a very powerful tool to visually inspect the pixel-by-pixel screenshots of each device side by side. Testdroid Recorder has a “take a screenshot feature” you can use during a test recording that lets you specify when you want to take a screenshot during the tests. With these screenshots you can view how your application is handling visually on all your tested devices at once. For a detailed pixel-by-pixel inspection, Testdroid Cloud provides a magnifying glass tool that helps you to inspect for device specific rendering problems. Just click on the screenshot to get an enlarged view of the situation.
You can also see how screenshots can be reviewed with the following video.

Trends

In the Trends tab of Project view you can an overview of test results over time for each device. This view is very useful when you try to determine at which point tests on certain device started to fail and you can go back and review what was changed on that application version.

With screenshot baseline feature you can define a baseline for every screenshot on every device (i.e. this screenshot looks exactly as it should) and then every time your tests are executed new screenshots will be compared to your baseline screenshot and if they are not exactly the same you will be notified. Screenshot baseline feature even highlights the differing areas of the screenshot to help you notice how the screenshots are different from each other.

Contact Us

support@bitbar.com
http://assets2.desk.com/
false
bitbar
Loading
seconds ago
a minute ago
minutes ago
an hour ago
hours ago
a day ago
days ago
about
false
Invalid characters found
/customer/en/portal/articles/autocomplete