Recently we wrote about a hardware solution that we created to keep our test devices’ batteries from swelling. This time we want to talk about the way and the tools we use to test battery usage of mobile apps.
Nowadays everyone wants to extend their phone’s battery life. In fact, battery life is more important to consumers than brand, processor speed or camera quality, according to IDC. For example, the legendary Nokia 3310 phone only had a battery capacity of 900 mAh (nickel–metal hydride battery type). In comparison, the Apple iPhone 7 has a 1960 mAh lithium-ion battery and the Samsung Galaxy S7 Edge has an even bigger 3600 mAh battery. But even with such big batteries, phones can only last for a maximum of a couple of days. And more often than not, our phones need to be charged every day or even a few times a day.
There’s an interesting race going on right now. From one side, mobile chipsets are becoming more energy efficient. But on the other hand, apps are becoming more demanding in terms of processing power and we see a growing trend not only in the amount of data used by mobile devices, but in the speed of connections as well. As bigger batteries pose bigger threats of explosion, software developers should be working really hard to conserve as much power as possible in their apps. And we can help with that!
How can you measure battery usage?
There are a couple ways of measuring battery usage on mobile devices. The solution could be as simple as looking in a device’s settings or using freely available apps (for example Battery Doctor) that claim to measure (and sometimes save) battery usage. But as you can imagine, they can’t tell you the whole story. And on certain more restricted devices they can be of no use at all. Another way would be attaching the phone to special measuring equipment that can read battery measurements in real time. This is our way of testing battery usage.
Before we start testing
But before we connect our devices to testing equipment we need to prepare for testing. First of all, we must define a test case scenario and prepare the device itself. Let’s create a test case.
- Title – let’s give our scenario a name;
- Preconditions – before each test we need to do a factory reset on our test device;
- Data set on phone – the number of contacts must be determined, call logs must be set as well as SMS if they’re needed, or other parameters that can be set on a phone;
- Scenario – here all of the steps are shown, how the test will be executed and, of course, the sequence of the steps;
- Duration – time of the test.
Example of a full test description:
- User has a factory reseted device;
- Device is connected to a mobile network (LTE);
- Testing application has been installed;
Data set on phone:
- 500 contacts;
- 300 call logs;
- Close all background tasks in phone;
- Start testing application;
- Start battery monitoring;
- Lock the device;
Duration: 140 mins
For a more accurate result, before each test we do a phone baseline test. A baseline test is done with a freshly resetted device, while it’s brand new and clean so that we can know how much the device’s background processes, screen, etc., use on the battery. We include baseline results in the test statistics as well, so we can be sure that we only provide measurements of the applications being tested. Of course, sometimes we face some problems, when test is interrupted, that’s why we implement the test 3-5 times, and all “bad” results are excluded.
Let’s start testing!
At TestDevLab we use the Power Monitor power measurement tool to provide needed measurements, the Wireshark data sniffer to collect data samples and a router to sniff the packets we need. One clear benefit of Power Monitor is that it can collect 5000 samples per second that gives accurate measurements even if the test runs for just a couple of seconds. The number of current samples can be dynamically changed and it allows you to find the best number of samples needed for current measurement.
The test device is attached to Power Monitor (only one device can be attached to a single Power Monitor) and it is connected to a computer. Both measurement tool and computer are on the same network, which is specially configured and equipped with special tools to capture data traffic from the phone.
Components of our battery testing lab
The Power Tool software and the Power Monitor hardware provide a robust power measurement solution for mobile devices. The PowerTool software and the Power Monitor hardware can analyze the power on any device that uses up to a 4.55V battery. The Power Monitor can measure data on three channels: Main, USB, and Auxiliary. We use the Main Channel to gather data from the device and graphically present the data.
To attach a device to the Power Monitor we need to isolate the device’s (+) pin, and connect it directly to the monitoring device and the (-) pin is attached to Power Monitor and the battery of the phone together. This is what it looks like schematically.
And here’s how we connect an actual test device to Power Monitor.
The distributor of Power Monitor, Monsoon Solutions, provides a tool with a graphical interface as well, as it is shown below.
Using this tool we can see and configure the environment for the testing stage. To power-up the device, first set the Vout voltage. The voltage can be set from 2.01 V to 4.55 V in .01 V increments. The maximum voltage of a lithium battery is usually 4.2 V, so the software will default to a maximum of 4.2 V, and a warning window will pop up the first time a voltage above this value is selected. As seen in the screenshot below, the voltage shows 4.36 V, the value is higher than usual because the device is attached to a USB, which charges the device as well.
The Legend dialog box controls which channels are displayed, and what kind of data is shown in the graph.
The Scale dialog box controls how data is displayed in the graph. The time scale is always shown along the X axis. The Power, Current, and Voltage scales are shown along the Y axis depending on the data selected in the Legend dialog box. The Unit dropdown lets you select the unit to be used for display.
But this is not the only great feature of Power Monitor, it allows you to get the necessary results in a selected interval of time. Whereas the graph shows the average data based on the timescale, the Capture Statistics region provides averages over the entire sampling run as well as instantaneous values for the run. Average power is derived from instantaneous current and voltage, and then averaged over the duration of the sampling run. Instantaneous voltage is shown for Vout. Instantaneous Current and Power can be viewed by toggling between the mA and MW button. Selecting a certain area with the mouse on the graph and by either double clicking or using the scroll mechanism on a mouse (if one is available) will zoom in on the selected area. Note, even though the screen is not updating during this time, sampling is still occurring in the background.
BatWEB turns raw data into informative graphs
OK, now we have raw data about battery usage. We have developed our own BatWEB solution that can help with managing testing projects and test cases. This tool can be of great help if you have tens or hundreds of test cases. With this tool you can not only search data by test cases, devices, application versions and network conditions, but even compare results. For example you can easily see how the application is using battery power depending on the version.
We also use BatWEB to represent results. There is also a neat feature that allows you to create a unique link to the test report so that the client has one less job to do – just click on the link and see if the newest version of their app uses more or less battery power.
Our experience with Power Monitor is positive – the device can precisely measure power usage in real time and we can use this data for BatWEB reports. Developers say that BatWEB can really help them to see how battery and data usage changes in every version of their application. If the situation gets worse, we can work together to find the problem and improve the app.
At the end of the day we’re happy that our work helps improve the battery life of mobile devices because it’s such an important factor for so many customers.