When a company launches a new mobile/OTT app, it is very important that the mobile/OTT app works properly. This may sound simple, but you would be surprised how many issues arise during the development of a mobile/OTT app.
As a digital analyst, I am often tasked with providing quality assurance for the development builds of my clients’ new apps, which includes testing their functionality and data collection. Doing a quality assurance check of an app before launching is vital to the success of the app.
If data is missing or if an essential part of an app is not functioning correctly, it might negatively affect their ability to plan effective advertising and marketing campaigns. I currently work with two large media companies that collect vast amounts of data. It is very important in my day-to-day work to have processes in place that allow me to QA all their apps in their latest development builds.
Getting the build for mobile apps is typically easier than for OTT devices because there is a centralized way for developers to provide the individuals doing the testing, or the testers, with access to dev builds. Using Test Flight (for testing over the air installation and mobile applications) or The Hockey App (a mobile crash analytics and app distribution service) a developer can provide a tester access to their builds through their accounts. Testers can then sign in to their account and download a development version of the app to the appropriate mobile devices.
But the emergence of OTT devices has created a unique challenge in the QA process. The process of placing a development build on OTT devices is often more complicated than for mobile devices. Below I have provided tips for testers to put development builds on four different OTT devices, along with some tips as to making the testing process for OTT devices easier.
Application integration is one of the biggest challenges for deploying and maintaining an OTT service.
For QA on Apple TV (fourth generation or older) TestFlight is available to distribute development builds to testers. Developers may invite up to 25 internal testers within their organization, as well as 2,000 external testers outside their organization.
When a tester is invited to test via TestFlight, they can download the development app onto their device via the free TestFlight app. Apps uploaded to TestFlight are available to invited testers for 90 days. TestFlight offers developers the ability to track tester engagement and app performance by viewing the build status metrics in iTunes Connect. TestFlight also gives developers the opportunity to get email feedback from testers through a preset feedback email.
To download a development app onto a Roku device, the device must be first set to development mode. Next, the tester will need to have access to a development build in file form.
After obtaining the development build, the tester will need to determine the IP address of the Roku device they will be using to perform the QA. Finding the IP address is as simple as going to the settings of the Roku device and looking under the “player info” section. The tester will also need to ensure both the computer and the Roku are on the same network. Then, they will open a browser on their computer and enter the IP address of the Roku:
When the screen opens, they can click the browse button and proceed to the location of the development build file. Then, they’ll select the file and click install. A success screen will appear and the file should install and begin playing immediately.
Fire TV created the most intricate method of downloading and installing development builds. To begin, a tester would install a copy of Android Debug Bridge onto a computer. Next, they should open a command prompt and enter the command ‘ADB connect’ followed by the Fire TV IP address. The Fire TV IP address can be found under settings > system > about on the Fire TV device. Once connected, a message will appear in the command prompt indicating connection.
After they are connected, they will need enter command “ADB Install” followed by the .apk file path of the development build that resides on the computer.
As an example, the file path could be as follows C:\Users\Admin\Downloads\client-0112.apk. If the file is successfully installed onto the Fire Stick, a success message will appear in the command prompt.
Once the development build is installed onto the Fire TV device it can be accessed via Settings > Applications > Manage Installed Applications.
Chromecast is unique to OTT devices because a development build does not have to be downloaded onto the Chromecast device to test it. The development build lives on the device from which a user is casting, such as an iPhone or Android tablet. This makes the functional QA of Chromecast straightforward. One can download the development build on a device, cast and QA.
When it comes to performing a full QA of all data collected or passed by the Chromecast device, it is required that the Chromecast device be registered as a developer’s device. This can be done online through the Google Cast SDK Developer Console. The Developer console also supplies a step-by-step process of debugging a Chromecast device.
With trends indicating an increase in usage for 2017, it is essential that apps on these devices both work properly and collect the correct data.
Always ensure the computer and device are on the same network. A tester may have multiple wireless networks at their disposal at any given time. They should not forget to check that the OTT device is on the same wireless network as the computer they are using. This will cause errors during installation.
Make sure an older version of the app is not already installed on the OTT device. On certain OTT devices, a new version of a development app will not overwrite the app currently on the device causing errors during the installation process. Not all devices and installation processes make it clear why these errors are occurring, leading to large amounts of time wasted.
The tester should have access to the most recent development build in TestFlight and the Hockey app. If clients don’t provide the most recent builds, a tester can perform an intensive QA on wrong build, wasting a lot of time. They will always want to ensure they have and know the name of the latest development build prior to beginning a QA.
The tester should also know the specifications of the app they are testing, so that they understand what they are supposed to see during the testing process. Still, I often get a request to QA a build without receiving specs for functionality or analytics. When specifications are not available, the tester can compare the analytics and functionality to the app that is currently in production.
In 2016, more than 70% of US households streamed at least one OTT service. With trends indicating an increase in usage for 2017, it is essential that apps on these devices both work properly and collect the correct data. Without processes in place for proper QA, it will be difficult to collect the data you need to launch successful marketing and advertising campaigns.