MEDIA APP TESTING. The challenges of launching high quality media and entertainment apps

Transcription

1 MEDIA APP TESTING The challenges of launching high quality media and entertainment apps

2 01 INTRODUCTION THE CHALLENGES OF MEDIA & ENTERTAINMENT APPLICATIONS A s media and entertainment companies fight to retain audiences and emerging organizations attempt to attract more users adapting to the latest trends in technology is essential to maintaining a loyal user base. Today s consumers are device-savvy, with multiple screens available at all times. As a result, media and entertainment companies are exploring new distribution options to keep their brands alive or launch new endeavors. The adoption of smartphones, tablets and ereaders continues to have a ripple effect on the way people consume news, content and video. According to a study by comscore, 39.5% of users in the US, 32.2% of users in Europe and 57.6% of users in Japan have used mobile devices to get news and information. Nearly 23% of those surveyed in Japan have watched TV and videos on their smartphones, followed by 5.7% in Europe and 5.6% in the US. Digital media has the potential to attract a massive audience. Apps empower media and entertainment companies to constantly engage their fans, build greater loyalty and increase bottom-line growth and revenue through in-app purchases, promotions and advertising. That is if the apps and websites work every time, every where, on every device.

3 FUN FACT TABLETS ARE BECOMING ONE OF THE MOST POPULAR ELECTRONIC DEVICES AROUND THE WORLD FOR CONSUMING MEDIA AND ENTERTAINMENT CONTENT 54% OF TABLET USERS WATCH VIDEO AND 41% USE TABLETS TO GET LOCAL NEWS. (SWIPE-TELECOM)

4 03 DISTRIBUTION IN-THE-WILD TESTING In-the-wild testing is the practice of moving part of your QA out of the lab and into the real world. Testing in real-world conditions provides a better perspective on how apps will work in the hands of real users. It also gives you access to a greater range of devices, operating systems, versions and carriers. CLICK HERE TO LEARN MORE. DISTRIBUTION THE PROBLEM Media and entertainment companies are judged not only for the content they create, but also for their distribution innovation, efficiency and stability. Being able to deliver content over apps to millions of concurrent users is an interesting challenge. While focusing on that technical challenge, don t forget to consider what the users actually experience. Whether users are watching a video or reading a breaking news story, using a native mobile app or visiting a webpage, the delivery of your content needs to make sense and look good in that format. Apps needs to load quickly and consistently across all platforms, wherever your users are accessing the information from. HOW IN-THE-WILD TESTING HELPS The best way to make sure your distribution methods make sense and perform as anticipated is to do a trial run. Have testers around the region, country or world access your app to make sure everything loads correctly and quickly. This will also help you determine if the content is easy to consume in your chosen delivery method and design (e.g., don t make people reading on mobile devices scroll left to right).

5 04 DISTRIBUTION ONE OF THE GREATEST ATTRIBUTES OF APPLAUSE IS THAT YOU CAN FOCUS YOUR TESTING ON ANY DEVICE, ANYWHERE IN THE WORLD AGAINST NEARLY ANY DEMOGRAPHIC OF PEOPLE. Christian Henderson, News International APPLAUSE EXPERIENCE When News International decided to offer an ipad version of The Times and The Sunday Times, the company knew it needed some indepth testing performed under real-world conditions. WITH APPLAUSE, NEWS INTERNATIONAL WAS ABLE TO COVER A RANGE OF DEVICES AND LOCATIONS JUST LIKE ITS READERS. Another major media company was releasing daily web builds in four regions across the US. Applause provided testers around the country to load the main page for 30 minutes each day to ensure the offers displayed correctly. Applause provided testing coverage in all four regions for two weeks every month.

6 05 REACH REACH THE PROBLEM Traditional media delivery options (such as TV or print) can no longer stand alone when it comes to reaching your audience. Consumers are going online, so that s where you need to be as well. But the online world is giant and can be headache-inducing. Between traditional webpages, mobile sites, native apps and video channels, your content needs to be optimized for online display in more ways than one. The gamut of devices, operating systems, browsers and software versions adds to the optimization and reach struggle. THE ONLINE MATRIX How many have you considered? Web browsers: Websites should work well on Internet Explorer, Chrome, Safari and Firefox at a minimum (don t forget popular overseas browsers if that s your target audience). Consumers aren t the most diligent software updaters, so you should also support recent back versions. Mobile operating systems: ios, Android and Windows Phone are the major operating systems and have their own standards when it comes to native apps. Creating apps for all the major OSes will expand your reach but is time consuming. Mobile devices: The range of screen sizes, resolutions, memory, processors, etc. is steadily growing. Users with phones, tablets and phablets all expect your app to look good on their particular device whether it s the largest tablet or the smallest smartphone on the market.

7 06 REACH EXTRA TIP Learn more about mobile app testing in this free ebook: The Essential Guide to Mobile App Testing LEARN MORE HOW IN-THE-WILD TESTING HELPS The idea of housing every device in every iteration is enough to make your CFO run for the door. Not to mention the fact that carriers, connectivity and physical location play a big role in the reach puzzle. Those are things that simply cannot be replicated in a lab. In-the-wild testing puts all these user-specific configurations (the right device, OS, version, carrier and location) at your finger tips, without having to buy a warehouse stuffed to the rafters with devices in every major city. APPLAUSE EXPERIENCE When a media company was preparing to launch a new app, Applause did extensive exploratory testing. APPROXIMATELY 60 UNIQUE DEVICES WERE INVOLVED AND MORE THAN 5,000 BUGS WERE UNCOVERED. Another company requested that their weather tracking app be tested on all major platforms, including Windows Phone. This platform was not commonly tested at the time and required testers with a developer s license. Applause searched our Windows Phone users and found a group of testers with developer licenses to successfully complete the required testing.

8 07 TIME TO MARKET TIME TO MARKET THE PROBLEM The success of media and entertainment companies hinges on timeliness. Whether it s being the first to cover breaking news or timing a promotion just right, the ability to release content and apps quickly (sometimes with little notice or planning) is a major challenge. When it comes to breaking news, apps need to be up to the task of supporting quick changes and allowing the important information to stand out. For entertainment companies, the bigger timeto-market challenge is making sure new apps are released on time, no matter what. Launching an app late won t help a promotion or attract users. HOW IN-THE-WILD TESTING HELPS News happens everywhere, and when it s breaking, it might be vital to get that message to users pronto. Test your app in various locations to make sure it can meet the challenge and catch users attention when necessary no matter which carrier they re on. If your challenge is getting an app tested on a tight timeline, in-thewild testing is an ideal solution. These testers don t just work 9-5 off-hours testing and extra eyes can help you make that deadline.

9 08 TIME TO MARKET IT S IMPORTANT FOR US TO HAVE A PRESENCE ON NEW MOBILE PLATFORMS FROM DAY ONE. WITH APPLAUSE, WE WERE ABLE TO START TESTING THE KINDLE APP ALL OVER THE WORLD ALMOST IMMEDIATELY. Tim Carlson, USA TODAY APPLAUSE EXPERIENCE USA TODAY prides itself on being cutting edge when it comes to mobile apps. The publication offers native apps for most major devices on the market today. LEADING UP TO THE KINDLE FIRE RELEASE, USA TODAY TURNED TO APPLAUSE TO MAKE SURE THEIR NEWEST APP WOULD LIVE UP TO THEIR 4.5+ STAR RATING STANDARD. Applause was able to produce meaningful results that led to a high-quailty release on a tight launch deadline. In another case, a major TV network wanted to send a signal that would give users at home ipad access to behind-the-scenes features during one of the network s most popular shows. Applause provided testers who were able to test the technology during shows before the promotion was officially launched. As always, utesters signed non-disclosure agreements to protect the project s secrecy.

10 09 LOAD TIME LOAD TIME THE PROBLEM Breaking news, big campaigns, viral videos and major events mean your app needs to be ready for unexpected peak traffic. An app might remain dormant for a long period of time, but when something big happens or people finally notice and start sharing your great video or article the app will be slammed with a lot of users all at once. If your app crashes under the pressure, users will go elsewhere for their news or be frustrated (and potentially vocally angry - Hello Twitter!) that they can t witness the latest viral must-see. If users can t get it the first time, they might not try again. HOW IN-THE-WILD TESTING HELPS Traditional load testing floods your app with synthetic traffic, but when it comes to written, video or image-based content it s important to include real users in load tests. Live testers can tell if videos play without skipping and if images and writing render completely. There s nothing worse than half-loaded content or a choppy video.

11 FUN FACT When a site takes too long to load: 28% of people said they would visit a competitor s site 27% would not trust the slow site 18% would not visit again 13% would think the site had been hacked (PromotionWorld) 10 APPLAUSE EXPERIENCE A national publication requested load testing on four of their flagship magazine apps. TESTERS WERE ASKED TO DOWNLOAD CONTENT THREE TIMES DAILY AT PEAK HOURS. THEY COMPLETED IN-DEPTH TEST CASES AND RECORDED LOAD TIMES TO ENSURE PROPER FUNCTIONALITY ACROSS DIFFERENT MARKETS. LOAD TIME This was conducted daily as a live load test.

12 FUN FACT FUN FACT PEOPLE WILL VISIT A WEBSITE LESS OFTEN IF IT IS SLOWER THAN A CLOSE COMPETITOR BY MORE THAN 250 MILLISECONDS (A MILLISECOND IS A THOUSANDTH OF A SECOND). FOUR OUT OF FIVE ONLINE USERS WILL CLICK AWAY IF A VIDEO STALLS WHILE LOADING. (NEW YORK TIMES)

13 12 COMPETITIVE HIGHLY COMPETITIVE MARKET THE PROBLEM There are entire companies dedicated to watching network ratings, tracking news subscriptions and otherwise keeping an eye on the media and entertainment market competition. Needless to say, you can t afford a slip up. This is especially true when it comes to mobile, where the cost of switching for a user is extremely low. Your content and your app need to work together to keep users coming back. Remember, your app is an extension of your brand as a whole. If users aren t pleased with your digital persona, they might switch loyalties all together. HOW IN-THE-WILD TESTING HELPS Media consumers are a fickle bunch anyone in the industry knows that. To make sure your app performs well, is intuitive, has good usability and provides users with what they want and expect you should test with people in your target demographic. In-the-wild testing gives you access to professional testers from all walks of life, including some who are already fans. Who better to test your app?

14 13 COMPETITIVE EXTRA TIP A successful media app is all about beating the competition. Don t rely on guess work and vague stats. Applause Analytics is a mobile app quality tool that lets you see what real users are saying about your app. Applause Analytics crawls the ratings and reviews in the Apple App Store and Google Play and produces actionable data that will help you pinpoint trouble areas. LEARN MORE APPLAUSE EXPERIENCE A premium entertainment organization wanted to test an app that updated with new content, polls, trivia and discussions during a live, televised event. They needed fans of the brand to test the app during a live three hour broadcast, to interact with a chat feature and provide usability feedback. APPLAUSE WAS ABLE TO FIND TESTERS WHO WERE EXISTING FANS OF THE ORGANIZATION AND THUS PROVIDE THE MOST USEFUL, IN-TUNE FEEDBACK POSSIBLE. The test led to optimization of social media integration and a change in the way the app delivers live streaming content.

15 YOU CAN LAUNCH A BEAUTIFULLY DESIGNED NATIVE APPLICATION, BUT IF IT CRASHES, THEN IT WILL RECEIVE A POOR RATING AND USERS WILL GO ELSEWHERE. OUR GOAL IS TO LAUNCH NOTHING SHORT OF A 4.5 STAR APP EVERY TIME - NO EXCEPTIONS. ANYONE CAN READ YOUR APP STORE RATING. THERE S NO WAY TO HIDE POOR QUALITY IN THE WORLD OF MOBILE. MICHAEL CROGHAN MOBILE SOLUTIONS ARCHITECT, USA TODAY

16 14 APPLAUSE CUSTOMERS SOME HAPPY CLIENTS

17 15 ABOUT APPLAUSE ABOUT APPLAUSE Applause is leading the app quality revolution by enabling companies to deliver digital experiences that win from web to mobile to wearables and beyond. By combining in-the-wild testing services, software tools and analytics, Applause helps companies achieve the 360 app quality they need to thrive in the modern apps economy. Thousands of companies including Google, Fox, Amazon, Box, Concur and Runkeeper choose Applause to launch apps that delight their users. Applause in-the-wild testing services span the app lifecycle, including functional, usability, localization, load and security. Applause app quality tools help companies stay connected to their users and the health of their apps with Mobile Beta Management, Applause Analytics and the 360 App Quality Dashboard. The company is headquartered near Boston, with offices in San Mateo, Seattle, Germany, Israel and Poland with resellers serving dozens of international markets. Since launching as utest in 2008, Applause has raised more than $80 million in funding, generated triple-digit revenue growth annually, made consecutive Inc. 500 appearances and was named the 7th Most Promising Company in America by Forbes in Pennsylvania Ave. Framingham, MA