Simplify Testing of IoT Apps and Connected Devices

Transcription

1 White Paper Application Delivery Management Simplify Testing of IoT Apps and Connected Devices

2 Table of Contents page The Internet of Things (IoT) Is Here to Stay... 1 More Than 8 billion Connected Things in Examples of IoT Applications at Work... 2 Dealing with The Internet of Complicated... 4 Advantages of Mobile and Service Virtualization Technologies for IoT... 7 Real-World Examples of Virtual Testing... 7 About Service Virtualization About Mobile Center

3 The Internet of Things (IoT) Is Here to Stay Internet of Things (IoT) predicted to grow to 26 billion units by has penetrated all of our lives, wherever we work and live. And although the influence of IoT is already seen across many vertical markets, we re just getting started. It will take on an even stronger presence with the proliferation of connected devices, appliances, smartphones and watches, wearables, and vehicles. IoT is a direct result of rapid changes in consumer and business lifestyles. We ve become accustomed to seeking instant response, instant gratification. Increasingly, we see that it s not enough to just be able to provide consumers with information and services on the go, 24/7. Businesses are clamoring to get ahead of the competition by using modern technologies such as IoT and mobile to take the consumer experience to the next level where information and services are delivered intuitively and in real time, based on the consumer s personal preferences, surrounding environment, activity in motion, and needs of the moment. Future digital experiences will continue to evolve at a rapid pace with the rise of mobile technology, availability of new types of devices, and consumer expectations. Huge Implications for App Development and Testing IoT has broad and significant implications for application developers, testers, and QA experts. Testing has been extended from traditional client-server, tiered platforms to more complex but open ecosystem environments. This means that cross-functional teams must collaborate very closely to address development and testing issues brought by this new paradigm. Testing and QA organizations need to look well beyond devices and sensors, and closely examine the added technical complexity that comes with the huge volume and complexity of data generated across a smart, connected infrastructure. IT teams must quickly master the new skills, tools, and architectures required by mobile applications and IoT. It s no longer enough to perform testing within just the traditional QA scope. More Than 8 billion Connected Things in newsroom/id/ Gartner, Inc. forecasts that 8.4 billion connected things will be in use worldwide in 2017, up 31 percent from 2016, and will reach 20.4 billion by Total spending on endpoints and services will reach almost $2 trillion in Greater China, North America, and Western Europe are driving the use of connected things, and the three regions together will represent 67 percent of the overall IoT installed base in Consumers will be the largest users of connected things, with 5.2 billion units in 2017, representing 63 percent of the overall number of applications in use. Businesses will use 3.1 billion connected things in

4 White Paper Simplify Testing of IoT Apps and Connected Devices The exponential growth in connected devices and applications brings with it dramatic changes in software development and testing. As Forrester recently noted, Just as rising customer expectations for consumer and enterprise applications catalyzed a sea change in software development, the increasing importance of software in IoT products means big changes for product development processes. 2 The Challenge of Testing Sensor Interactions Sensors are at the heart of IoT, embedded in a wide variety of applications from healthcare devices to thermostats to kitchen appliances to airplane engines. Data gathered from these sensors and devices holds the promise of increasing efficiency and productivity, and improving lives. Testing real-time interaction of all these sensors is a heavy challenge. Understanding the performance of application across a broad environment requires new approaches by application developers and testers. Not only do they have to configure an environment to validate performance, they also need to reproduce certain real-life situations that are challenging to test. For example, it s not easy to test loss of tire pressure while a car is traveling at 60 mph but this is an essential scenario to test for a connected device application. IoT interfaces typically involve a number of analytics engines and algorithms and require experience in simulation to create a valid test environment. Examples of IoT Applications at Work In several key industries, IoT applications and devices are already at work collecting and analyzing data that improves performance and provides added insights. Automotive The automotive industry is a great example. When it comes to connecting drivers and technology, the auto industry has a longer and richer track record than any other sector. Connected vehicles are now able to analyze real-time information from a vehicle to provide new insights to vehicle users, improving both performance and user experience. Sensors located throughout vehicles are now able to measure and report things such as tire pressure, fuel levels, and driver behavior (drifting, accelerating, etc.). This level of connectivity delivers big gains for today s automotive companies because they can use data from the car to analyze its performance and obtain valuable data on how drivers use their cars. And greater connectivity provides more ways for automakers to deliver their products and services to customers. But there can be significant challenges and costs in testing mobile applications connected to a physical car. And some uses cases may be quite dangerous to test with a real car. In automotive: Testing typically starts at late stages of the development process Due to high costs, the number of cars for testing is limited and not scalable 2 Forrester, The Internet Of Things Propels Product Development Into The Digital Era, May 4,

5 There is a highly fragmented test bed, with permutations of cars, app versions, diverse mobile devices and OS consideration Tests can be very costly and time-consuming The future of digital experiences will continue to evolve to a high degree of complexity. Manufacturers who are tasked with delivering quality apps and services at speed to customers will find very quickly that the consumer bar continues to rise with the expectation for more immersive, connected experiences. In a day-to-day driving scenario, you can envision that as a consumer you will automatically be notified of low fuel level, as you are driving, and you ll be directed to the closest gas station. You may even be notified that the gas station is offering a free car wash and soft drink this week, and your device mapping application will automatically update to take you to that station for refill and auto payment from your mobile phone. Retail In the retail industry, sensors and IoT are revolutionizing the customer experience. For example, in-store customers can now use smartphones or wearable devices to quickly scan an item and pull up relevant product information, reviews, or social media commentary. Retailers can also send personalized digital coupons to customers upon entering the store. In some stores, robots with touchscreens can help customers browse inventory and make buying decisions. Imagine a typical customer experience at the local shopping mall today: While entering the mall, you receive a text that says a sofa you looked at online yesterday is now on sale at your favorite home furnishing retailer. Directions to the retail store location in the mall pop up on your phone, and you go to the location, scan the photo and tag on the sofa, visualize how the sofa is going to look in the backdrop of your living room, then check customer reviews as well as current pricing. The retail experience is changing dramatically in these ways and more adding a level of complexity that we haven t seen previously. Mobile is at the center of an explosion of connected devices. Mobile-only Internet use has grown 28% in the last two years alone, reaching 41 million U.S. in Over 8 in 10 Internet users will use a smartphone to access the web regularly in We are seeing more than ever before, mobile empowering significant changes for both businesses and consumers. It is no longer looked at as merely a sales channel for businesses. Instead, it has become a strategic tool to help businesses transform themselves to stay ahead of competition Forrester Research Mobile and Tablet Commerce Forecast (US), While a recent study showed that mobile and tablet sales are expected to grow at an annual compounded rate of 17% through 2020, the actual shopping volume done over mobile is still fairly low (15% of overall retail sales by 2020) compared to desktop or tablets. In fact, analyst predictions for U.S. cross-channel retail sales are up to trillions in the next few years, as more consumers conduct pre-shopping across more categories on digital devices including phones, tablets, and watches while on the go, 24x7. 4 3

6 White Paper Simplify Testing of IoT Apps and Connected Devices The largest opportunity for mobile is in influencing the sales and consumer purchase journey. Mobile s key role in the near to future term is helping consumers make the right product decisions, find stores with inventory, and feel comfortable they are getting the best deal possible. While mobile is not a platform for actual purchasing, it is increasingly used to help recommend and influence consumer behavior. This means that when designing and testing mobile apps, developers need to consider all the various backend and middleware systems, sensors, and IoT-related apps, evaluating how to test these different components. Serving customers in such rapidly evolving mobile landscape will require a different mindset and approach to testing the digital user experience. Figure 1. Test environment complexity Dealing with The Internet of Complicated Testing mobile applications in the IoT brings an entirely new set of challenges and level of complexity. Welcome to the Internet of Complicated. Consider, for example, the following development project: Problem Let s say you re building an app that interacts with a hardware device and sensors across the IoT. You have to test your app, but you also have to test its interaction with a connected device or devices. If you understand that mobile app development and testing is hard, adding IoT to the mix makes it even harder and much more complicated. 4

7 Delivering mobile apps that excite and engage users is challenging on its own typically you have to deal with opinionated users with short attention spans coupled with inflated expectations of quality. But on top of that, your development and testing efforts must accommodate multiple mobile platform vendors (such as Apple, Google, etc.) and a large catalog of devices from a broad array of phone manufacturers worldwide (Samsung, Apple, etc.), all with their own individual tweaks and enhancements to the operating system. Cause What causes this complexity in testing? Many factors, including: Multiple revisions of apps Multiple operating system versions on the across similar devices The challenge of maintaining older devices while keeping up with OS updates The complexity of IoT devices: Multiple hardware revisions; multiple communications protocols; limitations on how many IoT devices a mobile device can connect to simultaneously How can you know that all mobile devices will communicate effectively with all compatible hardware devices (IoT)? Do you create a mobile device lab, separate from the IoT lab, in the cloud or on-premises? This would be quite expensive and time-consuming. You can t have enough IoT devices for every mobile device used in testing. You basically need two complete labs, with a full array of smartphones, tablets, and the like (multiple versions of each, so you can accommodate testing scale as well as multiple OS versions). You also need a complete catalog of all compatible IoT devices to test. In all likelihood, you will never have enough of both to accommodate all of the possible permutations. Without a huge lab containing every popular smartphone and multiple copies of each compatible hardware (IoT). You can t have enough IoT devices for every mobile device used in testing. You basically need two complete labs, with a full array of smartphones, tablets, and the like (multiple versions of each, so you can accommodate testing scale as well as multiple OS versions). You also need a complete catalog of all compatible IoT devices to test. In all likelihood, you will never have enough of both to accommodate all of the possible permutations. Without a huge lab containing every popular smartphone and multiple copies of each compatible hardware (IoT) 7 device, your task is impossible. Consider also the need for parallel testing. Physical hardware devices make this impossible to do without having more devices than you want to maintain. Solution So, what do you do? One proven approach is to dramatically reduce the complexity of your test environment through mobility and virtualization technologies. Micro Focus mobility and virtualization solution have been designed to do exactly that. 5

8 White Paper Simplify Testing of IoT Apps and Connected Devices Instead of building a custom solution that enables you to share your IoT devices across all of your mobile devices, you virtualize the IoT devices. Then, you test the app on each of the smartphones in your lab, with every possible compatible (virtual) hardware device in your catalog. You build a simulation by testing the application against a physical IoT device, with the Service Virtualization (SV) server in between the two (acting as a proxy for the inter-device communication). SV accurately records every transaction between the mobile app and the IoT Bluetooth-enabled devices such as smart home audio speakers and appliances. Figure 2. The Service Virtualization recording process and testing via Mobile Center During testing, the simulated device is used instead of the original hardware device. Application-side transactions complete, just as they did before, but the key difference is that you re not using a physical device to do it. Benefits Using virtualization to perform IoT connected mobile application testing has several key advantages: Delivers scale in a way that s not achievable any other way Assures complete coverage for all IoT device to mobile device permutations Enables broader test coverage and reach Eliminates need to maintain every possible BLE-enabled device in quantity to support your testing IoT devices can be borrowed you get one from a manufacturer, record its behavior, and send it back Dramatically simplifies regression testing (testing a new version of an app against all of the previously compatible hardware devices) With test automation, enables continuous testing of apps with speed and agility 6

9 Advantages of Mobile and Service Virtualization Technologies for IoT We have developed key mobility and virtualization technologies to improve the testing of IoT applications. These technologies are already at work helping manufacturers and product designers improve the way IoT is deployed in their businesses, and deliver better apps faster. Through virtual models, you can improve the delivery process in these ways: Accelerate and Automate Testing Remove resource constraints and bottlenecks by virtualizing your devices Test in parallel with manufacturing Parallelize testing execution with virtualization Enhance the Infrastructure Reduce the cost of your physical lab Enable scalable virtual labs Reduce physical lab and maintenance costs Fix Defects Faster Get instant feedback on code changes Simplify provisioning of virtual lab for debugging Quickly reproduce failing conditions from complex end-to-end tests Improve Coverage Cost effective permutations of connected things, app version, mobile device and OS Scalable virtual labs to support more test flows Easily automate your tests Continuously test your app Real-World Examples of Virtual Testing Example #1: Testing Smartphone Applications for a Car Virtualization can help test and improve your car-connected smartphone applications using a virtualized model of your car. You can simulate situations that are expensive or impossible to set up, and you can fully automate testing. 7

10 White Paper Simplify Testing of IoT Apps and Connected Devices Testing can be difficult when you deal with complex physical devices, such as a car. While it might be simple to test things like whether a seatbelt works or if the brake lights go on, other functions present more of a challenge. For example, how do you test things like low tire pressure, insufficient coolant levels, or conditions that only happen at high speed? How do you test a crash? This challenge becomes even more complex when you need to test more than one model of a car with several models and versions of smartphones running on different platforms. Mobility and service virtualization solutions can make this type of testing much easier. Mobile Center enables you to select specific smartphone models to test with your car-connected application. You simply select the real-world mobile device you need to use, and then can start testing the app from a single central platform on your desktop. You can also use Service Virtualization to build a virtual model of your car. You can easily and quickly create services to access and communicate virtually with your car, making your smartphone think it is connected to a real car or a real car s sensors. Service Virtualization simulates the car s sensors and captures data about how the car behaves. The communication between the car s sensors and the smartphone application is intercepted and recorded. Gradually, the system learns how the car responds to your application calls, and subsequently it analyzes the data to create a virtual model of your car. Once this model is created, you no longer need a physical car for testing everything can be done using the virtual car. You are able to not only automate testing but you can also continuously test various use cases. With simulation models, you can test without the risks you might encounter with a real physical test (for example, tire pressure loss in a car while driving at high speeds). You can repeat test scenarios again and again, perform testing from your desktop, automate testing, and include it as part of the overall Agile, continuous testing effort. For more details, please see our video demonstration at: As more and more connected things are combined with the mobile experience, testing becomes more challenging. Developers and testers are asking these important questions: How can I efficiently test variety of coffee machines, app versions, mobile devices, and operating systems? How can I reduce the waste of time consumed by manual operation over the machine? How can I test cases that happen after a certain amount of usage? How can I reduce the maintenance cost of the testing machines? 8

11 MOBILE CENTER AND SERVICE VIRTUALIZATION ADDRESS THESE QUESTIONS AND MORE No manual operation and physical maintenance is required for virtual machine Easily test any permutation of machine, app version, device, and OS Automate your tests, increase efficiently and quality Dramatically reduce testing costs Figure 3. Testing and simulation of connected car apps Example #2: Testing Bluetooth-Connected Devices As noted earlier, testing smartphone applications can be complex and difficult. And integration with a physical device (a home appliance for example) brings a whole new level of complexity. But through virtualization and simulation, you can cost-effectively and quickly test the performance of your Bluetooth-connected device and the application that controls it. For example, consider the process of testing an application on a smartphone that controls a coffeemaker. The app is designed to not only brew a cup of coffee but to also signal when the machine needs to be cleaned (descaled), when the water level is low, when overheating occurs, etc. As a developer or tester, you want to make sure the application performs well on not just one smartphone model, but on all models supported, and all platforms used. How do you do this without performing the same set of tests on dozens of phones and dozens of coffeemakers? A set of mature products from Micro Focus Mobile Center and Service Virtualization, help you perform testing on several devices virtually, by creating simulations. With Mobile Center, you can select one or more smartphone models to test. Then, with Service Virtualization, you can build a virtual model of the 9

12 White Paper Simplify Testing of IoT Apps and Connected Devices coffeemaker and execute multiple application tests using the virtual version of the coffeemaker instead of the real appliance. SV intercepts Bluetooth messages between the smartphone and the appliance, while a new behavior model is being built. When a sufficient number of messages have been recorded and analyzed via the testing mode, you can stop the learning mode in Service Virtualization and finalize the virtual device model. From now on, you won t need a physical coffeemaker test your app; the virtual coffeemaker simulates everything the physical appliance might do. While there are testing scenarios that may be much more complex than this, the coffeemaker examples shows how you can dramatically simplify testing, cut costs, and improve quality delivery by creating a virtual model. For more details, please see our video demonstration at: Example #3: Wind Turbine Simulation Arrays of large turbines, known as wind farms, are becoming an increasingly important source of intermittent renewable energy and are used by many countries as part of a strategy to reduce their reliance on fossil fuels. As more and more sensors are embedded in wind turbines in large wind turbine parks, it s too complex to test all environmental conditions. For example, how do you test a large variety of use cases, such as nature-related cases like high winds and storms? How can you reduce the number of physical turbines needed for testing? And how can you safely roll out software changes to production, with the assurance that they have had sufficient testing? With virtualization, you can easily simulate the OPC (OLE for Process Control) traffic from the wind turbines and tools without using physical turbines. No manual operation and physical maintenance of the machines is required, which reduces your costs significantly. You can easily test a wide variety of scenarios including disasters. You can also fully automate the tests, increase efficiency of testers, and achieve quality delivery and stability. Virtual sensor EDGE SERVER IoT PLATFORM APPLICATION PLATFORM SERVICE VIRTUALIZATION UFT/LEANFT/ APPIUM/SELENIUM Virtual Actuator Virtual Sensor Application Logic Rule Engine Data Access API (Shadows, Historical) Analytics Device Management Application Logic Virtual Lab Virtual 3 rd Party Services Web UI MOBILE CENTER End-To-End Tests Sensor Data Virtual Sensors, Actuators Mobile App TEST INFRASTRUCTURE, ASSETS APPLICATION UNDER TEST SIMULATED ASSETS Figure 4. Smart and Efficient IoT Application Testing with Service Virtualization and Mobility solutions 10

13 About Service Virtualization Service Virtualization software removes the development and testing dependencies that can slow delivery of applications. It enables your teams to easily create realistic simulations of service behavior so you can stay on schedule even when dependent application components or systems aren t available. With resource constraints no longer impeding your development and testing tasks, you can now focus on delivering quality applications and services at the speed and agility expected by your customers. Faster Delivery and Higher Quality A core issue in application development and testing today is the growing interdependency among functional components. This often leads to reliance on resources from third-party vendors, which can lead to wait time and extra cycles. Service Virtualization enables your application teams to easily create virtual services that can replace targeted services in a composite application or multi-step business process. By accurately simulating the behavior of the actual component, it enables developers and testers to begin functional or performance testing right away, in parallel even when the real services are not available when data access is restricted, when data is difficult to attain, or when the services are not suitable for the particular test. At the same time, Service Virtualization can reduce costs by eliminating the requirement for access to constrained business-critical infrastructure, third-party systems, or pay-per-use cloud components for testing. Other benefits include: Elimination of time and access constraints for dependent components and services Broader test coverage and fewer defects released to production Reduced risk and faster release cycles through earlier functional and load testing Lower costs related to provisioning and managing complex test environments Enhanced collaboration for teams across the lifecycle About Mobile Center Mobile Center is a single mobility gateway that helps you test and monitor your applications for an enhanced user experience across digital touch points. It enables you to manage real devices, emulators, and apps as you develop and test mobile applications all on a single, central lab-management platform. 11

14 White Paper Simplify Testing of IoT Apps and Connected Devices Superior Cross-Platform Testing You can meet complex enterprise requirements by relying on Mobile Center s out-of-the-box support for widely used open source tools such as Appium and Selenium. It enables quick and easy parallel execution of test scripts and reuse of these scripts as needed. With Mobile Center, your mobile development teams can continue to use their preferred testing framework and development language while taking advantage of enterprise-grade capabilities powered by Mobile Center. These capabilities help your organization easily scale mobile application performance testing projects, achieve aggressive testing goals, run tests on multiple devices under different network conditions in parallel, and rapidly release superior applications to the market. Learn More At /mobiletesting 12

15 Additional contact information and office locations: H 08/ Micro Focus or one of its affiliates. Micro Focus and the Micro Focus logo, among others, are trademarks or registered trademarks of Micro Focus or its subsidiaries or affiliated companies in the United Kingdom, United States and other countries. All other marks are the property of their respective owners.