Platform-as-a-service Feature Preference Study

Transcription

1 Research Report Abstract: Platform-as-a-service Feature Preference Study By Stephen D. Hendrick, Principal Analyst with Bill Lundell, Senior Research Analyst & Jennifer Gahm, Senior Project Manager May 2015

2 Introduction Research Objectives As part of ESG s application development and deployment (AD&D) research, this survey was primarily focused on understanding interest in platform-as-a-service (PaaS). There were 4 specific objectives in fielding this survey: 1. Quantify PaaS adoption 2. Understand the level of satisfaction with PaaS and why 3. Identify what PaaS features are most important to IT managers and developers 4. Examine what languages, frameworks, and development environments are most preferred by developers This report will examine the third objective PaaS feature preference. Other reports will examine PaaS adoption and satisfaction and development environments in use. ESG surveyed 326 IT professionals and application developers representing predominantly midmarket enterprise (100 to 999 employees) and large enterprise-class (1,000 employees or more) organizations in North America. For the purposes of this survey, the following definitions of platform-as-a-service (PaaS) were used: Platform-as-a-service (PaaS): A PaaS provides development, deployment, and a variety of runtime middleware services to facilitate the construction, testing, and of operation of applications. Public PaaS: A PaaS that runs in a public cloud, whose lifecycle (bug fixes and upgrades) is determined by the vendor, whose resources may be shared (multi-tenant), and whose development and runtime services are specifically defined. Private PaaS: A PaaS that runs on private infrastructure (in the organization s data center or privately hosted for the enterprise), whose resources are dedicated to the enterprise, and whose development and runtime services can be tailored to the needs of the enterprise. Hybrid PaaS: A PaaS that supports both public and private personas and is able to treat these resources as a single federated pool. This survey was designed to answer the following PaaS feature questions: What vendor characteristics and current/future features do IT professionals and application developers want to see in platform-as-a-service solutions? How does the importance of these features vary by type of respondent PaaS user or evaluator, developer or IT manager and respondent age? Are organizations satisfied with the platform-as-a-service product currently in use? Survey participants represented a wide range of industries including manufacturing, financial services, health care, communications and media, retail, government, and business services. For more details including how ESG screened respondents, please see the Research Methodology and Respondent Demographics sections of this report.

3 Research Methodology To gather data for this report, ESG conducted a comprehensive online survey of IT professionals and application developers from private- and public-sector organizations in North America (United States and Canada) between October 28, 2014 and November 6, To qualify for this survey, respondents were required to be responsible for evaluating and/or selecting software application development and deployment tools for their organizations. Respondents who were not involved in the decision making process for software application and development tools were disqualified. Respondents were also required to be 26 years old or older and have 2 or more years of experience working in an IT or application development role. If a respondent was under 26 years old or had less than 2 years work experience in an IT or application development role, she was disqualified. All respondents were provided an incentive to complete the survey in the form of cash awards and/or cash equivalents. After filtering out unqualified respondents, removing duplicate responses, and screening the remaining completed responses (on a number of criteria) for data integrity, we were left with a final total sample of 326 IT professionals and application developers who either were currently using PaaS or intended to use PaaS within the next 2 years. Please see the Respondent Demographics section of this report for more information on these respondents. Note: Totals in figures and tables throughout this report may not add up to 100% due to rounding. Platform-as-a-service Feature Analysis A key objective of this survey was to understand what PaaS features were most important to developers, IT managers, PaaS users, and PaaS evaluators. ESG identified 20 features spanning four categories that ESG wanted to test. In order to obtain a precise understanding of the relative importance of these 20 features, conventional survey-based research approaches would fall short of providing the necessary resolution. The approach ESG used was to leverage a form of discrete choice analysis called maximum-difference analysis or max-diff. Max-diff uses a combination of multi-scenarios questions and instructions to the respondent to select the most important and least important feature for each scenario. In this survey, the 20 features were presented in 15 questions or scenarios. Each scenario showed the respondent four features for which they were to select the most important feature and least important feature. The scenarios were randomly generated and each feature was shown three times, each time with different accompanying features. Max-diff requires a special run-time engine to calculate results. The engine is able to score and rank order all 20 features for each respondent. The scoring process for each respondent takes into account both the respondent s choice and then normalizes these choices relative to all of the other choices made by the other respondents. The result is a set of preference scores for each feature for each respondent. Scores are calibrated so that 100 is the exact center of the distribution. Therefore, a score of 100 denotes average importance. Preference scores are linear so that a feature with a score of 125 is 25% more important than the average. Likewise, a feature with a score of 75 is 25% less important than the average. This report presents mean preference scores for developers, IT managers, PaaS users, PaaS evaluators, and by respondent age group. All mean scores are shown as their percent above or below average importance. This makes it easier to identify and understand the relative importance of each feature. Most mean scores fall between 150 and 50. Consequently, a score of 150 (or +50%) is very close to being the most important feature and a score of 50 (or -50%) is very close to being the least important feature. It is important to recognize that this approach to understanding feature importance is based on a relative scale keyed off the average importance across all 20 features. Due to the relative nature of the comparison, ESG cannot say that the least important feature is not important. Likewise, the most important features are most important in comparison to the other features tested but may not be most important when compared to other features outside of the 20 that ESG tested.

4 PaaS Features Defined The following 20 PaaS features were evaluated to determine developer, IT manager, PaaS user, and PaaS evaluator preference: 1. High productivity development environments (model-driven, visual development, or BPM style) 2. High control development environments (code-centric development, C#, Java, Java EE, JavaScript, PHP, Ruby) 3. A mobile-first and multi-channel development strategy 4. Integration support (robust APIs, ESB, SOA, connectors) 5. Support for database, caching, and big data 6. Vendor provides hybrid (both private and public) PaaS capabilities 7. Vendor provides an application marketplace 8. Vendor hosts a vibrant developer network 9. Vendor provides free and fee-based product support options 10. Vendor has a proven track record supporting large customers and complex application deployments 11. Vendor has strong open source ties 12. Team development (repository, check in/out, versioning, collaboration) 13. Testing tools (unit, system, load, device) 14. Continuous delivery and integration tools 15. Enterprise worthiness (disaster recovery, high availability, dynamic scalability, high performance VM options, automated updates with zero downtime) 16. Proactive application management (performance optimization and automated health management) 17. Enterprise security (user authentication, user authorization, encryption) 18. Proven reliability and scalability (access to 000s of VMs, 99.5% or better SLA, global access, auto scaling) 19. Back-end-as-a-service (BaaS) / (Mobile back-end-as-a-service (MBaaS) capabilities (like those from Parse, Kenvy, Firebase) 20. Specific control over the underlying IaaS resources in order to tune/optimize system performance Feature Categories A total of 20 PaaS features were analyzed. These features spanned 4 categories that included the following: Development features. Features that are closely tied to the act of developing applications (high control development environment, high productivity development environment, integration capabilities, mobile development capabilities, and data-centric features) Vendor features. Vendor specific features (track record, customer support, reliance on open source) and depth of PaaS support (hybrid availability, developer network, application marketplace) Lifecycle features. Features that help manage application development and deployment (team development, application testing, continuous integration, and continuous delivery) Operations features. Features to ensure the ongoing management of the application when it is in production (enterprise security, disaster recovery, high availability, reliability, scalability, health management, IaaS configurability, and back-end-as-a-service)

5 Respondent Categories PaaS features were analyzed across five respondent segments: Developers. All respondents who select a developer role. N=176 IT managers. All respondents who select an IT manager role. N=150 PaaS users. Respondents who work for an enterprise that is currently using a PaaS. N=227 PaaS evaluators. Respondents who work for an enterprise that is not currently using a PaaS but intends to use a PaaS within the next two years. N=99 Respondent age group. Three age groups were tested: years old, years old, and 46 or more years old.

6 Respondent Demographics The data presented in this report is based on a survey of 326 qualified respondents. Figure 1 - Figure 8 detail the demographics of the respondent base, including individual respondents current job role/title and years of experience, as well as respondent organizations total number of employees, primary industry, and annual revenue. Respondents by Job Role or Title Respondents current job role or title is shown in Figure 1. Figure 1. Survey Respondents, by Job Role/Title Which of the following best (i.e., most closely) describes your primary job role or title within your organization? (Percent of respondents, N=326, one response accepted) Manager of Information Technology Chief Technology Officer Director of Information Technology Director of Application Development Chief Information Officer Software Engineer Manager of Application Development Senior Software Engineer Vice President of Application Development Vice President of Information Technology Enterprise Architect Computer Systems Manager 2% 2% 4% 4% 5% 7% 8% 9% 11% 13% 16% 19% 0% 5% 10% 15% 20% Respondents by Level of Purchasing Involvement Respondents involvement in selecting software application development/deployment tools is shown in Figure 2. Figure 2. Survey Respondents, by Role in Selecting Software Application Development and Deployment Tools 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Which of the following best describes your involvement in evaluating and/or selecting software application development and deployment tools for your organization? (Percent of respondents, N=326) 79% 19% I am a key decision maker I directly influence the decision I do not directly influence the decision but do provide feedback necessary to the decision makers 2%

7 Respondents by Age The age of survey respondents is show in Figure 3. Figure 3. Survey Respondents, by Age Please select your age group. (Percent of respondents, N=326) 46 to 55, 21% Over 55, 10% 26 to 35, 35% Respondents by Years of IT or Application Development Experience 36 to 45, 34% The number of years respondents have been working in an IT or application development role is shown in Figure 4. Figure 4. Survey Respondents, Number of Years Working in an IT or Application Development Role For how long have you been working in an IT or application development role? (Percent of respondents, N=326) More than 20 years, 16% 2 to 5 years, 7% 11 to 20 years, 31% 6 to 10 years, 46%

8 Respondents by Number of Employees The number of employees in respondents organizations is shown in Figure 5. Figure 5. Survey Respondents, by Total Number of Employees Worldwide How many total employees does your organization have worldwide? (Percent of respondents, N=326) 10,000 to 19,999, 7% 20,000 or more, 10% 50 to 99, 3% 100 to 249, 7% 250 to 499, 10% 5,000 to 9,999, 8% 2,500 to 4,999, 19% 500 to 999, 14% 1,500 to 2,499, 8% 1,000 to 1,499, 14% Respondents by Industry Respondents were asked to identify their organization s primary industry. In total, ESG received completed, qualified respondents from individuals in 19 distinct vertical industries, plus an Other category. Respondents were then grouped into the broader categories shown in Figure 6. Figure 6. Survey Respondents, by Industry What is your organization s primary industry? (Percent of respondents, N=326) Communications & Media, 5% Government (Federal/National, State/Local), 5% Health Care, 9% Other, 18% Business Services (accounting, consulting, legal, etc.), 9% Financial (banking, securities, insurance), 21% Retail/Wholesale, 14% Manufacturing, 18%

9 Respondents by Annual Revenue Respondent organizations annual revenue is shown in Figure 7. Figure 7. Survey Respondents, by Industry 20% 18% 16% 14% 12% 10% 8% 6% 4% 2% 0% 7% Less than $50 million What is your organization s approximate total annual revenue ($US)? (Percent of respondents, N=326) 18% 16% 16% 10% $50 million to $ million $100 million$500 million to $ to $ million million $1 billion to $4.999 billion 12% 12% $5 billion to $9.999 billion $10 billion to $ billion 6% $20 billion or more 2% Not applicable (e.g., public sector, nonprofit) Respondents by Length of Organizations Existence The length of time respondents employers have been in existence is shown in Figure 8. Figure 8. Survey Respondents, by Length of Organizations Existence For approximately how long has your current employer been in existence? (Percent of respondents, N=326) More than 50 years, 19% 1 to 5 years, 2% 6 to 10 years, 13% 11 to 20 years, 33% 21 to 50 years, 33%

10 Contents List of Figures... 3 Executive Summary... 4 Report Conclusions... 4 Introduction... 5 Research Objectives... 5 Research Findings... 6 PaaS Feature Preference Trends... 6 IT Managers Drive Agenda for PaaS Deployment Features Age Drives Major Differences in PaaS Feature Importance Conclusion Research Implications for Technology Vendors Research Implications for IT Professionals Research Methodology Platform-as-a-service Feature Analysis PaaS Features Defined Feature Categories Respondent Categories Respondent Demographics Respondents by Job Role or Title Respondents by Level of Purchasing Involvement Respondents by Age Respondents by Years of IT or Application Development Experience Respondents by Number of Employees Respondents by Industry Respondents by Annual Revenue Respondents by Length of Organizations Existence... 25

11 List of Figures Figure 1. Platform-as-a-service (PaaS) Features in Order of Importance... 6 Figure 2. Platform-as-a-service Development Features: Comparing Users and Evaluators... 7 Figure 3. Platform-as-a-service Vendor Features: Comparing Users and Evaluators... 8 Figure 4. Platform-as-a-service Lifecycle Features: Comparing Users and Evaluators... 9 Figure 5. Platform-as-a-service Operations Features: Comparing Users and Evaluators Figure 6. Platform-as-a-service Development Features: Comparing Developers and IT Managers Figure 7. Platform-as-a-service Vendor Features: Comparing Developers and IT Managers Figure 8. Platform-as-a-service Lifecycle Features: Comparing Developers and IT Managers Figure 9. Platform-as-a-service Operations Features: Comparing Developers and IT Managers Figure 10. Platform-as-a-service Development Feature Preference by Age of Respondent Figure 11. Platform-as-a-service Vendor Feature Preference by Age of Respondent Figure 12. Platform-as-a-service Lifecycle Feature Preference by Age of Respondent Figure 13. Platform-as-a-service Operational Feature Preference by Age Figure 14. Survey Respondents, by Job Role/Title Figure 15. Survey Respondents, by Role in Selecting Software Application Development and Deployment Tools. 22 Figure 16. Survey Respondents, by Age Figure 17. Survey Respondents, Number of Years Working in an IT or Application Development Role Figure 18. Survey Respondents, by Total Number of Employees Worldwide Figure 19. Survey Respondents, by Industry Figure 20. Survey Respondents, by Industry Figure 21. Survey Respondents, by Length of Organizations Existence All trademark names are property of their respective companies. Information contained in this publication has been obtained by sources The Enterprise Strategy Group (ESG) considers to be reliable but is not warranted by ESG. This publication may contain opinions of ESG, which are subject to change from time to time. This publication is copyrighted by The Enterprise Strategy Group, Inc. Any reproduction or redistribution of this publication, in whole or in part, whether in hard-copy format, electronically, or otherwise to persons not authorized to receive it, without the express consent of The Enterprise Strategy Group, Inc., is in violation of U.S. copyright law and will be subject to an action for civil damages and, if applicable, criminal prosecution. Should you have any questions, please contact ESG Client Relations at

12 20 Asylum Street Milford, MA Tel: Fax: