Michael Koester. Sr. Product Marketing Manager, Visual Studio Online, ALM & DevOps

Size: px

Start display at page:

Download "Michael Koester. Sr. Product Marketing Manager, Visual Studio Online, ALM & DevOps"

William Shaw
5 years ago
Views:

2 Michael Koester Sr. Product Marketing Manager, Visual Studio Online, ALM & DevOps Part of the Visual Studio product team and a developer technologies specialist Based in Frankfurt, Germany (it s the airport ) Worked for Microsoft for 12 years in a number of different roles: Program Management, Product Management, Sales, Marketing, Consulting

3 The end-to-end tools, processes & culture enabling a software team to turn a customer scenario into deployed product code and improve it through a continuous learning system Why does this matter? Agile and DevOps practices are changing the way teams develop software An organization s ability to deliver quickly and reliably is a key competitive differentiator Being responsive means understanding what s going on in a deployed application The engineering system is central to developer productivity and job satisfaction

4 Tools Process Culture

5 Practicing Build, Measure, Learn Adopting the best of OSS Focus on quality Change one thing change everything!

6 Scheduling We did ask for feedback after each milestone planning, Beta, RTM We did find bugs with this process and fix them - no problems there But we couldn t react to anything customers using the product were telling us For the most part, we would tell everybody sorry and push things to the next release Schedule Planning Code Test & Stabilize Code Test & Stabilize 2,5 years? Beta? RTM

7 Our planning process has changed - from long upfront planning cycles to shorter build, measure, learn cycles with continuous alignment to our long term vision

10 W O R K S F O R E V E R Y O N E M O D E R N C O D I N G M A N A G I N G T E C H N I C A L D E B T F I R S T P A R T Y I S T H I R D P A R T Y A G I L E A T S C A L E C O N T I N U O U S D E L I V E R Y A C T I O N A B L E I N S I G H T S

11 Visual Studio 2015 Enterprise Agile

12 Our engineering system is hosted on Visual Studio Online: The same Cloud service we offer to our customers All new code is deployed to SU0 (our own branch) first - before our customers get it

13 Shared Platform Services San Antonio VSO SU1 VSO SU0 VSO SU5 Chicago San Antonio Amsterdam

15 All code is deployed, but feature flags control exposure Flags provide runtime control down to individual user Users can be added or removed with no redeployment Enables dark launch Mechanism for progressive experimentation & refinement

18 Previous project creation experience Too many actions and distractions can prevent customers from taking the next natural step of creating a new project

19 Web IDE Focused project creation experience - integrated into the IDE flow

20 % of accounts creating projects the day of account creation January 22 to February 11 (all sources) % of accounts creating projects the day of account creation January 22 to February 5 (IDE only) A second experiment, running in parallel, invalidated results First project creation jumps from 15% to 50% First project creation from IDE jumps from 3% to 20%

23 Activity Logging Traces KPI Metrics Job History Customer Intelligence Synthetic Perf Counters Dependent Services Operational Intelligence Business Intelligence Customer Intelligence

24 Activity Logging Traces KPI Metrics Job History Customer Intelligence Synthetic Perf Counters Dependent Services Operational Intelligence Business Intelligence Customer Intelligence

25 Activity Logging Traces KPI Metrics Job History Customer Intelligence Synthetic Perf Counters Dependent Services Operational Intelligence Business Intelligence Customer Intelligence

26 Activity Logging Traces KPI Metrics Job History Customer Intelligence Synthetic Perf Counters Dependent Services Operational Intelligence Business Intelligence Customer Intelligence

27 Activity Logging Traces KPI Metrics Job History Customer Intelligence Synthetic Perf Counters Dependent Services Operational Intelligence Business Intelligence Customer Intelligence

28 Activity Logging Traces KPI Metrics Job History Customer Intelligence Synthetic Perf Counters Dependent Services Operational Intelligence Business Intelligence Customer Intelligence

29 Front-doors, conversion funnel, engagement, top customers Input metrics vs. output metrics Watch out for vanity metrics!

30 Visual Studio 2015 Application Insights

31 We are bringing the benefits of open source and services culture to our internal development workflow while maintaining the rigors of enterprise assets A transition of this magnitude will take time, but we re already well on our way!

33 source across the company is available to anyone any developer can offer improvements to anything in the company IP the company has built up over the years is made of re-usable components anybody can find and potentially re-use components from anywhere else developers are rewarded for creating popular components minimal lag from a developer change to when the rest of the company sees it developers can move to another team and already know how to work products built this way meet the rigors of quality and support our customers expect

34 One code base with multiple delivery streams Single master branch, multiple release branches Shared abstraction layer (VSSF) Update 2 Update 1 Update N

35 Use Be able to understand what it does Can easily pick it up and use Fork/Follow/Favorite Download Answer questions, write blog posts Fix/add documentation Log bugs Fix typos Stick around - influence the direction of the project Become a core committer (get write access) Accept/validate code contributions Nurture new people Contribute time Contribute code Own Actually contribute code patches that fix bugs Contribute entirely new features Maintain platforms Translate

36 Our move to Git Migrated a team of 250+ engineers from TFVC to Git Left history behind no complaints so far! You can never do enough Git training, hands-on workshops is the way to go Git scales reasonably well with file size Using a one-branch model with good results (made feasible by significant investments in speeding up our build time) Pull requests have formalized our code review process

37 Git Starts with the characteristics and workflows for sharing Brings the culture of sharing, reinforces componentization mindset A modern source control system, a popular tool in the industry Investing to build an enterprise class & scale cloud Git service

38 Quality in the Cloud cadence optimizes code velocity and release velocity Change in the engineering roles was needed to enable this shift This doesn t mean our quality bar to customer is lowered - quite the opposite!

39 Ship frequently with minimum friction Dream of a system that can do hands-free push of a hotfix to production in 15 minutes Fast and predictable Inner Loop Inner Loop = Build >Deploy >Test cycle which runs reliably hundreds of times a day Quality approach designed for Cloud cadence Continue to meet a rigorous on-premises quality bar

Product is designed for testability Test code is product code,

40 Principles Tests should be written at the lowest level possible Write once, run anywhere - includes production system Product is designed for testability Test code is product code, only reliable tests survive Testing infrastructure is a shared service

41 New Engineer role combines responsibilities from dev and test Everyone has a new role, everyone is learning new skills Every engineer and team has E2E accountability No more handoffs for testing, stronger emphasis on peer reviews Big cultural shift across the company

42 Visual Studio 2015 IntelliTest

43 Code Complete Planning Code Test & Stabilize Code Test & Stabilize Beta RTM

45 Discover Value Acquire Product Learn Customer Journey Customer Program Manager understands customer needs designs great features Use Features Pay Optimize Value

Discover Value Acquire Product Learn Customer Journey Use Features Pay Customer Program Manager understands customer needs designs great features attracts

46 Discover Value Acquire Product Learn Customer Journey Use Features Pay Customer Program Manager understands customer needs designs great features attracts customers into the funnel describes the product value documents product usage forms business models analyzes and obsesses over customer data Optimize Value

47 Live Site Health/Debt Time to Detect, Time To Mitigate Incident prevention items Aging live site problems Customer support metrics (SLA, MPI, top drivers) Engineering Health/Debt Bug cap per engineer Aging bugs in important categories Pass rate & coverage Velocity Time to build Time to self test Time to deploy Time to learn (Telemetry pipe)

48 We are modernizing the process for documentation, engineering and community to easily create developer content Documentation writing is coming to the Cloud Cadence Building a system and culture that treats Doc as Code

49 It takes a full sprint or release until docs come together and work is only visible to a small set of users and stakeholders Drafts First draft of documentation Review Changes are incorporated Preview Writer publishes to sandbox for preview Go-Live Writer promotes content to live Documentation process Schedule Review Draft doc sent out for review Staging Writer promotes content to staging Published Planning M1 Feature Development M2 Feature Development 1 Result: Specifications 2 Feature Result: Complete

Drafts Specs created, initial docs authored by PMs API docs authored by PM/Engineers before any

refined and updated Each update is just a small increment Review Automated review mail is sent to

Content owner/pm gets a notification and merges Changes Anyone wanting to add a doc change simply

50 Drafts Specs created, initial docs authored by PMs API docs authored by PM/Engineers before any code is written 1 Week Planning Sprint 3 Weeks 2/3 Week Coding Refining Docs continue to get refined and updated Each update is just a small increment Review Automated review mail is sent to the team and stakeholders Docs are automatically published to an internal staging website Merging Content owner/pm gets a notification and merges Changes Anyone wanting to add a doc change simply does a pull request Published Content updated to live branch, automatically pushed to live public website

51 Microsoft is creating One Engineering System (1ES) for the whole company designed for the Cloud Cadence Visual Studio Online is the foundation for this system Our team is building many of its components and leading the change

Summary Internal shared source experience

52 Summary Internal shared source experience An agile work management system Source code management strategy using Git New quality approach Distributed cloud based build system A componentization system We are building all this in Live site management tools Telemetry

What is Continuous Integration. And how do I get there

What is Continuous Integration And how do I get there Related Workshops Introduction to DevOps Transform your Organization with DevOps Concepts DevOps Implementation Boot Camp Comprehensive literacy on