A System of Tribes, Squads and Shared Services

Transcription

1 A System of Tribes, Squads and Shared Services

2 Optimize delivery speed within agreed stability constraints. Rob Tuley engineering.skybettingandgaming.com

3 My job is a feedback loop... Focus on increasing delivery speed to realise more value Risk of service problems likely to increase Low risk of service problems outside agreed boundaries Focus on service risk reduction to maintain stability

4 I work with a Head of Product: a positive but potentially dangerous feedback loop I think we should get this feature live. Healthy tension leads to +ve middle ground I think we should reduce service risk.

5 Unchecked reinforcing loop...

6 Small, autonomous squads using a pull workflow who are provided with appropriate tools.

7 Small, autonomous squads using a pull workflow who are provided with appropriate tools.

8

9

10

11

12 4 weeks

13 Inspired & Evolve from Spotify paper...

14 This is our system... Gaming Tribe Account Tribe Trading Tribe Sportsbook Tribe Gaming P&L Data Tribe Bet P&L Platform Services Infrastructure Tribe Service Ops Central Technology L

15 Small, autonomous squads using a pull workflow who are provided with appropriate tools.

16 What does autonomy actually mean for us? Accountability & ownership of the team mission. Clear cost-benefit trade-offs within boundaries rather than total freedom. Provide choice on dependencies.

17 Accountability & ownership of the mission. Focus on increasing application stability so not woken at 3am Gap between actual & acceptable level of 3am on-call support wake ups. Acceptable frequency of 3am wake-up Number of service incidents & thus callouts decrease. Shorten & Amplify feedback loops 3 ways of DevOps

18 Clear cost-benefit trade-offs within boundaries rather than total freedom.

19 Provide choice on dependencies. One team at least will forge the way and accept higher cost of introducing alternative. Gap between needs and service provided by existing tools/service. An alternative tool/service now exists.

20 Small, autonomous squads using a pull workflow who are provided with appropriate tools.

21 Pull Workflow Kanban or Scrum, or a mixture. Well instrumented: key metrics cycle time, throughput, redo work and delivery efficiency. Classes of Service useful team health check & governance mechanism.

22 e.g.

23 System effect: gaming the metrics... Don t move that back on the board, it ll affect our redo stats Does measuring and sharing throughput as a story count incentivise smaller stories? Do we want that anyway?

24 Local team system drives higher order needs e.g. release contention/automation is a common problem for us.

25 Small, autonomous squads using a pull workflow who are provided with appropriate tools.

26

27 At our scale delivery efficiency teams provide value. Platform Services Tribe. Delivery Engineering in-tribe.

28 Platform Control deploy pipeline.

29 Tools making it easier affects the system... Number of code repos

30 Shared services are a critical part of the system... Gaming Tribe Account Tribe Trading Tribe Sportsbook Tribe Gaming P&L Data Tribe Platform Services Bet P&L Infrastructure Tribe Service Ops Central Technology L

31 Achieving shared services or objectives Product Tribe A Product Tribe B Service Tribe C

32 Follow My Leader support transitioned to shared service team C: Support & Evolve A: Design, Use & Support A: Use B: Use, Following A e.g. Splunk was introduced by Bet Tribe during mobile sportsbook build. Usage demonstrated value of structured log analysis and Account & Gaming started using. Transitioned to shared Platform Services who better equipped to manage it.

33 Fork & Evolve A: Design, Use & Support design forked and proceeds independently B: Modify, Use & Support e.g. Tiny PHP toolset originates in Bet Tribe. Gaming need similar for Casino, so kickstart by forking to Pico and progress independently.

34 Monopoly A: Use C: Design, Evolve & Support B: Use e.g. Account & Infrastructure services. Currently breaking the infrastructure monopoly by investment in AWS, whose other services are also providing helpful competition back to local tooling.

35 Celebrate Diversity C: Design, Evolve, Use, Support shared learning A: Design, Evolve, Use, Support B: Design, Evolve, Use, Support e.g. datastore technology: Bet chose Mongo; Account chose Couchbase based on learnings from Bet; Gaming prefer MySQL.

36 Eventual Consistency C: Use shared learning A: Design, Evolve, Use, Support B: Design, Evolve, Use, Support B: Use e.g. Internal text comms; various groups try new solutions, some stick but eventually consistent. e.g. IRC, Skype, Hipchat, eventually all to Slack.

37 Central Kickstart handover A: Evolve, Use, Support C: Design, Evolve, Use, Support handover B: Evolve, Use, Support e.g. our DR capability had a poor RTO across the board due to lack of infra automation. Central team tasked with reaching a threshold capability and then transitioned responsibility to tribes to progress.

38 Grow to Devolve A: Use C: Design, Evolve, Support B: Use A: Evolve, Use, Support B: Evolve, Use, Support E.g. our data insight team used to be one central team. As it grew we devolved that service into small teams in multiple different tribes.

39 Co-operative commonly a cross-tribe working group formed to drive progress. A: Design, Evolve, Use, Support C: Design, Evolve, Use, Support B: Design, Evolve, Use, Support e.g. Platform Split programme to separate cross-tribe shared hosts. A cross-tribe working group was formed and worked on the programme together.

40 A system of small, autonomous squads using a pull workflow who are provided with appropriate tools.

41 is a complicated but fascinating thing.

42 Questions? engineering.skybettingandgaming.com skybetcareers.com