Importance of data. Jay Daley, ICANN Helsinki 2016

Transcription

1 Importance of data Jay Daley, ICANN Helsinki 2016

2 Why is.nz presenting? Small registry - Big data pioneer 22 server Hadoop cluster installed in 2012 Full packet capture of DNS servers Data science oriented research team 18% of total staff World class registrar data portal Main business priority for new development Actively building data products Recent success with broadbandmap.nz Open data portal for.nz data 2

3 Case study 3

4 Why data matters Evidence based policy Organisational/community development Cleaner and safer DNS Business - more, new, better Public trust 4

5 Evidence based policy 5

6 Current use of data in policy Already some excellent examples RSTEP IANA SLE approximations by Marc Blanchet But data is not public and so No reproducibility Agenda set by those who pay the analysts Very low throughput of research Some people have their own data Gives them real power in the debate e.g. Root server operators and WPAD debate data is the new oil Open Data means Open Debate 6

7 Case study hoarding? Number of registrants by portfolio size Number of registrants Portfolio size 7

8 Organisational development 8

9 Modern business principle Openness and transparency Applied to Diversity, remuneration, expenses, etc Significant organisational benefits Reinforces best behaviour Create culture of community/customer audit 9

10 Case study Travel funding ICANN funds travel for some attendees Until recently this data was hard to use Only published in PDFs Some meetings missing Names spelled inconsistently No summaries or multi-meeting analysis At Dublin meeting spent several hours Extracting data from PDFs Tidying up names Creating reports Publishing data 10

11 Example PDF 11

12 Example output Average travel funds received by Number of meetings attended $14,000 $12,000 $10,000 $8,000 $6,000 $4,000 $2,000 $

13 Example publication 13

14 Case study Diversity Report from AFNIC on ICANN Diversity Based on public data but not easy! 14

15 Example output 15

16 Cleaner and safer DNS 16

17 Very active area Highly developed use of data Multiple research teams, cooperative forums, NFP services and commercial providers Multiple data resources, extensive data sharing Tools: Entrada, Turing, hadoop-pcap, zonemaster Data collection source evidence Passive monitoring (e.g. DNSDB, PassiveTotal) Hand produced by threat researchers Strong sharing culture via data feeds 40+ feeds available (both NFP and commercial) Track domains, IPs, credentials, URLs, etc 17

18 Case study threat sharing Data shared with cooperative forum then shared with registrars 18

19 Business more, new, better 19

20 Data driven business Market intelligence Targeting marketing New products, same customers New products, new customers 20

21 Market intelligence - basic Registrants prefer shorter names right? Number of domain names by number of characters

22 Market intelligence - advanced Domain name categorisation by industry S Other Services R Arts and Recreation Services Q Health Care and Social Assistance P Education and Training O Public Administration and Safety N Administrative and Support Services M Professional, Scientific and Technical Services L Rental, Hiring and Real Estate Services K Finance and insurance J Information Media and Telecommunications I Transport and storage H Accommodation, Food Services G Retail trade F Wholesale trade E Construction D Electricity, gas and water supply C Manufacturing B Mining A Agriculture, forestry, fishing and hunting 0% 5% 10% 15% 20% 25% 30% 35% Registry Registrar 22

23 Case study - registrar portal 23

24 Targeted marketing Domains most likely to renew for 10 years Those in top 20% by observed traffic Domains in danger of cancelling No MX record TLD cross-sell opportunities Simple data matching Industry verticals Machine learning classifier using web site text Expiring domains Valued by algorithm 24

25 Case study expiring domains 25

26 New products, same customers Domain name popularity 26

27 New products, new customers SaaS product market sizing Specific DNS record indicators for each product Counted by regular zone scans Data sold for competitor analysis 27

28 Public trust Add Presentation Name June 26,

29 Open data and public trust Sunlight is the best disinfectant Open data can prove: Effectiveness of markets and regulators Social benefit Where problems remain 29

30 Case study Web Index 30

31 Case Study - CCT Competition, Consumer Trust and Consumer Choice (CCT) Metrics Reporting So near but yet so far 31

32 Case study - ISOC 204 datasets only one from ICANN (NRO)! 32

33 Taking this to the next level 33

34 Toes in the water ICANN F17 Strategic Plan includes: Deploy automated systems to collect data and compute ratio of registered domain names to active IP addresses. Deploy automated systems to collect data and compute ratio of registered domain names to Internet users regionally and globally. Publish analyses of data collected, implications of changes in data over time, and other topics relevant to the use of unique identifiers and evolution of identifier technologies Document growth in ratios in developing regions 34

35 A goal to consider Data enabled community & industry Data openly shared by all market participants Data is easy to find and use and can be trusted Strong community use of data Authoritative publications ICANN Industry Report Domain Names Contribution to Society Index 35

36 Steps to make it happen Commit employ a Chief Data Officer Begin cultural change Broaden the principle of openness to include data Set the vision of the benefits Engage community social license Community expectations of openness vs privacy Put legal framework in place Adjust contracts, policies and processes to support open data Determine privacy protection rules 36

37 Thanks Any questions? Contact: