Integrating Data to break down silos

Transcription

1

2 Integrating Data to break down silos Randall Brugeaud Deputy Australian Statistician Australian Bureau of Statistics March 2018, QT Hotel Canberra

3 Presentation purpose To demonstrate how the Australian Bureau of Statistics is working with customers and partners to integrate data and deliver insights that were not previously possible 1

4 Outline 1. Context 2. Process 3. Key assets 4. Impact 5. Risks and Issues 6. Questions 2

5 Context Data Sources Diverse and increasingly complex set of internal and external data sources Combination of Survey/Census and Administrative data Many different formats, methodologies, governance arrangements, legislative and privacy requirements Health Welfare Finance Education Property Employment Taxation Culture Labour Environment & Agriculture Indigenous Geospatial 3

6 Context Providers and Users Providers and users are equally diverse We work with our customers and partners to deliver respond to their needs and deliver the best possible outcome Providers Federal Government State Government Private Enterprise Users Academics Non - Government Organisations Not for Profit Organisations Individuals 4

7 Process ABS data integration lifecycle Authorised under the Census and Statistics Act and/or Statistics Determination Operates within the Five Safes Framework Direct identifiers are removed Access Assemble Problem Definition Information Core Link Design Share Collect Census, PIT, Medicare claims, Tax declarations ABS acquires under law Custodians disclose under law Work with customers and partners define and/or understand policy and research questions Also work together to define the right data product/tool mix Without co-design, we would be building supply rather than demand-driven datasets Desired outcome is to inform important decisions 5

8 Process Comparison with national security Access Assemble Problem Definition Information Core Information Core Link National / International Intelligence Law Enforcement Partners Private Enterprise & Industry Partners Design Collect Share Covert collection Transactional Data Community Also informs important (sometimes life and death) decisions Differences include, The question is not always clear at the outset The data is often very messy Insights need to be timely (often in real-time) Analysis is often built up gradually over time Predictive capabilities are critical Collection and usage provisions are very different 6

9 Key assets MADIP A cross-portfolio project linking nationally important datasets to maximise the value of existing public data for policy analysis, research, and statistical purposes Multi-Agency Data Integration Project MADIP Data Themes MADIP Parties MADIP Uses Education Income & Taxation Families & Households Australian Bureau of Statistics Australian Taxation Office Department of Health Department of Human Services Department of Social Services Answer policy questions Program evaluation Empirical research on socio-economic issues No identification of individuals Health Social Support Department of Education and Training 7

10 Key assets BLADE An enduring statistical asset containing detailed information on the characteristics and finances of Australian businesses Business Longitudinal Analysis Data Environment (full and truncated) BLADE Uses Partnership between the: Australian Bureau of Statistics Australian Taxation Office Department of Industry, Innovation, and Science Integrates survey and administrative data on Australian businesses, at the firm level. Allows firm level analysis of microeconomic drivers of performance, competitiveness, and productivity. Answer complex policy questions Program evaluation Empirical research on theoretical issues, e.g. to understand: Business demography Firm performance and dynamics Firm characteristics 8

11 Key assets ACLD Rich longitudinal resource allowing people to understand how Australian society is changing over time Australian Census Longitudinal Dataset Integrates a 5% sample from the 2006, 2011 and 2016 Censuses (1 million people) Provides the capacity to draw insights on social and economic change Cost effective way to obtain detailed information on small/targeted groups Previous ACLD release used by 1,200 analysts from 80+ organisations Future Census collections will continue to add value 9

12 Impact How breaking down silos helps We can now answer questions we couldn t previously How are socio-economic factors impacting Medicare benefits? What effect are changes to the minimum and award wages having on the labour market? How are payroll tax threshold changes impacting on jobs and growth? Can the occupational characteristics of entrants into long-term welfare enable better informed early interventions? What can we learn from Australians journeys through life? 10

13 Impact Unemployed persons 11

14 Impact Year 11 and 12 students 12

15 Impact Education and retirement These findings demonstrate how the better use of existing public data can help guide education and industry policies to improve the wellbeing of older Australians 13

16 Impact Income and health services These findings demonstrate how the better use of existing public data can improve our understanding of the socioeconomic differences of Australians accessing health care services and inform policy which will help to improve access to these services 14

17 Risks and Issues Privacy and Security Managing privacy and security through the application of the Five Safes Framework which provides the technical, legislative and management controls Social license Maintaining trust and describing the value proposition to providers and users to ensure their ongoing support Accuracy Ensuring that information is representative by using cleansing, processing and weighting techniques to adjust for errors Relevance Helping subject matter areas maintain strong relationships with partners, providers and users and develop fit for purpose solutions Skills Resourcing teams with rare and in-demand skills through training, resource sharing and targeted recruitment 15

18 Questions? 16

19 Backup The Five Safes Framework Safe People Is the researcher authorised to access and use the data appropriately? Are the statistical results non-disclosive? Safe Output Safe Project Is the data to be used for an appropriate purpose? Has appropriate and sufficient protection been applied to the data? Safe Data Safe Setting Does the access environment prevent unauthorised use? Secondment Access to unit record microdata to employees (not contractors) of government agencies (Cth, State, Territory) To assist in carrying out of the functions of the Statistician ABS DataLab Provides access to de-identified microdata within the ABS secure IT environment, physically at ABS offices and remotely Can be used with a variety of statistical software, including SAS, R, and STATA All analysis results are checked by ABS output checkers before release Applied Programming Interface (API) Collaboration between ABS and Data61 Customised aggregate data that is confidentialised on the fly Available through applications and online (NationalMap) 17