Introduction to using data to support decision making in public policy
Meghan Stephens, Principal Advisor, Analytics and Insights, New Zealand Treasury
Principal Advisor in the Analytics and Insights team in Treasury, currently leading work on the Tax and Welfare Analysis model. Extensive experience in statistical modelling and data analysis informing government policies, particularly in reducing child poverty and shaping tax and welfare reforms. Previous roles also involved transforming intricate data into actionable insights in areas such as weather forecasting and energy consumption.
Jason Timmins, Principal Advisor, New Zealand Productivity Commission
Worked in research and evaluation teams across economics and public policy private and public institutions, including Motu Economic Research, Department of Labour, Ministry of Business, Innovation and Employment, Social Policy, Evaluation and Research Unit, the Education Review Office, and the Productivity Commission.
Created and taught the Masters of Analytics course: Public Policy Analytics: Methodology, Design and Data at Massey University, 2017 to 2019
Evidence is important to prevent policies being developed based on gut instinct, ideology, anecdote, or popular belief. As the costs of gathering and analysing data fall, the availability of data-based evidence is growing. For example, the last decade has seen much greater use of government administration data to increase the evidence base to inform policy decisions.
Having more data and evidence is not enough on its own to improve the quality of policy advice, you need to know how to effectively use data and the evidence it creates. To make good use of evidence you need to know the best types and sources of data and evidence to draw from, any problems associated with those sources, how to select and use the most relevant evidence, and how to communicate decision makers.
The course will introduce students to using data and evidence in public policy. You will learn about how data and evidence can be used to inform different parts of the policy process, how it is created and what types of questions it is most appropriate for. The course will then take you through how to practically apply data and evidence when formulating policy advice.
On completion of this course, participants will:
· Understand what types of data and evidence are needed at different points in the policy decision making process.
· Learn about the different types of quantitative data and evidence that can be used to inform policy development.
· Learn how to use data and evidence to develop policy advice and costings.
The course is for people who want to learn how to use data and evidence in their policy work. This could include people in policy teams or staff involved in delivery of public services.
This is not a statistics course. It will be focused on learning how to use data and evidence, and not how to analyse data and create evidence.
The course is divided into three parts:
Part 1: Learn when to use data and evidence in the policy process.
The module will look at using data and evidence to identify and quantify a problem, estimate the impact of different interventions, and monitor and evaluate progress of a policy intervention. The module will include examples of how data and evidence is used to inform policy processes, including regulatory impact analysis, cost benefit analysis, and intervention logic models.
Part 2: Learn about the different types of data and evidence that can be used to inform policy development.
The second part of the course will build your understanding of the strengths and limitations of different types of data and evidence, including how data is created (e.g., different ways of measuring and collecting contextual data), and how it can be analysed to create research evidence. For example, the best types of data and evidence for evaluating the impact of different behavioural changes on a policy outcome and the most appropriate method for measuring the effectiveness of a policy solution.
Part 3: Learn how to use data and evidence to develop policy advice and costings.
The final part of the course will help you learn how to assess limitations of data and quantitative evidence (eg, gaps and uncertainties), the importance of documenting key decisions and assumptions, the advantages and disadvantages of using simple calculations over complex calculations, and what makes good quality assurance. The course will end with tips for how to present data and evidence in policy advice, including data visualisation, communicating risk, and describing the impacts of key assumptions and data limitations.
Session outline and delivery
· Session 1: Part 1
· Session 2: Part 2
· Session 3: Part 2/3
· Session 4: Part 3 continued
Course specific information
Is there homework? The course will include a research example that students can use to cement their learning.
Is there a group element to the course? The course will include class exercises that will involve students working in groups.
Is there pre-reading? Not essential, but we will identify a few texts to introduce the course and for further reading:
· Superu (2018), Making sense of evidence: A guide to using evidence in policy, Social Policy Evaluation and Research Unit, https://thehub.swa.govt.nz/resources/making-sense-of-evidence-a-guide-to-using-evidence-in-policy/
Is there a course handbook? No, but we will share the class slides, which will include notes.
Is there any assessment involved? The course will include a small project for students to complete that will based on the research example used during the course.
Teaching and Learning Methods:
This will be an introductory course that is based on a semester course that involved 16 hours of teaching. The course will:
· use lecture style teaching and powerpoint slides to provide an overview of a topic, including explaining key concepts using real world examples;
· small group work to help explore and embed key concepts, such as working out how to measure homelessness;
· class discussion time for students to bring examples and questions from their own work.