NSF Funds Research in Statistics Education

To help highlight the high-quality research in statistics education being funded by the National Science Foundation—and specifically the Education and Human Resources (EHR) Directorate—the ASA asked three principal investigators for summaries of their NSF-funded research.

Distribution of Items Across Practice and Process of Statistical Problem Solving Emphases

Levels of Conceptual Understanding in Statistics (LOCUS) project (DRL-1118168)
Tim Jacobbe, University of Florida

The inclusion of statistics in the K–12 curriculum has been gaining momentum over the past 30 years, starting with the efforts of the Quantitative Literacy Project (Schaeffer, 1986) and the influence of that project on the development of the national mathematics standards (NCTM, 1989, 2000). These efforts led to statistics becoming a major presence in the Common Core State Standards in grades 6–12 (National Governors Association Center for Best Practices and Council of Chief State School Officers, 2010).

However, increased expectations regarding students’ understanding of statistical concepts have not resulted in a change in the way statistics is assessed to align with those expectations. The National Science Foundation–funded Levels of Conceptual Understanding in Statistics (LOCUS) project aims to change the way statistics is assessed in the classroom and on high-stakes assessments.

LOCUS has developed a series of assessments designed to measure statistical literacy. The intent of these assessments is to provide teachers, educational leaders, assessment specialists, and researchers with a valid and reliable assessment of statistics consistent with expectations from the field of statistics education. These materials are beginning to serve as exemplars for the development of high-stakes assessments or curriculum materials. LOCUS assesses students’ understanding across levels of development as identified in the Guidelines for Assessment and Instruction in Statistics Education (GAISE) Report: A Pre-K–12 Curriculum Framework. LOCUS addresses the “A” (assessment) component of the framework to provide items that assess statistical literacy in the spirit of GAISE.

The LOCUS assessments were designed to assess statistical literacy developmentally, as defined by the GAISE framework. Given that the goal of statistical literacy is far-reaching, the questions have been successfully used with the following populations: students in grades 6–12 (including AP Statistics); pre-service and in-service teachers; and students enrolled in traditional and randomization-based college-level introductory statistics courses.

Emphasis on Practice and Process of Statistical Problem Solving

• Formulating Statistical Questions & Collecting Data (40%)
• Analyzing Data and Interpreting Results (60%)

LOCUS Assessments

Two Versions of the Assessments

• Beginning/Intermediate Statistical Literacy
• Intermediate/Advanced Statistical Literacy

Two Formats for the Assessment

• Paper-Based: 23 multiple choice questions and 5 constructed response questions (90 minutes)
• Online: 30 multiple choice questions (60 minutes)

Two Forms per Version and Format (Equated Pretest and Post-Test)
Administered to 3,430 students in grades 6–12 with mean stratified alpha of 0.83 across all forms

Register for an account and start using the assessments today.

Contact: Tim Jacobbe, University of Florida

Playing Games with a Purpose: A New Approach to Teaching and Learning Statistics

(2011-2015) NSF TUES DUE #1043814
Shonda Kuiper, Grinnell College

The objective of our NSF Transforming Undergraduate Education in Science, Technology, Engineering, and Mathematics (TUES) grant is to advance statistics education by designing and developing interactive, inquiry-based online games and associated lab materials that simulate data-based decisionmaking in a research-like experience. Instead of teaching statistics and data science as a collection of facts and mathematical calculations, these activities excite the power of innovation and creativity that occurs within great research in any discipline. While games are engaging, the key motivation for our game-based labs is that they allow students to collect their own unique data to make decisions, develop their own research questions, and then tie their conclusions to actual research.

Textbook data sets provided to students are typically carefully vetted and cleaned to illustrate a key statistical topic or method. Rarely are real studies and data so straightforward. We build upon educational research from multiple disciplines by creating modules that bridge the gap from smaller, focused textbook problems to real-world problems. The result is that core statistical issues such as working with messy data, bias, data relevance, and reliability are taught as concepts essential to every study involving data.

To make the materials easily accessible to any type of undergraduate statistics course, a full set of instructor resources is freely available. This includes student handouts, online videos, sample data sets, and instructor guides that discuss the following:

• Level of the material and any needed prerequisites
• Relationship of the games to statistics and data science
• Sample articles that put the research question within a context understandable to first-year college students
• Learning goals
• Details about accessing the free online games
• Expected instructor and student time needed to complete the activity
• Key discussion questions
• Suggestions for future work or class projects students could pursue

Initial evidence shows these game-based labs offer several advantages in quantitative undergraduate courses. While students enjoy the unusual experience of using a game for learning statistics, most students identified the best part of these game-based labs was they helped them learn. Many students are engaged in a unique way and are enthusiastic about the opportunity to pose their own research question and control their own data.

Even though most of our current student labs and associated instructor resources focus on the introductory course, each game is carefully designed to be highly adaptable, allowing for complex multivariate data collection. For example, we have introductory course materials that focus on conducting a two-sample t test. However, the same online game allows students to simultaneously test multiple factors and has been used to teach repeated measures ANOVA and fractional factorial designs in more advanced courses. Thus, these activities can challenge students at any level to “think with data” by having them work with real-world, unstructured data sets and train them to better communicate nuanced statistical ideas.

Principal Investigators are Shonda Kuiper and Rod Sturdivant. Advisors include Ginger Holmes-Rowell, George Cobb, Michelle Everson, Danny Kaplan, Dennis Pearl, Michael Posner, and Frank Wattenberg.

Transforming Undergraduate Education in Science, Technology, Engineering, and Mathematics, Phase II

NSF DUE-1323210
Nathan Tintle, Dordt College

When Fisher (1936) and Pitman (1937) first suggested we should base inferences on permutation distributions, computer simulation was still far in the future. Now, almost 80 years later, this NSF-funded project aims to “broaden the impact and evaluate the effectiveness of recent efforts to use simulation of those distributions to teach the logic of inference in a first statistics course.” Our premise is that the Fisher-Pitman approach offers students a simpler, more direct path to the fundamental logic of inference than the traditional approach by way of the Central Limit Theorem and the t-distribution.

The current project builds on our previous NSF grant (DUE 1140629) to develop complete materials for an algebra-based introductory statistics course using simulation-based inference methods. This follow-up has a three-part focus: workshops, online resources, and assessment.

We are conducting faculty development workshops to encourage teachers (undergraduate, community college, AP, high school, etc.) to think about the possible advantages of a simulation-based curriculum, using active participation to show how this can work in the classroom. Upcoming workshops are scheduled at Penn State in conjunction with USCOTS (May 2015), Hollins University in Roanoke through the MAA’s PREP program (July 2015), and JSM in Seattle (August 2015).

We are developing online resources to support the growing community of adopters. We have a listserv and blog site, where ongoing discussions occur and resources are posted.

We are implementing an ongoing, multi-institutional assessment program to evaluate students’ conceptual understanding and attitudes in courses using simulation-based inference vs. traditional curricula, along with a host of secondary research goals. In this project, we are collaborating with developers of several other simulation-based curricula for introductory statistics to ensure an ecumenical view of current options.

To date, more than 500 instructors have participated in our workshops, and about 3,500 introductory statistics students have participated in the assessment project during the first year of the three-year grant. We expect those numbers will continue to grow over the remaining two years of the project.

As our workshops reach increasing numbers of instructors who are considering using simulation-based curricula, we have ongoing opportunities to share cutting-edge ideas with colleagues, grapple with the challenge of their skepticism, and help them think about how simulation-based inference might work in their classrooms. We continue to add new content to our blog, where experienced users and novice users can wrestle with questions of implementation, curriculum, technology, and more.

Our electronic mailing list currently offers more than 300 participants a place to discuss (in ‘real time’) the key issues from their classrooms with an audience of instructors worldwide. Frequently asked questions from the listserv generate themes for blog posts from a variety of contributors and assessment projects.

Finally, as we begin to analyze assessment data from our first batch of students, we find improved performance on key learning outcomes related to the logic and scope of statistical inference, as well as improved performance of weaker students in courses using simulation-based inference. Upcoming analyses will attempt to pinpoint key curricular, pedagogical, and institutional variables related to improved student performance.

For more information, click here.

Principal investigators include Nathan Tintle, Beth Chance, Dennis Pearl, Soma Roy, and Todd Swanson.

More online

To see EHR funding solicitations, click here.

EDITOR’S NOTE

This material is based on work supported by the National Science Foundation. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.