INDIGO Home University of Illinois at Urbana-Champaign logo uic building uic pavilion uic student center

Designing an Evidence-based Assessment of Conceptual Understanding and Misunderstandings in Statistics

Show full item record

Bookmark or cite this item:

Files in this item

File Description Format
PDF Jorion_Natalie.pdf (5MB) (no description provided) PDF
Title: Designing an Evidence-based Assessment of Conceptual Understanding and Misunderstandings in Statistics
Author(s): Jorion, Natalie C.
Advisor(s): Pellegrino, James
Contributor(s): Castro-Superfine, Alison; Martinez, Mara; Yin, Yue; Stout, William
Department / Program: Graduate College
Graduate Major: Learning Sciences
Degree Granting Institution: University of Illinois at Chicago
Degree: PhD, Doctor of Philosophy
Genre: Doctoral
Subject(s): Conceptual understanding misconceptions statistics education evidence-centered design concept inventories assessment validity
Abstract: This study investigates the extent to which an assessment can diagnose learner misconceptions in the domain of statistics. A Statistics Concept Inventory (StatCI) was created using an evidence-centered design framework (Mislevy, Steinberg, & Almond, 2003). This assessment draws from a comprehensive literature review of student thinking in statistics, in which clusters of items correspond to a major concept, and each distractor maps onto a misconception. Professors and graduate students with expertise in statistics checked the assessment for face validity. Four studies were run to validate aspects of this assessment. First, a student protocol study was conducted to examine how students interpreted the items. Second, a 30-item pilot version was administered to 100 participants on Amazon Mechanical Turk. Preliminary psychometric tests were run on these data to identify items to modify and delete. A beta version was administered to another 100 participants. Finally, an updated was administered to 750 participants. Participant performance data was analyzed for response patterns demonstrating conceptual and errorful thinking. In particular, data was analyzed by items, conceptual structure, distractors, and demographic groups. These results provided evidence that the assessment is measuring the targeted constructs and is able to identify learner misconceptions and errors. The outcomes of this program of research included: (1) a design pattern template that can be broadly applied to create other assessments in statistics; (2) a final assessment instrument that can be used in undergraduate first-year statistics courses; (3) a methodology for applying ECD to concept inventory design.
Issue Date: 2016-10-18
Genre: thesis
Date Available in INDIGO: 2016-10-18
Date Deposited: 2016-08

This item appears in the following Collection(s)

Show full item record


Country Code Views
United States of America 60
China 43
Russian Federation 40
Ukraine 5
United Kingdom 4


My Account


Access Key