Temple University Japan:

TUJ Distinguished Lecturer Series: Analyzing Rating Data Using a Many-Facet Rasch Measurement Approach

Date: Saturday, October 12th, 2019 Time: 2:00 PM - 5:00 PM

Speaker: Carol Myford (The University of Illinois at Chicago, U.S.A.)

Description:
In performance assessment settings, raters use measurement tools such as rating scales or scoring rubrics to evaluate students' performances or products. Raters assign their ratings based upon information that they gather and synthesize as they examine each performance or product. However, by virtue of their being human, raters sometimes unavoidably introduce error (rater effects) into the assessment process. Data analysts can employ a many-facet Rasch measurement (MFRM) approach for analyzing ratings to detect and measure the impact of rater effects such as leniency/severity, central tendency, and halo effect on the ratings. Using the output from these analyses, administrators in charge of monitoring quality control for assessment systems obtain detailed, practical information about how various "facets" (e.g., students, raters, rating criteria) of their assessment systems are performing. They can use that information to help them determine to what extent their systems are under statistical control and to initiate meaningful changes in an effort to improve their systems.

This seminar will introduce participants to the many-facet Rasch measurement approach for analyzing ratings. Topics we will cover include the following:

  • The role of raters and their influence in open-ended assessments
  • Rater effects that can influence ratings
  • Statistical approaches that researchers have used to analyze rating data and characterize rater performance
  • The conceptual and mathematical foundations of many-facet Rasch measurement
  • Group-level and individual-level questions that a MFRM analysis can answer
  • Creating a data file for a MFRM analysis
  • Preparing a specification file for a MFRM analysis
  • Running MFRM analyses and reformatting output
  • Creating judging plans
  • Making sense of output from MFRM analyses
  • MFRM anchoring procedures for equating
  • Strategies for resolving disconnected subsets issues
  • Conducting bias interaction analyses
  • Hybrid MFRM models and their uses

Prerequisite Skills and Knowledge Needed for Seminar: Participation The target seminar participants are researchers, statisticians, data analysts, and advanced graduate students who want to learn how to use many-facet Rasch measurement for analyzing rating data. Having a basic understanding of the conceptual and mathematical foundations of Rasch measurement would be helpful but not required, as would prior experience using Rasch measurement analysis software (e.g., Winsteps).

Organization: Temple University Japan

Cost: free

Venue: 1-14-29 Taishido, Setagaya-ku, Tokyo, Japan 154-0004 7-minute walk from Sangen-jaya station (Tokyu Den-en-toshi Line) (map)

Location: Tokyo, Tokyo Metropolis, Japan

iCal Add this to iCal
(Need help?)

vCal for Outlook Add to Outlook
(Need help?)

0

You can add this event to your iCal calendar.

  1. Click on the iCal icon. Your iCal software will start.
  2. Click 'Subscribe':
    click subscribe
  3. Under 'Auto Refresh', select 'Every day' in case the the basic details change:
    auto-refresh daily

You can add this event to your Microsoft Outlook calendar.

  1. Click on the MS Outlook icon.
  2. See what happens.
  3. Tell us what happens. I don't have MS Outlook on a Windows computer, so I can't test it.
  4. If you click on the icon and nothing happens, do this:
    1. Right-click on the icon and save the file.
    2. According to Microsoft's support page, in Outlook's File menu, you should click Import and Export.
    3. Click to select Import an iCalendar or vCalendar file (*.vcs), and then click Next.
    4. Click to select the vCalendar file you've just saved, and then click Open.

Contact Temple University Japan

Website: www.tuj.ac.jp/tesol/seminars/

Temple University Japan
Email QR Code:

Phone (work): 03-5441-9800
ABAX