Recent Changes - Search:

ECD

ACED

Bayesian Networks in Educational Assessment

Cognition and Assessment

PmWiki

pmwiki.org

edit SideBar

BN /

EMSM

Evidence Model/Student Model test

This is a small toy example created to test the evidence model/system model algorithm Almond and Mislevy (1999) Invalid BibTex Entry!; see also Almond et al (1999) Invalid BibTex Entry!. Full details can be found in Invalid BibTex Entry!

Purpose

Almond and Mislevy (1999) noted that the complete model (Motif?) for an assessment could be broken up into pieces: a student model (also called a system model, to make the language not so educationally specific) and a number of evidence models?. As long as the footprint?--the set of student model nodes referenced in the evidence model--formed a clique (or a subset of a clique) in the student model, then the calculations could be distributed across the two fragments by passing messages between the footprint clique in the student model and evidence model subnets.

Mislevy, Almond and Steinberg (1998) Invalid BibTex Entry! commissioned Eddie Herskovitz to add this functionality to Ergo?. This sample network was part of the requirements document.

Proficiency Models

At this point in time, the proficiency model was still called a student model (as we were not yet learning? parameters from data, the distinction was unimportant). As we were also looking at applications outside of educational testing (Almond et al, 1999), we called it a system model (think about diagnosis a problem with a car).

The proficiency model (called system) contained three proficiency variables, Skill 1, Skill 2 and Skill 3. Skill 1 had three levels, the other two skills had two levels each. Skill 2 was a parent of Skill 3, but Skill 1 was a priori independent of the other two.

System (student) model before fill-ins

Note that Evidence Model 1 (below) has a footprint consisting of Skill 1 and Skill 2. To make the algorithm work, an edge must be filled-in between them to make this work. In Netica? this is done by adding a special CliqueNode? as a child of both of variables that must be joined.

System (student) model after fill-ins

Task Models

There aren't really task models per se in this assessment design, because it is all about the Bayes nets, and not a real assessment. The fiction is that there are two task models:

Task 1
has one with two dependent right/wrong observables
Task 2
has a single partial credit observable.

There are two variants of each task.

Note that the variant version of this example maintains this strict hierarchy to illustrate the relationship between evidence models? and links?. In particular, Tasks 1A and 1B are variants of Task 1 and Tasks 2A and 2B are variants of Task 2.

Evidence Models

There are two different evidence models corresponding to the two tasks.

Evidence Model 1

This evidence model is used for Tasks 1A and 1B. There are two observable variables, and two proficiency variables, Skill 1 and Skill 2, are tapped. Note that the footprint? of this evidence model is Skill 1 and Skill 2 is not a subset of a clique? in the proficiency (system) model, so an edge needs to be filled in? in the proficiency model.

Bayes net fragment for Evidence Model 1

The (somewhat) amusing names for the variable are to test the handling of spaces and special characters in the names for observable variables.

Evidence Model 2

This evidence model is used for Tasks 2A and 2B. There is a single observable variables with three possible states, and two proficiency variables, Skill 2 and Skill 2, are tapped. Note that the footprint? of this evidence model is Skill 2 and Skill 3 which is already a clique? in the proficiency (system) model, so this evidence model does not require any fill-in? edges in the proficiency model.

Bayes net fragment for Evidence Model 2

Assembly Model

The complete form for this assessment is Task 1A, Task 1B, Task 2A and Task 2B. The complete motif? graph is shown below.

Complete Bayes net for EMSM example

Data Sets

The complete model for this example can be found at:

http://pluto.coe.fsu.edu/BNinEA/EMSM/ (tarball)

There is a subfolder in this example

http://pluto.coe.fsu.edu/BNinEA/EMSM/LinkModels (tarball)

which illustrates the correspondence between the task model?--task? and evidence model?--link? hierarchies.

References

Invalid BibTex Entry!

Invalid BibTex Entry!

Invalid BibTex Entry!

Edit - History - Print - Recent Changes - Search
Page last modified on November 28, 2014, at 10:37 AM