
BN /
MixnumMixed number subtractionThis Method B from Kumi Tatsuoka's classic mixednumber subtraction tests, one of the most wellknown cognitively diagnostic assessments. Invalid BibTex Entry!. Bob Mislevy translated Tatsuoka's rule space model into a Bayesian network Invalid BibTex Entry!. PurposeKlein et al. Invalid BibTex Entry! performed a cognitive task analysis of how elementary students approached mixednumber subtraction. The identified two methods:
Tatsuoka Invalid BibTex Entry! applied the Rule Space? method to a sample of students taking a test designed after the cognitive model of Klein et al. She was able to separate out the students using Method A from those using Method B, and identify which of the component skills of each of those methods the students had mastered. Since then, the data has be reanalyzed with a number of other Cognitively Diagnostic Models?. Mislevy Invalid BibTex Entry! translated the model for the Method B assessment to a Bayesian network, in part as an illustration of Bayesian network. At the same time he dropped one of the skills (finding a common denominator). The test analysed here uses only 15 of the original 40 items. Proficiency ModelsBased on the Klein et al. analysis, Method B requires the following skills:
Note that Skill 3 is a prerequisite of Skill 4. Consequently, Mislevy introduced a node Skill WN to model the relationship. This version also adds three other intermediate nodes with logical distributions, primarily to make constructing the CPTs for various evidence models easier. The nodes are s12 and s125. A filled in version of the proficiency model was created by adding dummy nodes for the combinations of skills used in various evidence models. This version can be found in: Task ModelsThe tasks are all mixed number subtraction tasks with two parts, the minuend and the subtrahend. Both the minuend and subtrahend may be mixed numbers, with both a whole and fractional part. The whole number and numerator and denominator of the fractional part are positive integers: algebraic expressions are not allowed. At least one of the minuend or the subtrahend must have a fractional part. The test was administered with pencil and paper and the work product was also a mixed number. As the tasks were designed they were coded with what operations (i.e., skills described above) were required to solve them using both Method A and Method B. This coding was recorded in a Qmatrix?. (Note that the original version of this assessment had a Qmatrix? for both Method A and Method B.) The tasks and Qmatrix for the 15 item subtest used in this example are given below. Evidence ModelsThe term Evidence Model? was not either in the original construction of the test (Tatsuoka, 1984) or when it was converted to a Bayesian network (Mislevy, 1994). However, inspecting the Qmatrix below reveals six unique patterns, which are the evidence models. The column EM in that table indicates which evidence model the task is from. The rules of evidence? for these tasks all were to compare the given solution with the key. The observable? outcome variable was a dichotomous indicator whether the outcome was correct or incorrect. (Note that the tasks which required reducing the result to a simpler form were eliminated, so in this version no rule is needed about equivalent representations.) The statistical form of the evidence model, as described in Mislevy (1994), were deterministicinput noisyand (DINA?) models. Each evidence model has a set of skills required for solving tasks of the appropriate type (given in the Qmatrix?). Examinees who possess all of these skills should be able to solve the task with probability 1s_{j}, where s_{j} is the slipping? probability for Task j. Examinees who lack one or more of the required skills will get the item writing the probability g_{j}, where g_{j} is the guessing? probability for Task j. Note that the xs in the Qmatrix below indicate which proficiency variables are parents of the observable outcome variable. Each row of the conditional probability table? corresponds to one configuration of skills. The values in that row would be either 1g_{j} and g_{j} or s_{j} and 1s_{j} depending on whether that skill pattern had all of the required skills. For scoring purposes, if the guessing and slip parameters are known, then it is simple to construct the conditional probability tables. Many rows will have the same values, but that is not a serious problem. When learning? the parameters from data, however, the values need to be constrained so that the rows which should have the same probabilities will have the same probabilities. The simplest way to do this is to introduce a deterministic node into the evidence model which combines all of the skills into a single skill pattern. This is then the input into the probabilistic node which represents the observable. Sinarhay and Almond Invalid BibTex Entry! explored some alternative evidence models where the probability of success was proportional to the number of the required skills the examinee had mastered. (The filledin proficiency model above is used for this model.) In particular, they found that the guess parameter was smaller for students who had mastered no skills than for students who had mastered at least one of the required skills. Assembly ModelThe original test was represented with a QMatrix?. The original test ([bninea.bib,Tatsuoka1984]) had 40 items; the second 20 were exact isomorophs? of the first 20. Following Mislevy (1994), we used only the first 20 tasks, and dropped five that required finding a common denominator. The Qmatrix for the reduced test is given below: Data SetsThree versions of this data are available:
a version using fixed tables, suitable for scoring, but not calibration. a version usable for calibration. Tarballs containing all of the files are available from http://pluto.coe.fsu.edu/BNinEA/MixedNumberSubtraction. We do not have permission to redistribute the data from the original Tatsuoka (1984) analysis, but the mixnum contains a random data set. ReferencesInvalid BibTex Entry! Invalid BibTex Entry! Invalid BibTex Entry! Invalid BibTex Entry! Invalid BibTex Entry! Invalid BibTex Entry! 