• View
• Edit
• History
• Print

QMatrix

ACED Evidence Models, Assembly Model and Q-Matrix

Evidence Models

All of the ACED tasks have a single observable outcome variable that is scored dichotomously (correct or incorrect). For this reason each Evidence Model? corresponds to a unique row pattern in the Q-Matrix?. These are replicated according to the number of tasks that use that evidence model for scoring (i.e., the number of tasks built from the Task Model? corresponding to the Evidence Model).

The + indicates that the corresponding proficiency variable (names are abbreviated) is relevant, the ++ indicates that the variable is the primary one for this task type.

The names of the task and evidence models are created by removing the spaces from the names, making the first letter lower case, and appending the work 'Task' (for task models) or 'Evidence' (for evidence models).

The design pattern of 'Compensatory' means that the experts thought the skills entered in a compensatory manner.

The anchor column indicates which Scale Anchor? set this evidence model type belongs to.

Difficulty Targets and Link Model Priors

The ACED team decided to build three different variants of of each type of task: an easy, a medium difficulty and a hard task. To model this, we used a variant of the evidence model called the Link Model Prior?, a sub-model that differs from its evidence model parent by having a different prior distribution for the parameters. In this case "easy" tasks were thought to have a difficulty of around -1 (on an IRT scale); "hard" tasks were thought to have a difficulty of around 1; and "medium" tasks were thought to have a difficulty of around 0.

Notationally, the Link Model Prior names were formed from the Evidence Model names by adding 'Hard', 'Medium', or 'Easy', and 'LMP'. The Task Specifications? (a sub-model of the task model) names were formed by adding just the name. The task names ended in a number indicating the difficulty target (1=easy, 2=medium, 3=hard).

The "Solve Geometric Problems" tasks

In the initial design of ACED, the the team identified one class of problems as "Solve Geometric Problems" tasks and their Q-Matrix entry indicated that they tapped the "Solve Geometric Problems" proficiency model (the highest level one). In later analysis, these tasks proved to be highly influential. On closer scrutiny, it turns out that the tasks involved recognizing whether or not a sequence was geometric. Thus, maybe the proficiency model was really missing a "Recognize Sequence Type" node, which is the one this should be attached to. The original model was the one used for task selection and analysis in all of the papers.

(Note that the CBAL Mathematics Team at ETS, noted the prevalence of a pattern in which students would try to solve proportional reasoning problems using addition instead of multiplication. It may be the sequence which typically forced the recognize sequence problems first, helped students later. Thoughts?)

The complete task list

ACED used an extended Task ID system (designed for StatShop) that used a URL-like syntax to indicate the ECD models associated with a task. The extended IDs look like:

ecd://"Project"/"Assessment"/"Model"/"Submodel"/"ID"."Version"

In this case "Project" is always K12Math, "Assessment" is always Formative and "Version" is always 1. The "ID" part is unique, always starts with a 't' and ends with a number indicating difficulty. This shorter ID is used in a number of places.

The complete list of tasks is as follows:

ecd://K12Math/Formative/commonRatioTask/commonRatioMedium/tCommonRatio2b.1
ecd://K12Math/Formative/solveGeometricProblemsTask/solveGeometricProblemsMedium/tSolveGeometricProblems2a.1
ecd://K12Math/Formative/solveGeometricProblemsTask/solveGeometricProblemsMedium/tSolveGeometricProblems2b.1
ecd://K12Math/Formative/solveGeometricProblemsTask/solveGeometricProblemsEasy/tSolveGeometricProblems1a.1
ecd://K12Math/Formative/solveGeometricProblemsTask/solveGeometricProblemsEasy/tSolveGeometricProblems1b.1
ecd://K12Math/Formative/visualExtendTableModelVerbalRuleGeometricTask/visualExtendTableModelVerbalRuleGeometricMedium/tVisualExtendTableModelVerbalRuleGeometric2a.1
ecd://K12Math/Formative/solveGeometricProblemsTask/solveGeometricProblemsHard/tSolveGeometricProblems3a.1
ecd://K12Math/Formative/solveGeometricProblemsTask/solveGeometricProblemsHard/tSolveGeometricProblems3b.1
ecd://K12Math/Formative/visualExtendVerbalRuleModelGeometricTask/visualExtendVerbalRuleModelGeometricMedium/tVisualExtendVerbalRuleModelGeometric2a.1
ecd://K12Math/Formative/tableExtendGeometricTask/tableExtendGeometricMedium/tTableExtendGeometric2a.1
ecd://K12Math/Formative/visualExtendGeometricTask/visualExtendGeometricMedium/tVisualExtendGeometric2a.1
ecd://K12Math/Formative/verbalRuleExtendModelGeometricTask/verbalRuleExtendModelGeometricEasy/tVerbalRuleExtendModelGeometric1a.1
ecd://K12Math/Formative/verbalRuleExtendModelGeometricTask/verbalRuleExtendModelGeometricMedium/tVerbalRuleExtendModelGeometric2a.1
ecd://K12Math/Formative/verbalRuleExtendModelGeometricTask/verbalRuleExtendModelGeometricEasy/tVerbalRuleExtendModelGeometric1b.1
ecd://K12Math/Formative/tableExtendGeometricTask/tableExtendGeometricMedium/tTableExtendGeometric2b.1
ecd://K12Math/Formative/tableExtendGeometricTask/tableExtendGeometricHard/tTableExtendGeometric3a.1
ecd://K12Math/Formative/commonRatioTask/commonRatioMedium/tCommonRatio2a.1
ecd://K12Math/Formative/commonRatioTask/commonRatioHard/tCommonRatio3a.1
ecd://K12Math/Formative/commonRatioTask/commonRatioHard/tCommonRatio3b.1
ecd://K12Math/Formative/commonRatioTask/commonRatioEasy/tCommonRatio1a.1
ecd://K12Math/Formative/commonRatioTask/commonRatioEasy/tCommonRatio1b.1
ecd://K12Math/Formative/tableExtendGeometricTask/tableExtendGeometricHard/tTableExtendGeometric3b.1
ecd://K12Math/Formative/visualExtendGeometricTask/visualExtendGeometricHard/tVisualExtendGeometric3a.1
ecd://K12Math/Formative/modelExtendTableGeometricTask/modelExtendTableGeometricMedium/tModelExtendTableGeometric2a.1
ecd://K12Math/Formative/visualExtendGeometricTask/visualExtendGeometricEasy/tVisualExtendGeometric1a.1
ecd://K12Math/Formative/extendGeometricTask/extendGeometricMedium/tExtendGeometric2c.1
ecd://K12Math/Formative/extendGeometricTask/extendGeometricMedium/tExtendGeometric2a.1
ecd://K12Math/Formative/extendGeometricTask/extendGeometricHard/tExtendGeometric3a.1
ecd://K12Math/Formative/tableExtendGeometricTask/tableExtendGeometricEasy/tTableExtendGeometric1a.1
ecd://K12Math/Formative/extendGeometricTask/extendGeometricHard/tExtendGeometric3b.1
ecd://K12Math/Formative/modelExtendTableGeometricTask/modelExtendTableGeometricEasy/tModelExtendTableGeometric1a.1
ecd://K12Math/Formative/visualExtendGeometricTask/visualExtendGeometricEasy/tVisualExtendGeometric1b.1
ecd://K12Math/Formative/examplesGeometricTask/examplesGeometricMedium/tExamplesGeometric2a.1
ecd://K12Math/Formative/tableExtendGeometricTask/tableExtendGeometricEasy/tTableExtendGeometric1b.1
ecd://K12Math/Formative/visualExtendGeometricTask/visualExtendGeometricEasy/tVisualExtendGeometric1c.1
ecd://K12Math/Formative/modelExtendTableGeometricTask/modelExtendTableGeometricEasy/tModelExtendTableGeometric1b.1
ecd://K12Math/Formative/extendGeometricTask/extendGeometricEasy/tExtendGeometric1a.1
ecd://K12Math/Formative/verbalRuleModelGeometricTask/verbalRuleModelGeometricHard/tVerbalRuleModelGeometric3a.1
ecd://K12Math/Formative/examplesGeometricTask/examplesGeometricMedium/tExamplesGeometric2b.1
ecd://K12Math/Formative/extendGeometricTask/extendGeometricEasy/tExtendGeometric1c.1
ecd://K12Math/Formative/verbalRuleExplicitModelGeometricTask/verbalRuleExplicitModelGeometricMedium/tVerbalRuleExplicitModelGeometric2a.1
ecd://K12Math/Formative/verbalRuleExplicitModelGeometricTask/verbalRuleExplicitModelGeometricHard/tVerbalRuleExplicitModelGeometric3a.1
ecd://K12Math/Formative/modelExtendTableGeometricTask/modelExtendTableGeometricMedium/tModelExtendTableGeometric2b.1
ecd://K12Math/Formative/examplesGeometricTask/examplesGeometricHard/tExamplesGeometric3a.1
ecd://K12Math/Formative/visualExplicitVerbalRuleModelGeometricTask/visualExplicitVerbalRuleModelGeometricHard/tVisualExplicitVerbalRuleModelGeometric3a.1
ecd://K12Math/Formative/modelExtendTableGeometricTask/modelExtendTableGeometricHard/tModelExtendTableGeometric3a.1
ecd://K12Math/Formative/visualExplicitVerbalRuleModelGeometricTask/visualExplicitVerbalRuleModelGeometricHard/tVisualExplicitVerbalRuleModelGeometric3b.1
ecd://K12Math/Formative/modelExtendTableGeometricTask/modelExtendTableGeometricHard/tModelExtendTableGeometric3b.1
ecd://K12Math/Formative/examplesGeometricTask/examplesGeometricHard/tExamplesGeometric3b.1
ecd://K12Math/Formative/examplesGeometricTask/examplesGeometricEasy/tExamplesGeometric1a.1
ecd://K12Math/Formative/recursiveRuleGeometricTask/recursiveRuleGeometricMedium/tRecursiveRuleGeometric2a.1
ecd://K12Math/Formative/examplesGeometricTask/examplesGeometricEasy/tExamplesGeometric1b.1
ecd://K12Math/Formative/recursiveRuleGeometricTask/recursiveRuleGeometricMedium/tRecursiveRuleGeometric2b.1
ecd://K12Math/Formative/recursiveRuleGeometricTask/recursiveRuleGeometricHard/tRecursiveRuleGeometric3a.1
ecd://K12Math/Formative/recursiveRuleGeometricTask/recursiveRuleGeometricHard/tRecursiveRuleGeometric3b.1
ecd://K12Math/Formative/recursiveRuleGeometricTask/recursiveRuleGeometricEasy/tRecursiveRuleGeometric1a.1
ecd://K12Math/Formative/recursiveRuleGeometricTask/recursiveRuleGeometricEasy/tRecursiveRuleGeometric1b.1
ecd://K12Math/Formative/explicitGeometricTask/explicitGeometricMedium/tExplicitGeometric2a.1
ecd://K12Math/Formative/explicitGeometricTask/explicitGeometricMedium/tExplicitGeometric2b.1
ecd://K12Math/Formative/explicitGeometricTask/explicitGeometricHard/tExplicitGeometric3a.1
ecd://K12Math/Formative/explicitGeometricTask/explicitGeometricHard/tExplicitGeometric3b.1
ecd://K12Math/Formative/explicitGeometricTask/explicitGeometricEasy/tExplicitGeometric1a.1
ecd://K12Math/Formative/explicitGeometricTask/explicitGeometricEasy/tExplicitGeometric1b.1

HomePage ACED Data Model People Publications

ACED development and data collection was sponsored by National Science Foundation Grant No. 0313202.