ID 553 - Module 4: Objectives and Aligned Assessment

Assessment Alignment and Rubrics

Estimated reading time: 30 minutes.

Video: Assessment Alignment and Rubrics. Lecture: assessment before strategy, criterion-referencing, matching item types to objectives with aligned and misaligned examples, and building analytic rubrics. (Instructor: build with make_videos.py or record and embed your own.)

Assessment comes before strategy

In Dick and Carey, assessment instruments are developed immediately after objectives and before instructional strategies. Backward Design makes the same move: decide what evidence you will accept before you plan activities. The reason is alignment. If the assessment is designed last, it tends to measure what was convenient to teach rather than what the objective required, and learners learn to the test rather than to the performance.

An assessment is aligned when the performance it elicits, the conditions it imposes, and the standard it applies match the objective. Check all three, every time.

Types of assessment by purpose

Type	When	Purpose
Entry-skill test	Before instruction	Confirm learners have the prerequisites; route those who do not
Pretest	Before instruction	Measure what learners already know of the objectives; allow test-out; provide a baseline
Practice tests and embedded checks	During instruction	Give learners feedback and the designer formative data; retrieval practice improves retention
Posttest	After instruction	Measure achievement of the objectives
Performance assessment	After instruction, or on the job	Measure the actual performance under realistic conditions

Not every project needs all five. Every project needs the last two, and most benefit from embedded checks.

Criterion-referenced, not norm-referenced

Instructional design uses criterion-referenced assessment: each learner is measured against the standard in the objective, not against other learners. Grading on a curve makes no sense when the question is whether a nurse can administer medication safely. The criterion in the objective is the passing standard; if learners fail in large numbers, the instruction is the problem, not the learners.

Matching item types to objectives

Objective type	Appropriate assessment	Inappropriate assessment
Verbal information (state, list, describe)	Short answer, matching, multiple choice, completion	Extended performance tasks
Intellectual skill: discrimination and concept classification	Classify new examples and non-examples; multiple choice with well-designed distractors	Recall of the definition alone
Intellectual skill: rule use and problem solving	Solve new problems; scenario-based items; worked cases with a decision	Items that repeat examples used in instruction
Motor skill	Observed performance with a checklist	Written description of the procedure
Attitude	Observed choice in realistic situations; behavioral records over time; self-report as supplement only	A quiz on what the right attitude is
Create-level cognitive objectives	Products judged with a rubric	Multiple choice

The most common alignment error is testing a rule-use objective with a recall item. If the objective says "decide," the assessment must present a decision.

Writing good selected-response items

Stem states a complete problem; the learner should be able to answer before reading the options.
Options are parallel in length and grammar; the correct answer is not the longest.
Distractors are plausible and, ideally, drawn from the common errors identified in the task analysis. A distractor no one chooses is wasted.
Avoid "all of the above," "none of the above," and negatives in the stem.
One item, one objective. Tag each item with the objective number so you can analyze results by objective.

Performance assessment and rubrics

When the objective is a performance or product, the assessment is a task plus a scoring guide.

Checklists list the required steps or features; each is present or absent. Use for procedures with a correct sequence.

Rating scales score each criterion on a scale (for example, 1 to 4) without describing the levels. Fast, but unreliable across raters.

Analytic rubrics describe each criterion at each level of performance. They are the most reliable and give the most useful feedback. Every assignment rubric in this course is analytic. Build one by:

Listing the criteria from the objective and task analysis (three to six is typical).
Describing the exemplary level for each criterion in observable terms.
Describing the beginning level.
Filling in the intermediate levels by naming what is present and what is missing.
Testing the rubric on two or three real or simulated samples with a second rater, and revising where you disagree.

Holistic rubrics give one overall score with a description per level. Faster to apply, less diagnostic. Use for large-scale screening, not for feedback.

Feedback

Assessment during instruction is only formative if the learner gets feedback they can use. Effective feedback is specific to the criterion, tells the learner what to do next, and arrives while the learner can still act on it. Automated feedback in quizzes should explain why the correct answer is correct and why the chosen distractor is wrong; the knowledge checks in this course model that pattern.

Validity, reliability, and fairness

Validity: the assessment measures the objective. Alignment is the main validity check for instructional assessment.
Reliability: the assessment gives consistent results across items, occasions, and raters. Enough items per objective, clear rubrics, and rater training are the levers.
Fairness: the assessment does not disadvantage learners for reasons unrelated to the objective (reading load, cultural references, inaccessible formats). Apply the UDL principle of multiple means of action and expression where the objective allows.

Preparing for the instructional plan

For each objective in your plan (Module 6) you will specify an assessment: the type, the item or task, and the scoring guide. Draft one now for your terminal objective and bring it to the Objective Clinic along with the objectives.

References

Dick, W., Carey, L., & Carey, J. O. (2015). The systematic design of instruction (8th ed.). Pearson. Chapter 7.

Haladyna, T. M., & Rodriguez, M. C. (2013). Developing and validating test items. Routledge.

Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189.

Wiggins, G., & McTighe, J. (2005). Understanding by design (2nd ed.). ASCD.