UDL On Campus: UDL and Assessment

What

What is this resource about? This resource provides an overview of the Types of Assessment, a section on Assessing Variable Learners, examples of Construct Relevance, and a section about the UDL Principles and Assessment.

Why

Why is this important for higher education? From cultural and linguistic proficiences to unbridled enthusiasm for study, to desperate anxiety about the challenges ahead, students vary. Reducing cognitive, linguistic, executive, and affective barriers is of vital importance as students negotiate college and university expectations differently, according to their widely ranging background experiences.

UDL Connection

Provide multiple means of engagement: Stimulate interest, motivation, and persistence in learning. Just as students learn more effectively when they are engaged and motivated, their performance on assessments can be enhanced by increasing engagement.

Provide multiple means of action and expression: Offer different methods for students to demonstrate what they are learning such as through writing, multimedia, or demonstration.

Provide multiple means of representation: Consider the ways in which the items are presented (e.g.,text, graphs, charts, images, videos, demonstrations, objects to manipulate) and if they create barriers for students.

Types of Assessment

The purpose of assessment in postsecondary courses varies as well. Assessments are often designed to gather student data that will yield information about accountability, student progress, and instruction.

Accountability: Assessing student performance with respect to job preparation, prerequisites, and university or college program goals
Student Progress: Assessing changes in student performance over time as a result of instruction (assessment of learning)
Instruction: Probing student responses to instruction in order to optimize the course of learning (assessment for learning)^¹

Assessment is used in courses to determine how well students are meeting goals that have been set (e.g., goals around job performance, goals around changes in knowledge). Measurable outcomes from assessments should be comparable with or benchmarked against set course goals. Assessment outcomes, in turn, should inform further instruction.

Assessing Variable Learners

Provision of options within the design of both formative and summative assessment helps to ensure that all learners can act on new information and demonstrate what they know. This requires a distribution of the demands and benefits of any one kind of assessment among all students. For example, students with Attention Deficit Disorder (ADD) may have issues with working memory that would make long multiple choice assessments challenging. If this is the only way students are assessed in a course, students with working memory challenges will have more demands and fewer benefits when it comes to assessment. If writing long responses to text passages is the only option for assessment, students who are not native English speakers will have more demands and fewer benefits, especially if they do not have access to aids such as a glossary or dictionary or if there is time pressure to respond. In sum, greater attention must be paid to issues of learner variability in the design of assessments. Further, consideration needs to be given to embedded design features in digital assessments (such as text-to-speech capability, availability of key word definitions, hints or coaching tips, etc.) so that assessments support students that vary in terms of their strengths, weaknesses, and learning needs.

Transcript

[White text appears on magenta background: "UDL On Campus.” The colors are inverted and a wave of orange slides over the “UDL On Campus” magenta text. Grey text appears below: “Universal Design for Learning in Higher Education.”]

FEMALE VOICE: UDL On Campus: Universal Design for Learning in Higher Education.

[The title appears onscreen.]

FEMALE NARRATOR: An Introduction to UDL and Assessment.

[The screen opens to a montage of scenes on a college campus: a health sciences classroom, an academic support center, a welcome center, a campus directory map, and students working on computers in computer labs and libraries. The last two scenes show a female student on her laptop and a male student and his tutor working together on a paper-based assignment.]

FEMALE NARRATOR: Reducing learning barriers is of vital importance in postsecondary settings as learners arrive in colleges and universities with a diverse set of experiences; what may be a preferred learning mode for one individual may be an obstacle for another.

[A montage of more college and university life: a lounge area, a student working with a Bunsen burner in a lab, and an auditorium of students. The same female student is spotlighted: we see her working on a mass and wind project on her laptop. She looks at different visual data and engages with a colorful diagram.]

FEMALE NARRATOR: In learning environments, individual variability is the norm, not the exception. Instructors of postsecondary courses set goals for learners. Assessment is used in courses to determine how well learners are meeting instructional goals. Goals need to be clear, and separate from the means of achieving them. Assessments should offer more than one route for demonstrating mastery.

[The scene changes to a split screen of close-ups of two students: the female student working on her laptop and the male student writing on paper. The images split open like sliding doors to reveal text underneath: “Construct.” The definition fades in below: “the term used to define the knowledge, skills, and abilities to be assessed.”]

FEMALE NARRATOR: A critical part of assessment is the construct. “Construct” is the term used to define the knowledge, skills, and abilities to be assessed.

[Different text appears onscreen: “Construct Relevant Factors.” The definition fades in below: “type of presentation or response that are related to the construct that the test is intended to measure.”]

FEMALE NARRATOR: Construct relevant factors refer to factors such as type of presentation or response that are related to the construct that the test is intended to measure.

[Another montage of scenes appears: a hand pressing a button on a graphing calculator, a close-up of the graph it formed, and a handwritten chart of math symbols.]

FEMALE NARRATOR: Many assessments include features that are not relevant to the construct being assessed. These are considered construct irrelevant items. These features may pose barriers for some learners and impact their performance on the assessment.

[Text appears onscreen: “Universal Design for Learning.” Three sections fade in below the heading: “Affective Networks,” with a brain highlighting these portions in green; “Recognition Networks,” with a brain highlighting these portions in purple; and “Strategic Networks,” with a brain highlighting these portions in blue. Text fades in below: “the WHY of learning,” “the WHAT of learning,” and “the HOW of learning.” Each network then appears individually with their corresponding brain: Recognition Networks and “Multiple Means of Representation,” Strategic Networks and “Multiple Means of Action and Expression” and Affective Networks and “Multiple Means of Engagement.”]

FEMALE NARRATOR: That’s where Universal Design for Learning, or UDL, comes into play. As a set of three principles for curriculum development, UDL provides a blueprint for creating instructional goals, assessments, methods, and materials that work for the widest possible range of learners. This flexible approach to curriculum development can positively impact how learners perform on assessments and result in a more authentic appraisal of learner achievement.

[Lines move in against a white background. A purple box and the text “Multiple Means of Representation” push in. Video footage is shown of the female student working on different diagrams for her mass and wind project. Text appears below: “presentation of information.”]

FEMALE NARRATOR: The first principle, representation, focuses on the ways in which information, including assessment items, is presented. These can be charts, graphs, images, videos, demonstrations, or objects to manipulate.

[Lines move in against a white background. A blue box and the text “Multiple Means of Action and Expression” push in. Video footage is shown of a different female student working on a whiteboard, followed by a whiteboard animation. Text appears below: “learners demonstrate what they have learned.”]

FEMALE NARRATOR: The second principle, action and expression, focuses on how learners demonstrate what they have learned. Do they have the option to write or draw? Do they have the option to present their capabilities through multimedia or a recording?

[Lines move in against a white background. A green box and the text “Multiple Means of Engagement” push in. Video footage is shown of the male student working with his tutor on a paper-based assignment. Text appears below: “enhancing learner motivation and persistence.”]

FEMALE NARRATOR: The third principle, engagement, focuses on enhancing motivation and persistence. When their achievement is assessed, are they sufficiently but not overly, challenged? Do they have options and variety in how they are assessed to help reduce anxiety?

[The screen changes to show a teacher in a mathematics classroom addressing students. A montage of students follows: a student using a Braille writer, the female student working on her mass and wind project, the male student and his tutor working on his paper-based assignment, and a student working on a physics assignment with an enlarged screen display.]

FEMALE NARRATOR: It is important for instructors to be aware of the inherent variability of learners—the existence of barriers that may impede both the acquisition of information and expressing their competencies. Designing assessments with these challenges in mind will allow for greater flexibility to ensure learner success.

[A white background appears with the following credits:

A Production by CAST
Script Rhianon Gutierrez, Skip Stahl
Narration Mary O’Malley
Editing Ge Vue, Rhianon Gutierrez

Developed as part of the Open Professionals Education Network (Logos for OPEN, CAST, and Creative Commons)

References
Rose, D. H., Hall, T. E., & Murray, E. (2008, Fall). Accurate for all: Universal design for learning and the assessment of students with learning disabilities. Perspectives on Language and Literacy, 23-28.

This video features the following media:
“Welcome to Bristol Community College,” by Bristol Community College,
Source: http://www.youtube.com/ Creative Commons Attribution license (reuse allowed)
“Juna Gjata Video Series,” & “Alessia’s Story,” by NCAIM at CAST, Inc.
Source: http://aem.cast.org/
Used with permission from NCAIM at CAST, Inc.

(Logos for Creative Commons CC, BY, and SA)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0
International (CC BY-SA 4.0) license: http://creativecommons.org/licenses/by-sa/4.0

UDL On Campus: Universal Design for Learning in Higher Education]

Construct Relevance

Assessments are designed to measure knowledge, skills, and abilities. Constructs are the knowledge, skills or abilities being measured by an assessment. By their nature, however, most assessments include features that are not relevant to the construct being assessed. Often the methods and materials used in assessments require additional skills and understanding. These are considered to be construct irrelevant. Construct-irrelevant features of assessments may pose barriers for some students, preventing an accurate measurement of the construct.

Assessment Example

Math assessment that includes word problems to assess students’ understanding of math concepts.

Essay exam in a Biology class that is both timed and closed book.

Construct-Irrelevant Factors

The ability to read fluently is construct irrelevant. Even though it is an important skill, it is not part of the construct being measured. Learners who have difficulty with reading may miss certain items even though they may have a good grasp of the underlying math concepts.

Construct-irrelevant factors include motor coordination (handwriting or typing skills), short-term and working memory, organization and time management, attention, and the ability to work under pressure. The additional measurement of these many factors can prevent gaining an accurate picture of a student’s Biology content knowledge.

Minimizing construct-irrelevant factors does not lessen the rigor of an assessment but instead gives a more accurate picture of what learners are actually learning in terms of the knowledge, skills, and abilities identified in the course goals. Having an accurate picture of student learning is particularly important for formative assessments, where results can be used to revise instruction as a course progresses. Minimizing construct-irrelevant factors helps to focus in on where students are actually struggling with content, skills, or abilities that the assessment is meant to measure.

It is important to consider construct relevance when creating assessments. For example:

Supports can be provided to reduce the measurement of construct-irrelevant factors, such as providing text-to-speech so that reading ability is not being measured in an assessment of content knowledge in mathematics.
Students can be provided with options for how they demonstrate what they know, such as presenting a project either through an open-book essay or an oral presentation.
When construct irrelevance cannot be avoided, such as when administering a standardized test that has not been designed with attention to construct relevance, it is important that the instructor is aware of learners for whom there may be barriers and how it would impact their performance. The demands and benefits of any one form of assessment will differ depending upon who is taking it.

By providing supports and options around how to demonstrate knowledge, and by developing awareness of what barriers different forms of assessment create for different students, faculty can better distribute the demands and benefits of any one form of assessment among all learners.

UDL Principles and Assessment

In learning environments, individual variability is the norm, not the exception. Learners differ from one another, and an individual learner differs over time (students may be tired or hungry on a given day), depending on the task in front of them (students may be bored or feel over challenged), and the context for the assignment (students may consider the material to be controversial or feel that the material is not relevant to them).

Three primary principles of UDL, which are based on research in the learning sciences, guide UDL. These principles can be helpful in thinking about the design of assessments.

Provide Multiple Means of Engagement

Providing multiple means of engagement (the why of learning) means supporting interest, motivation, and persistence. Just as students learn more effectively when they are engaged and motivated, their performance on assessments can be enhanced by increasing engagement.

Do students think that they can be successful? Emphasizing the importance of effort and motivation and expressing confidence that students can meet high expectations can improve their performance.
Do assessments provide different levels of challenge? One way to do this is to provide options on essay exams so that students can choose a question they feel they can answer well. Another way is to allow students to answer essay questions in different formats. Perhaps students could write a classic essay, create a short play, or create a video response. Once an instructor has addressed the question, "What do I really want the learner to learn?" (i.e., construct relevance) then the individual motivations and desires of learners and the time constraints of their instructors may be the only limits to the possibilities.
Are different formats used for assessments over the course of a semester? As mentioned earlier, the demands and benefits of any one form of assessment will differ for each student. Therefore, the options and supports provided for the first two UDL principles (representation and action and expression) can enhance engagement in the assessment process.

Provide Multiple Means of Action and Expression

Providing multiple means of action and expression (the how of learning) means providing different ways for students to work with information and content and to demonstrate what they are learning.

In assessment, consider the ways in which students will demonstrate what they have learned.

Will they need to write or draw?
Will they demonstrate an action?
Do they need to organize information mentally, or can something be provided to help them organize the information (e.g., concept mapping software)?

Again, consider which actions are actually relevant to the construct being measured and which ones can be supported or varied in order to gain an accurate picture of what each student has learned.

Provide Multiple Means of Representation

Providing multiple means of representation (the what of learning) means presenting information and content in different ways and making connections between them. When planning assessments, consider the ways in which the items are presented—text, graphs, charts, images, videos, demonstrations, objects to manipulate.

Do the ways in which items are presented create barriers for any students?
Are there alternatives that could be used that would still allow accurate assessment of what students should know or be able to do?
Are the representations used construct relevant?For example, if students need to be able to interpret information in graph format, then the graph itself is relevant. If they only need to be able to use the information in the graph, consider providing different formats for displaying the information. Tables, charts, infographics, or even plain text may be a more accessible format for some students. Remember, the goal is to find out what students are actually learning. Keep in mind basic accessibility requirements for images (including images of tables, charts, and graphs). Any text in an image needs to be readable by a computer or other assistive device that a student might need for access. All images should have alt text. Additionally, a long description is needed when an image is complex and cannot be described briefly.

Applying the UDL Principles

When applying UDL principles to assessment, it can be helpful to consider first where there might be barriers that would impact the performance of some learners. Below are examples of barriers and some options for minimizing them. Keep in mind that constructs can be assessed in many ways, and the goal is to minimize factors that are construct irrelevant.

Engagement

Possible Barriers

Assessments that create anxiety in some students

Assessments that do not engage some students

Assessments that require persistence

Assessment Options

Remember that every assessment instrument is automatically measuring each student’s reactions to the motivating conditions present in that assessment. For example, some students are enthusiastic about spontaneity and novelty around assessment conditions, while other students experience great amounts of anxiety under such conditions and seek comfort and support.

Offering choice and variety in assessment conditions can reduce these barriers. For example, developing a web site vs. giving a presentation. Offering options between quizzes, journaling, and specific assignments or projects can provide variety.

Action & Expression

Possible Barriers

Assessments that have a single response mode (asking learners to draw diagrams for every answer)

Using the same format for response for all assessments (using only multiple choice, written response)

Assessment Options

The demands associated with responding to any assessment are not always construct relevant. In other words, if the goal of the assessment is to understand the processes involved in cell division, then asking the learner to write an essay and grading the construction of the essay may not be relevant to the task of understanding that the learner really needed to know. Supporting strategic and organizational abilities and allowing students to express acquired knowledge and skills in diverse ways can be built into a variety of assessments.

Choices, again, can be helpful. For example, an assignment on identifying barriers to economic expansion in post-World War II Europe could include the options of an essay, PowerPoint presentation, video, or infographic to achieve the same goal. Demonstrating understanding of text can be expressed by students choosing keepers (items from the text the learner finds important and relevant) and generating queries (questions created by the learner to be answered later) about the content. These can be expressed in written or verbal form, in a group or alone.

Representation

Possible Barriers

Assessments that provide relevant information in a single format (for example, using tests from a textbook in the same format for every assessment, such as end-of-chapter questions)

Using the same forms of representation for all assessments (for example, requiring students to integrate new information from their instructor with a PowerPoint presentation only)

Assessment Options

Side-stepping a one size fits all assessment allows for students to access key ingredients. For example, highlighting critical features in a text or graphic, providing definitions for vocabulary, and scaffolding reading strategies to support comprehension can open up a variety of options for students to engage in the goals of the assessment. It is, of course, important that these supports do not impact construct relevance. In other words, if it is important for the student to be able to glean relevant information from a graphic representation (such as they might encounter in the workplace), then the assessment must include relevant information in a graphic representation.

In order for students to show what they know, providing choices in how to access information is key. For example, start with content students have had success with in the past; then work from there, checking to see what other options for representation could be available. Ask, Are there other media available; are there more choices learners can have to show mastery?

Examples in a Mathematics Course

There are many ways to apply the guidelines of UDL within assessment in a specific course. Below are just a few specific examples that could help to address barriers for some students.

Provide options for physical actions
- Provide open-book quizzes
- During quizzes and tests completed individually, provide scheduled breaks so students can communicate with each other and their instructor in order to become unstuck while working on complex equations and problem solving. For example, during tests, one mathematics instructor writes some of the complex equations on whiteboards around the room. During a ten-minute break, students have the option to approach a problem and talk through their thinking in order to progress to the next step in the activity if they are stuck.
Provide options for expression and communication
- Allow for small group work
- Use a computer program, such as software for creating drawings, to demonstrate lesson goals and objectives
Provide options for executive functions
- Allow students to take a video of themselves solving a problem and talking through their thought processes
- Provide a choice of mathematical problems for students to complete to demonstrate mastery of a learning objective

Review

Ongoing assessment is an essential part of any course. Learners vary in the ways in which they learn and can demonstrate what they are learning. Because of this variability, construct-irrelevant features in assessments can create barriers that prevent some learners from accurately demonstrating what they have learned. The principles of universal design for learning (UDL) can be applied to help identify and address construct-irrelevant barriers in assessments.

Notes

¹Rose, D. H., Hall, T. E., & Murray, E. (2008, Fall). Accurate for all: Universal design for learning and the assessment of students with learning disabilities. Perspectives on Language and Literacy, 23-28.