EDUC 603

EDUC 603: Assessment of Student Learning

Credit Hours: (3)

Students improve their educational practice through learning and applying effective assessment and evaluation principles and procedures. Case studies, projects and readings address topics such as designing valid and reliable assessment items, interpreting assessment data, and deciphering standardized test scores. Additionally, the real-world cases and projects provide meaning and context to the course's instructional goals and objectives by providing opportunities for students to evaluate assessments created with artificial intelligence (AI) tools, design assessment instruments for instructional programs and interpret data collected within educational research and evaluation studies.

Detailed Description and Course Syllabus

This lecture and project-based course is designed to address important topics associated with designing and implementing a variety of assessment types to meet specific instructional needs. Topics addressed in the course include broad, theoretical concepts such as evidence of validity, estimates of reliability, and fairness. Design issues associated with strategies for developing effective selected-response as well as constructed-response assessment items constitute a major focus in the course, and performance assessment design such as rubrics is included as well. Course content and case-based project experiences also provide important background knowledge regarding state assessment programs, legal and ethical considerations in assessment, and strategies for adapting various assessments to the needs of all learners. Finally, specific procedures for improving teacher-developed assessments are facilitated.

Detailed Description of Conduct of Course

The course is divided into four distinct parts, each centered on a detailed real-world case requiring students to review documents and other materials to make sense of assessment-related problems and propose possible solutions. In the process, information and examples of the assessment-related concepts explored are presented, along with practice aligning with specific learning objectives. Assignments associated with each case will involve completing a number of small projects.

Goals and Objectives of the Course

The following specific instructional objectives will be facilitated and measured within the course.

Why Do Teachers Need to Know about Evaluation and Assessment
1. Identify/describe common reasons why teachers test.
2. Define assessment and explain how assessment can be different from testing.
3. Identify and describe the following “variables of interest” that are important to educators (not just achievement variables), including:
  1. Expected Terminal Outcome Performance
  2. Unexpected Terminal Outcome Performance
  3. Embedded Practice Performance
  4. Expected Attitudes
  5. Unexpected Attitudes
  6. Instructional Component Perception
  7. Social Interaction Concerns
  8. Instructor Concerns
  9. Learner Concerns
4. Distinguish between definitions and examples of assessment versus evaluation.
5. Define “curricular aim” and describe common analogous terms.
6. Describe some of the important reasons why educators should apply effective assessment and evaluation principles and procedures within their professional practice.
7. Distinguish between definitions and examples of formative as well as summative assessment.
8. Identify and describe the role both formative and summative assessments play in the practice of educators.
9. Describe the characteristics of well-written instructional objectives.
10. Define “operationalize” as it relates to instructional objectives.
11. Revise poorly-written objectives, including operationalizing intended learning outcomes.
12. Given an objective, categorize the type of learning outcome as verbal information, intellectual skill, motor skill, or attitude.
13. Explain the role well-written objectives play in the assessment design process.
Reliability of Assessment
1. Given a description of reliability measures applied to a specific set of data, determine whether stability (t-rt), alternate-form, or internal consistency reliable has been estimated.
2. Distinguish between internal and external reliability.
3. Describe the purpose of the standard error of measurement.
4. Use the standard error of measurement to help interpret the meaning of individual test scores.
Validity
1. Distinguish between definitions and examples of reliability estimates and evidence of validity.
2. Distinguish between internal and external validity.
3. Given descriptions of tests conducted to determine evidence of validity for specific assessment data, determine whether content, internal structure (construct), or relations to other variables (criterion) is the source of validity evidence.
4. Describe the relationship between validity and reliability.
5. Define “Type I” and “Type II” errors, and explain the relationship between validity and each type of error.
Fairness
1. Given an assessment item, identify where it might be biased and suggest strategies for improving the item.
2. Describe major difficulties facing educators who wish to use empirical methods to identify and correct item bias within the tests they create themselves.
Strategies for Effective Selected-Response Tests
1. Edit poorly-written constructed-response assessment items to make them more well-written.
2. Given an instructional objective, determine the best possible selected-response assessment items that effectively measure the skill indicated within the objectives under the conditions stated (if applicable).
Strategies for Effective Constructed-Response Tests
1. Edit poorly-written constructed response assessment items to make them more well-written.
2. Given an instructional objective, determine the best possible constructed response assessment items that effectively measure the SKA indicated within the objectives under the conditions stated (if applicable).
Assessing Affective Variables of Interest
1. Identify and describe common affect-related attitudinal variables of interest for teachers.
2. Critique the design of a multifocus affective inventory.
Performance Assessments and Effective Rubrics
1. Critique rubric scoring guides for given instructional goals or tasks.
Portfolio Assessment
1. Describe the different roles portfolios can play within an instructional environment.
2. Given a specific portfolio assessment scenario, critique how well the portfolio adheres to the seven key ingredients of portfolio assessment.
3. Describe how a variety of computer-based technologies can be used to design and implement an electronic portfolio system.
Adapting Assessments
1. Determine appropriate adaptations for assessments to meet the needs of diverse learners.
Pre-Post Comparisons
1. Calculate differences between pretest and posttest assessment results, and ascertain their significance.
Formative Assessment Practices
1. Apply analytical skills to assessment results in order to inform ongoing planning and instruction.
2. Design strategies to help students understand their own progress and growth.
State Assessment Programs
1. Identify and describe state assessment programs and accountability systems, including assessments used for student achievement goal setting as related to teacher evaluation and determining student academic progress.
Legal & Ethical Considerations
1. Explain relevant legal and ethical aspects of assessments (including diagnostic tests, college admission exams, industry certification tests, and placement assessments).
Improving Teacher-Developed Assessments
1. Differentiate between examples of judgmental and empirical assessment item improvement strategies.
2. Given a data set from a selected response item assessment, conduct an item analysis and calculate difficulty as well as item-discrimination indices.
3. Given the results of an item (distracter) analysis for a multiple-choice assessment along with a general description of the purpose of the test, determine whether or not specific test items should be included in subsequent versions of the test.
Making Sense Out of Standardized Test Scores
1. Distinguish between examples of norm-referenced and criterion-referenced assessments.
2. Identify the primary characteristics and limitations of standardized test scores.
3. Compare and contrast definitions, descriptions and examples of percentile, grade-equivalent, and scale test scores.
4. Describe requisite Virginia assessment programs and accountability systems as well as additional common assessments used in preK-12 education (e.g. diagnostic tests, college admission exams, industry certification tests, and placement assessments).

*Professional Organization*	*Standards*
National Association for Gifted Children (NAGC)-Council for Exceptional Children (CEC)	Standard 4: S4: Pre-assess the learning needs of individuals with gifts and talents in various domains and adjust instruction based on continual assessment. Standard 8: K2: Uses, limitations, and interpretation of multiple assessments in different domains for identifying individuals with exceptional learning needs, including those from diverse backgrounds. K3: Uses and limitations of assessments documenting academic growth of individuals with gifts and talents. S1: Use non-biased and equitable approaches for identifying individuals with gifts and talents, including those from diverse backgrounds. S2: Use technically adequate qualitative and quantitative assessments for identifying and placing individuals with gifts and talents. S3: Develop differentiated curriculum-based assessments for use in instructional planning and delivery for individuals with gifts and talents. S4: Use alternative assessments and technologies to evaluate learning of individuals with gifts and talents.
National Association for the Education of Young Children-Advanced (NAEYC)	Standard 3: 3a: Understanding the goals, benefits, and uses of assessment 3b: Knowing about and using observation, documentation, and other appropriate assessment tools and approaches 3c: Understanding and practicing responsible assessment
Interstate School Leaders Licensure Consortium Standards (ISLLC)	Standard 2 E: Develop assessment and accountability systems to monitor student progress. I: Monitor and evaluate the impact of the instructional program.
Association for Childhood Education International (ACEI)	4.0 Assessment for Instruction: Candidates know, understand, and use formal and informal assessment strategies to plan, evaluate and strengthen instruction that will promote continuous intellectual, social, emotional, and physical development of each elementary student.
National Council of Teachers of English (NCTE)	4.10: Integrate assessment consistently into instruction by establishing criteria and developing strategies for assessment that allow all students to understand what they know and can do in light of their instructional experiences; interpreting the individual and group results of any assessments and drawing upon a variety of information in these assessments to inform instruction; assisting all students in becoming monitors of their own work and growth in speaking, listening, writing, reading, enacting, and viewing; explaining to students, parents, and others concerned with education how students are assessed.
National Council of Teachers of Mathematics (NCTM)	8.3 Uses multiple strategies, including listening to and understanding the ways students 8.6 Demonstrates knowledge of research results in the teaching and learning of mathematics.
National Middle Schools Association (NMSA)	Standard 3: 4. Are knowledgeable about local, state, and national middle level curriculum standards and of ways to assess the student knowledge reflected in those standards. 7. Understand multiple assessment strategies that effectively measure student mastery of the curriculum. Standard 5: 2. Know a wide variety of teaching, learning, and assessment strategies, and when to implement them. 3. Know that teaching higher order thinking skills is an integral part of instruction and assessment. 4. Know how to select and develop formal, informal, and performance assessments based on their relative advantages and limitations. 6. Know how to evaluate the effectiveness of teaching strategies.
National Science Teachers Association (NSTA)	Standards 8: a. Use multiple assessment tools and strategies to achieve important goals for instruction that are aligned with methods of instruction and the needs of students. b. Use the results of multiple assessments to guide and modify instruction, the classroom environment, or the assessment process.
National Council for the Social Studies (NCSS)	Pedagogical Standards 7: ASSESSMENT Social studies teachers should possess the knowledge, capabilities, and dispositions to use formal and informal assessment strategies at the appropriate school levels to evaluate and ensure the continuous intellectual, social, and physical development of learners. They should be able to assess student learning using various assessment formats, including performance assessment, fixed response, open-ended questioning, and portfolio strategies.
Virginia Department of Education Standards for the Professional Practice of Teachers (VA DOE)	Standard Three: Planning, Delivery, and Assessment of Instruction Key Element 7: The teacher communicates specific performance expectations and uses a variety of assessment strategies to plan instruction and to monitor and document student progress.

Assessment Measures

Assessments may include, but are not limited to:

Project-related activities included in the course case studies
Written exams

Other Course Information

Approval and Subsequent Reviews
4/24/09 Course syllabus revised and submitted Greg Sherman

May 10, 2024