CTB/McGraw-Hill eGuide to Interim and Formative Assessment

[Reprinted below is the eGuide to Interim and Formative Assessment, for reference only, in conjunction with the immediately preceding post. – JP]

Choosing the Right Assessment, Interpreting Student Progress, and Improving Achievement

Introduction and How to Use this Guide

Assessment is most effective when it is integrated into your overall plan to improve student achievement. There are several top-level considerations educators should evaluate when selecting assessment solutions.

CTB/McGraw-Hill is pleased to provide this eGuide to help you make informed and appropriate choices regarding interim and formative assessments. It is meant to help you navigate the kinds of assessments available to you and your colleagues, and understand the important elements of high quality assessment solutions.

While each of the items detailed below can and should be weighed and evaluated, the overall goal must always be improving instruction, and thereby, student achievement. Each element of your assessment system should demonstrably support teaching and student learning.

Use this guide to help you select a new assessment solution or supplement materials in your existing assessment program, such as user guides, online help solutions, professional development, and technical specifications.

The Value of Interim and Formative Assessment Solutions

Improving the quality of teaching and learning in K–12 schools is a national priority. To help all students excel in the rapidly changing global economy, educators have set proficiency goals, standards, and accountability exams.

Yet, today’s learners come to the classroom with diverse needs—from various socioeconomic and academic backgrounds to differing proficiencies in English language acquisition. In this unpredictable and changing environment, teachers and administrators also face a unique challenge: an expectation to improve achievement for every student at every level, but limited instructional time to understand and address individual student strengths and areas of need.

At the center of this is the classroom—the nexus of teaching and learning. To achieve better learning in the classroom, school districts are increasingly turning to interim and formative assessments.

Interim assessments help you construct a complete picture of student achievement by delivering periodic benchmarks of student progress. By linking assessment data to instruction, you can intervene early and effectively to actively shape how that achievement picture develops—from the individual student to the class, school, and district levels. Interim assessments can also include valuable predictive information.

Formative assessments are integrated with instruction and enable you to monitor and guide students’ progress throughout the school year. Using valid, reliable measures of student performance, you and your teachers can easily capture multiple snapshots of student learning and adjust instruction as necessary.

Top considerations

  • Assessment solutions can be used in a formative manner to quickly help your teachers diagnose student strengths and areas of instructional need
  • Prebuilt diagnostic and predictive assessments are available
  • Clear blueprints outline alignment of assessments to your state standards. When selecting an assessment, look for a thorough description of the production and editorial processes that determined assessment content
  • Predictive assessments reflect your state accountability exam in terms of structure and item types. Both selected- and constructed-response item types should be available (if applicable to your state exam).
  • Items are carefully crafted for each grade level and measure students using developmentally appropriate content
  • Helpful reports show item statistics, such as p-values of items, and allow educators to compare class performance to school and district, and school performance to district
  • Content coverage is carefully targeted for each grade level of the assessment
  • Algebra content reflects the Principles and Standards for School Mathematics as outlined by the National Council of Teachers of Mathematics (NCTM)

Data-Driven Teaching and Decision Making

You and your teachers need immediate access to student results in a format that supports informed classroom teaching and decision making throughout the school year.

A quality assessment solution should include best-of-breed solutions for assessment, so you receive valuable decision support and can aggregate and disaggregate data for application throughout your school or district.

Today, accountability requirements demand that data are translated into valuable information educators can immediately use to improve student achievement in the classroom. The right interim and formative assessment solution is a critical spoke in the hub of your student data system and ensures that quality information will drive informed decision making and lead to student achievement.

Interim and formative assessment solutions help you see previous student results, but they can also outline where students are headed and provide the resources to promote further achievement.

Using valid, reliable measures of student performance, assessment solutions should allow you to easily capture snapshots of student learning throughout the year and construct a complete picture of student achievement. Using targeted instructional resources and activities, you can actively shape how that picture develops—from the individual student to the class, school, and district levels.

Top considerations

  • Assessment components are closely aligned to your state standards
  • Solution gives you and your teachers the opportunity to create custom assessments
  • Flexible administration options include online, paper-and-pencil, and/or student response devices (or clickers)
  • Instructional resources are easily assigned from within student and class reports
  • Performance levels allow for compliance with Title I requirements

Importance of Strong Research Foundations

Quality feedback results from using valid and reliable assessments. Assessments must be based on statistically sound measurement models and appropriate research and development.

A rigorous development and review process by a trusted provider will ensure that assessments comprise only high-quality items. Meticulous planning and development are required. The process should begin with careful construction by assessment and content experts, followed by an examination for bias and fairness, and empirical data analyses to support inferences drawn from the items about student strengths and instructional needs.

Top considerations

  • Innovative, research-based assessment construction standards support valid and reliable assessments
  • Assessments are based on statistically sound measurement models
  • Predictive tests are scaled using state-of-the-art Item Response Theory (IRT) methodology
  • Assessments are developed by doctoral-level research scientists with backgrounds in psychometrics, statistics, and educational measurement
  • Assessments are continually enhanced to meet the changing needs of your classrooms and reflect your current state standards
  • Diagnostic assessment content matches your curricula and instructional practices

When looking for a new assessment program, or considering a supplement, be sure to discover whether empirical data from students in your state are used to select the best functioning items for the final assessment forms. Numerous item reviews, specific to your state content standards, will ensure accurate skill assessment and measurement of student growth relative to state standards.

The Basics of Test Theory

Valid and reliable assessments result from the application of best practices identified through decades of research in the area of educational measurement. Such assessments provide confidence that the results you receive will help you with instructional decision making in your class, school, or district. Classical Test Theory and Item Response Theory are used to develop high-quality assessments.

Classical Test Theory analyses—such as p-values, distractor analyses, point biserial correlations, Mantel-Haenszel differential item functioning (bias indices as sample sizes permit), and test reliability coefficients—should be applied to the creation of your assessment to support the quality of the items and accurate measurement of each student relative to each content area.

Item Response Theory (IRT) is a mathematical model that describes the relationship between a student’s level of achievement and the likelihood of a correct response on each test item. The use of IRT provides more accurate scoring and supports developmentally appropriate assessments by matching student abilities with the difficulty level of the assessment. IRT uses student data to determine empirically—that is, based on statistical observation—how much information is provided by each item. IRT models use information about each item, called item parameters, to provide accurate measurement of student achievement.

Top considerations

  • Items are developed and reviewed by assessment experts
  • Predictive assessment items are selected based on Item Response Theory analyses
  • Predictive assessments use the three-parameter logistic (3PL) model to characterize selected-response, or multiple-choice, items in terms of three parameters:
    • Item difficulty, which refers to the difficulty of the item
    • Item discrimination, which reflects the ability of the item to differentiate between higher and lower achieving students
    • Guessing, which indicates the likelihood of a correct response from students with very low levels of achievement
  • A Two Parameter Partial Credit (2PPC) model is often used for developing, scaling, and scoring constructed-response items. 2PPC defines a constructed-response item in terms of two parameters:
    • Item difficulty
    • Item discrimination

CTB was a pioneer in the practical application of IRT to large-scale assessment in the 1970s and continues to lead the industry today. Research conducted by CTB indicates that the 3PL model yields more accurate information than the 1PL model, which is why it is the default model used at CTB, and more specifically, for Acuity.

Predictive Assessments: Your Pathway to Success

In choosing an assessment solution, it’s important to consider some basic information about predictive assessments.

High-quality assessment solutions help educators make the right decisions by indicating your student population’s growth and progress toward meeting end-of-year goals relative to your state accountability exam.

The best assessment solutions use item content that reflects your state accountability exams to help you prepare students to achieve.

Top considerations

  • Assessments can provide predictions of student performance relative to your state accountability exam by reflecting your state test blueprints
  • Forms are clearly designed to be administered in stages prior to your state accountability exam
  • Predictive assessments monitor growth and progress by measuring student achievement on a common scale within and across grades
  • Predictive information is reported in a variety of ways using graphs and tables on user-friendly reports
  • Assessments include content that is developmentally appropriate for the specific time you are assessing students

The Value of Predictive Information

In addition to providing students with valuable practice, predictive assessments include forecasts of how students will likely perform relative to your state exam.

Why is predictive information important for your classrooms? This information supports targeted classroom instruction—in a timeframe that allows educators to respond to the results and adjust instruction to improve student achievement. Quality predictive information preserves classroom instructional hours by helping to focus teaching where students most urgently need intervention.

Top considerations

  • Predictive performance levels clearly indicate the expected level of achievement on your state accountability test
  • Information includes a point estimate and its corresponding performance level (e.g. Below Basic, Basic, Proficient, Advanced) using the performance level descriptions used in your state
  • Predictions are displayed using tables and easy-to-understand graphs and descriptions
  • Predictions are based on empirical data using rigorous statistical analyses

Customization Opportunities

Built-in customization for maximum flexibility should be a top consideration for your assessment program. The ability to create custom, state-specific assessments that are aligned to your standards can give you a multitude of ways to measure student progress. Look for assessment solutions that allow your teachers to create and share assessments with other educators in your school or district.

Item banks should include assessment items across multiple grades, content areas, and standards. Best-in-breed solutions will also include notations for distractor analysis that indicate common misconceptions.

Item authoring tools should be integrated within your assessment system to allow you and your colleagues to write your own selected- and constructed-response items. Permission-based tools should help you efficiently produce, manage, and organize initiatives.

Top considerations

  • Assessment solution includes a sizable item bank aligned to your state standards
  • Item authoring tools are available to allow you to further customize assessments
  • Item banks and item authoring tools are supported by a robust custom test builder
  • Science items include innovative item types to help educators assess and promote learning and success in science
  • Solution allows you to import test items automatically from ExamView® and other item banks

Reports: Assessment Data for Effective Teaching and Learning

The assessment solution you select should also deliver detailed reports that inform instruction in clear, easy-to-understand formats. You will want a selection of helpful reports that provide the data you need to take specific actions to improve student achievement.

Reports should allow you and your teachers to quickly and efficiently measure student progress and growth at the student, class, school, and district levels. You should be able to use the reports to track student growth within or across grades, or to disaggregate student performance according to your subgroups or demographic categories.

Unlike generic reports that simply state how many questions a student answered incorrectly, look for reports that pinpoint exactly where a student is having difficulty. Your teachers can use this data to adjust their classroom instruction, and organize small group or individual instruction to immediately address areas of concern.

For example, reports should help your teachers understand why a student chose a wrong answer or “distractor”. Using this data, your teachers can efficiently group students with similar learning gaps and provide tailored instruction.

Assessments should yield data that allow you to track student growth longitudinally, that is, within and across grades.

Top considerations

  • Report versions are suitable for many roles within your district, with information displayed for individual students, classrooms, schools, and districts
  • Reporting aligns results and information specifically to your state standards
  • Reports are designed to be actionable with information included in easy-to-interpret formats
  • Student and class-level online reports include drill-down features that allow for analysis of data down to the standard, item, and individual student levels
  • Solution supports item-level reporting of p-values at the student, class, school, and/or district level and comparisons of class performance to school and district, and school performance to district
  • Longitudinal reporting is available to report progress within and across grades
  • Distractor Analysis is available to help your teachers discern WHY students may have answered questions incorrectly
  • Reporting is available for a variety of demographic subgroups (e.g., Title I, SES, etc.)
  • Results can be exported to your school or district data management system
  • Reports are available online through a helpful data management system, but also in printable PDF format

Linking Assessment with Instruction

Your teachers might frequently ask, “Now that I know the needs of my students, what’s next?”

Assessments should help you and your teachers answer this question by providing linked instruction that can be assigned to students individually or as a group from student assessment results, so assessment leads directly to appropriate instruction.

Instructional components that are closely coupled with assessment data can help you easily deliver targeted, personalized instruction by skill or standard.

Instruction should be designed to engage your students, and provide extra practice and intervention in time to make a difference. Real world content, sound, and graphics further motivate students to achieve. Acuity also includes an individual student report so students are keenly aware of what skills they have mastered—and where they need the most help.

Top considerations

  • Assessment system includes embedded instructional intervention resources aligned to assessment results and your specific state standards
  • Solution enables you to search for all instructional resources mapped to a particular standard and skill and assign any of those resources to an individual student or group of students
  • Custom items and assessments can be linked to the instructional resources available in the system
  • Instruction can be delivered online and/or in paper-and-pencil formats

Importance of Professional Development and Support

To ensure successful use of an assessment solution, a provider should offer a complete package of implementation, professional development, and technical support.

You and your educators will need ongoing support to help your schools get started quickly and smoothly, to answer questions, and address any issues.

Professional development and training is critical for your teachers and district administrators to maximize your investment in an assessment solution.

Professional development should include a flexible menu of training options—from web-based delivery to on-site workshops and customized training developed around your specifications.

Top considerations

  • Support is offered in the form of a self-service website that allows users to search for answers, log a case or question, view helpful “how-to”instructional videos, download robust user documentation, and review additional online help features
  • Comprehensive online help is available
  • Convenient telephone and email support is available
  • Additional offerings, like an implementation guide and an interactive video, are available to help during the implementation of the system
  • An online community is available to help you provide feedback to the publisher and connect with other users to share tips and tricks and best practices
  • Provider has a proven track record for successful implementation in districts of similar or greater size and demographics
  • Professional development is available for a wide range of audiences and levels and topics (i.e. Data Analysis, Report Interpretation, Creating Quality Items and Assessments, etc.) and in a number of formats from Web-based to on-site training offerings. Professional development is also available throughout your implementation on a variety of topics.

Drive Student Achievement with the Award-Winning Acuity InFormative Assessment Solution

Developed by CTB/McGraw-Hill, a trusted leader in educational assessment and reporting for more than 80 years, the Acuity InFormative Assessment solution meets the highest standards for quality assessment.

With a staff of doctoral-level researchers, scientists, mathematicians, writers, and editors, CTB applies the best practices used in high stakes item and test development to Acuity. When state standards change, the company updates the assessments to meet your changing needs. Superior test items and assessment forms result in more accurate indicators of student progress for informed decision making at all levels.

Assessment for learning

Acuity helps you target classroom instruction with five integrated components aligned with your state standards: predictive assessments, diagnostic assessments, instructional resources, a state-aligned item bank, and a custom test builder with item authoring tools.

Acuity helps you boost student achievement by:

  • Diagnosing areas for student improvement
  • Predicting student performance on your state exams using research-based measurement and robust reporting
  • Delivering easy-to-use, actionable reports at the student, classroom, school, and district levels
  • Supporting targeted intervention by delivering instructional resources aligned with assessment results and your state standards
  • Aligning with common Response to Intervention (RTI) models, delivering progress monitoring tools, and customized assessment opportunities The Acuity InFormative Assessment system supports high-quality, informed instruction and fits a range of assessment needs, from formative to interim.
  • Flexible modules across the grades and content areas
  • Reading/language arts, math, and science: Grades 3–8
  • Algebra: Grades 6–12

Visit us online at AcuityForSchool.com to view a product tour or call 800.538.9547 to speak with an Assessment Solutions Consultant.

Copyright © 2010 by CTB/McGraw-Hill LLC. All rights reserved. Acuity is a registered trademark and InFormative Assessment is a trademark of  The McGraw-Hill Companies, Inc. All other products and brand names are properties of their respective owners and are not affiliated with this publisher.

The Acuity InFormative Assessment Approach

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s