and Electronic Learning Assessment Resources (ELAR)
By John Cradler
Introduction: What is Data-Driven Decision-Making (DDDM) and what should be considered before selecting and implementing Electronic Learning Assessment Resources (ELAR)? The following questions should be understood and/or addressed prior to making the time and funding investment to use an ELAR in your school or district:
- A. What are Data-Driven Decision-Making (DDDM) and Data Literacy?
- B. Why is DDDM important?
- C. Why is data literacy important?
- D. What is necessary for an education agency to implement DDDM?
- 1. Establishing a district/school level commitment to DDDM.
- 2. Implementing the California curriculum content frameworks and related standards.
- 3. Using objective assessments to determine the extent to which students attain the standards.
- 4. Using local assessments and related data to plan for individual student needs.
- 5. Considering information about the demographic and instructional variables that influence student performance as determined from the academic assessment components of DDDM.
- 6. Developing instructional units and lessons based on DDDM.
- 7. Making relevant assessment data available to educators for planning.
- 8. Providing professional development and ongoing support for school administrators and teachers in using assessment information to inform instructional planning at the district, school, and classroom levels.
- 9. Providing for technology applications that will equip educators with the current student assessment data on demand and in a user-friendly format.
- E. How does CLRN’s ELAR information resource support DDDM?
- F. What types of Electronic Learning Assessment Resources are reviewed?
A. What are Data-Driven Decision-Making (DDDM) and Data Literacy? DDDM involves the use of student assessment data and relevant background information, to inform decisions related to planning and implementing instructional strategies at the district, school, classroom, and individual student levels. “Data literacy” means that a person possesses a basic understanding of how data can be used to inform instruction.
B. Why is DDDM important? Research shows that if instructional plans at the state, county, district, school, classroom, and individual student levels are based on assessment information relevant to the desired learning outcomes for students, the probability is increased that they will attain these desired learning outcomes.
C. Why is data literacy important? Educators must understand that assessment of student performance is integral in the planning, implementation, assessment, and revision of instruction. Data literacy also implies that the educator must be able to determine whether or not an assessment is a valid and reliable measure of what is being taught and to know what types of assessments are appropriate for district level vs. classroom or individual student level planning. What is commonly referred to as a “cycle of inquiry” illustrates the basic steps in the application of data to inform instructional decision-making.
D. What is necessary for an education agency to implement DDDM? Teachers and school administrators need to be data literate. This can happen only when the district/school leadership actively encourages and assists educators in understanding the importance of relating valid student assessment information to instructional practice. A culture must be created in which student information is considered essential to improving student results, and education policy makers, education leaders and practicing educators agree on the following:
1. Establishing district level commitment to DDDM: The school district should adopt policies and strategies to ensure that all necessary resources and support services are funded and implemented to guide planning and related instruction through use of objective and relevant student assessment information. School leadership must understand that use of objective and relevant student assessment data will significantly increase the probability that learning will occur. This will result when cause and effect are understood, and when using student information to ask “Why?” and “What can we do better?” becomes standard in the school.
2. Implementing the California curriculum content frameworks and related standards: Implementation of the state content standards is a mandate at the state and federal level as dictated by No Child Left Behind. All school districts in California are adopting the California Content Standards and purchasing instructional materials that align to these standards. It may be assumed that the majority of instructional time is allocated to teaching to these standards.
3. Using objective assessments to determine the extent to which students attain the standards: The state has adopted the California Standards Test (CST) as a measure of the extent to which groups of students meet standards at the state, district, and school levels by grade level in English Language Arts, Math, and Science. The California High School Exit Exam (CAHSEE) is a proficiency assessment used to determine whether or not a student is eligible to graduate from high school. It is recognized that these assessments do not provide the information needed to determine specific instructional activities at the classroom or student level. However, the state assessments can be used to determine overall gaps in how a particular school compares with schools of similar demographics at a particular grade level in the subject area tested. The state assessments can be used to inform DDDM by assisting educators to identify a gap or area where students at a school are generally not meeting the state standards, and then moving to a deeper level of investigation. This means using alternative sources of data to help define the specific instructional needs at the classroom and student levels.
4. Using local assessments and related data to plan for individual student needs: Now that state assessment has helped pinpoint the school(s), grade level(s), subject area(s), and perhaps classrooms needing additional assistance, it is necessary to move to local and alternative assessments. Such assessments provide sufficient detail enabling educators to develop specific educational plans or plan revisions to address the general deficiencies identified by the state assessments. Following are some assessment strategies that school districts have adopted to enable DDDM for the purposes of classroom and student level instructional planning informed by assessment data.
- a. Locally developed performance tests based on the state content standards;
- b. Performance measures embedded in the adopted instructional materials used in the school;
- c. The use of standards-aligned rubrics to judge student performance;
- d. Selection of standards-aligned test items from published item-banks for the construction of locally relevant academic assessments;
- e. Teacher-assigned grades assigned based on guidelines that reflect state content standards;
- f. Other measures to be determined.
5. Considering information about the demographic and instructional variables that influence student performance as determined from the academic assessment components of DDDM: The use of student data alone is not sufficient to design instructional plans that effectively address the individual instructional needs of students. Information systems for DDDM must also provide data on student demographics: the amount of time in the district, socioeconomic status, achievement motivation and other behavioral indicators, parent involvement, attendance, and other factors as needed. Other considerations in DDDM are conditions under which instruction can be provided: availability of instructional resources, class size, access to technology, adequacy of school facilities, etc. These factors are often referred to as “opportunity to learn” variables.
6. Developing instructional units and lessons based on DDDM: In order for data to influence instructional practice, the school must institute a process whereby teachers continuously monitor assessment data and other information relevant to student academic performance, and translate that information into the delivery of curriculum and instruction. This component of DDDM requires the greatest effort on the part of educators, as the design and re-design of instruction based on student assessment data is time consuming and tedious. However, recent developments in technology applications can make this task less burdensome. Some basic considerations in preparing or revising instructional units and practice based on DDDM are:
- a. Are instructional unit objectives linked to the state content standards?
- b. Are technology-based and other supplements linked to the adopted text?
- c. Are assessments embedded into the instructional resources linked to the instructional unit objectives?
- d. Are the results of available state assessments considered and linked to any local assessments and are these referenced in the unit?
- e. Is periodic assessment of student performance provided, recorded and linked back to the unit objectives?
- f. Are the assessments valid in accurately assessing what students learn in relation to the standards?
- g. Is there a commitment to implementing units of instruction and administering the unit-prescribed assessments as scheduled?
- h. Do teachers collaborate on students’ progress and compare notes on using assessments in planning instruction?
- i. To what extent is technology used to help automate the design and monitoring of instructional units?
- j. Are teachers provided with the time and resources needed to plan, modify, and implement units based on DDDM?
- k. Is there significant value, evidenced by teachers and the school level administrators, in using student data and background information to inform instructional planning?
7. Making relevant assessment data available to educators for planning: The school district must be committed to using DDDM as a pre-requisite to the effective and sustained use of data to inform instructional planning at the school and classroom levels. This means the district will make the funding and staff time available to assist schools in the use of DDDM. It also means that district staff will make every effort to make the relevant assessment and demographic data and information available to schools when they need it to do planning. This includes the provision of longitudinal student data so that change in student performance over time can be determined and used in the local DDDM instructional planning.
8. Providing professional development and ongoing support for school administrators and teachers in using assessment information to inform instructional planning at the district, school, and classroom levels: As with any new initiative, the school district must provide the professional development necessary to build the capacity and desire of each educator to utilize DDDM. Professional development must extend beyond simply using data to inform others about student performance. It must ensure that data is used to design and revise instructional practice so that it more closely conforms to the state standards and results in increased student performance. Finally, educators need to be committed to DDDM as a strategy to help students improve their performance to the extent that the “performance gaps” or deficiencies documented by the CST and other state measures are eliminated.
9. Providing for technology applications that will equip educators with the current student assessment data on demand and in a user-friendly format: Staff and evaluators for the California Learning Resource Network (CLRN) have determined that a wide variety of technology applications are available, with many more emerging, that are intended to assist educators in making data-driven decisions as suggested. For this reason a new service has been added to CLRN. That is a “clearinghouse” of information that will help educators identify and select an Electronic Learning Assessment Resources (ELAR). These assessment tools provide assistance in accessing, analyzing, and making data-driven decisions using student data, and range from the district to the classroom levels.
E. How does CLRN’s ELAR information resource support DDDM? The ELAR component of CLRN provides an online “consumer guide” intended to assist educators in selecting data management resources which meet their local needs and state and federal reporting requirements. The ELAR component of CLRN also provides an emerging source of information resources to help educators become data literate and learn how to make the most effective use of an ELAR. The ELAR resource provides:
- a. A “one-stop” web-based location to review data management systems based on approved features
- b. State approved features enabling educators to compare/evaluate their current instructional management systems
- c. Research on effective use of data management systems in instructional decision-making
- d. Links to web resources supporting data-driven decision-making.
- e. Answers to frequently asked questions when selecting an ELAR.
- f. Professional development through the California Technology Assistance Projects (CTAP) and the Statewide Educational Technology Services (SETS).
F. What types of Electronic Learning Assessment Resources are reviewed by ELAR? The ELAR staff and advisors have adapted the definitions established by CTAP 4 in collaboration with the Stupski Foundation* to describe the categories relating to ELAR reviews. A few ELAR programs incorporate all five categories but most address fewer than the following five categories:
- 1. Data Warehouses: Data warehouses are large integrated databases that connect relevant information from other sources into a single accessible format. They allow users to look at data from multiple operational systems across multiple dimensions, including time. These systems have the advantage that they can import and analyze data from a variety of other systems that cannot otherwise communicate with each other.
- 2. Data Analysis & Reporting Systems: These systems provide the ability to examine aggregate student performance measured across a wide array of variables and oriented to any scale: district, school, program or classroom. These systems are designed primarily as analytical and reporting tools for district and site administrators, and can also provide useful summary data for classroom teachers.
- 3. Curriculum Management Systems: Curriculum management systems are designed to help teachers and site administrators plan and monitor the instructional program. These systems integrate teachers' curriculum planning, lesson plans, assessments and grade reporting into a standards-based system linked with data from the school's student information system.
- 4. Instruction & Practice Systems: Instruction and practice systems are designed to help teachers align their curriculum and instructional resources to state and local standards. These systems typically provide teachers standards-aligned lesson plans, skill sets and assessments and a standards-based reporting system to supplement the resources a teacher already uses.
- 5. Assessment and Diagnostic systems: These systems provide assessment tools to determine student strengths and weaknesses in various academic areas aligned with state curricular standards. They provide feedback and practice to students and teachers in what they should know and be able to do in relation to the standards. Assessment and diagnostic systems sometimes are imbedded into electronic learning resources and, in some cases, can be linked a specific ELAR.
G. What types of technology-based data management programs are not considered an ELAR?
- 1. Operational Data Systems (Student Information Systems): Operational systems are concerned primarily with day-to-day administration issues, such as tracking student attendance, grades, course enrollment, etc. SASI and Eagle are examples of operational data systems, which are transactional systems that update student data as soon as it changes, usually erasing older data. Although some analytical functions typically are built into operational data systems, they generally do not meet all of a district's data analysis needs. Many ELAR programs will import student demographic data from a SIS.
- 2. Direct Instructional Systems: Direct instructional systems, such as Accelerated Math, provide educational content directly to students and teachers in the classroom via electronic media. CLRN classifies these direct instructional systems as Electronic Learning Resources (ELR) as differentiated from Electronic Learning Assessment Resources (ELAR). CLRN has reviewed at least 40 ELR which have content-related embedded student assessment systems. In some cases those assessments can be linked to ELAR.
The ELAR website address is http://www.clrn.org/elar/
For additional information regarding this report and/or implementation of ELAR contact:
- John Cradler, Educational Support Systems and External Evaluator for the CLRN Statewide Educational Technology Service. firstname.lastname@example.org
- Bob Beuthel, Learning Resources and implementation manager of the ELAR service component of CLRN. email@example.com
- Ellis Vance, Stanislaus County Office of Education and Director of the California Learning Resource Network (CLRN). firstname.lastname@example.org