My name is Austina De Bonte. I co-chair the Northshore Highly
Capable Parent Advisory Board and am President of the state-wide
organization NW Gifted Child Association. My goal is to continue educating the
board on issues related to the Challenge program.
Today I will speak about a pervasive problem that affects the validity of educational research claims made about high achieving students. Because the middle school task force is relying heavily on this kind of research, this is of material interest for the Challenge debate. The central claim made by the AMLE research is that lower ability students do better in heterogeneous classes, while there is no impact on achievement levels of high ability students. On paper it sounds really good – everyone wins.
But the problem is that nearly all modern educational research is flawed in the same fundamental way. Because of this flaw, any claims that you hear about the achievement of high ability learners are highly suspect.
Here's the problem. The tests that we use to measure student achievement do not go high enough to measure the actual achievement levels of our high ability learners. This is called the ceiling effect, and nearly all of our standardized tests for decades have had this problem.
Here is a case in point. I just received a letter a few weeks ago with my then 6th grader's smarter balanced test results. It turns out that she got the maximum score on the math portion, 2748. I asked around and it turns out that lots of kids we knew got the same top score. One of my colleagues in NWGCA investigated this, and found that statewide, OSPI reports there are 972 6th graders, 1024 7th graders, and 1679 8th graders who got the top score on the SBA math test, which amounts to close to 2% of the population, and you’ll notice it becomes a bigger problem through the middle school years. The numbers are similar for English Language Arts. I say this not to brag, but to point out that even this testing instrument cannot distinguish between different levels of performance of our high ability students -- because they all hit the ceiling of the test.
This is despite the fact that the SBAC was supposed to have a higher ceiling, supposedly two grade levels higher, and is supposed to be dynamically presenting more difficult problems if the kid was passing the standard material. Yet even with this next generation, higher ceiling instrument, the SBAC still has the same problem that has plagued standardized tests for decades. Shockingly, the ceilings of previous generation instruments were even lower.
Why does this matter? Because the ceiling effect invalidates nearly every educational research study that makes a claim about the achievement of high ability learners. Rather, what these studies are often telling you is that the highest ability learners maxed out the test in both the control and the variable situation. So they conclude that there is "no impact" on high ability learners. The reality is that they have no idea what the actual impact was - positive or negative - because the test they used simply can't measure it.
While there is valid research available about high ability students using instruments like out of level tests like the SAT for middle school students, this research is shared only among a small group of “gifted & talented” experts, and is rarely discussed broadly, and so there is a fair amount of institutional ignorance on this issue. So, major methodological issues like this ceiling effect are passed over as being impossible to address given current broadly available testing instruments, and not important to the validity of the results anyway since the focus is on improving the achievement gap, and are rarely called out.
The bottom line is that to really assess the validity of the AMLE research on this point, you'd need to investigate each individual study to see how they measured student achievement to come to their conclusions. As you are looking at research presented, please look carefully at how achievement was measured. If it was via a typical standardized test or worse, by whether the student an A in the course, any claims they make about high achieving students is highly suspect.
Today I will speak about a pervasive problem that affects the validity of educational research claims made about high achieving students. Because the middle school task force is relying heavily on this kind of research, this is of material interest for the Challenge debate. The central claim made by the AMLE research is that lower ability students do better in heterogeneous classes, while there is no impact on achievement levels of high ability students. On paper it sounds really good – everyone wins.
But the problem is that nearly all modern educational research is flawed in the same fundamental way. Because of this flaw, any claims that you hear about the achievement of high ability learners are highly suspect.
Here's the problem. The tests that we use to measure student achievement do not go high enough to measure the actual achievement levels of our high ability learners. This is called the ceiling effect, and nearly all of our standardized tests for decades have had this problem.
Here is a case in point. I just received a letter a few weeks ago with my then 6th grader's smarter balanced test results. It turns out that she got the maximum score on the math portion, 2748. I asked around and it turns out that lots of kids we knew got the same top score. One of my colleagues in NWGCA investigated this, and found that statewide, OSPI reports there are 972 6th graders, 1024 7th graders, and 1679 8th graders who got the top score on the SBA math test, which amounts to close to 2% of the population, and you’ll notice it becomes a bigger problem through the middle school years. The numbers are similar for English Language Arts. I say this not to brag, but to point out that even this testing instrument cannot distinguish between different levels of performance of our high ability students -- because they all hit the ceiling of the test.
This is despite the fact that the SBAC was supposed to have a higher ceiling, supposedly two grade levels higher, and is supposed to be dynamically presenting more difficult problems if the kid was passing the standard material. Yet even with this next generation, higher ceiling instrument, the SBAC still has the same problem that has plagued standardized tests for decades. Shockingly, the ceilings of previous generation instruments were even lower.
Why does this matter? Because the ceiling effect invalidates nearly every educational research study that makes a claim about the achievement of high ability learners. Rather, what these studies are often telling you is that the highest ability learners maxed out the test in both the control and the variable situation. So they conclude that there is "no impact" on high ability learners. The reality is that they have no idea what the actual impact was - positive or negative - because the test they used simply can't measure it.
While there is valid research available about high ability students using instruments like out of level tests like the SAT for middle school students, this research is shared only among a small group of “gifted & talented” experts, and is rarely discussed broadly, and so there is a fair amount of institutional ignorance on this issue. So, major methodological issues like this ceiling effect are passed over as being impossible to address given current broadly available testing instruments, and not important to the validity of the results anyway since the focus is on improving the achievement gap, and are rarely called out.
The bottom line is that to really assess the validity of the AMLE research on this point, you'd need to investigate each individual study to see how they measured student achievement to come to their conclusions. As you are looking at research presented, please look carefully at how achievement was measured. If it was via a typical standardized test or worse, by whether the student an A in the course, any claims they make about high achieving students is highly suspect.
No comments:
Post a Comment