Standardized testing has been a staple of American education for more than a half century, and has become particularly prominent in public schools鈥攕ome might say invasive鈥攕ince the passage of the No Child Left Behind Act of 2001. But while it鈥檚 not likely that standardized tests in schools will be going away anytime soon, there are definite signs that the assessment field is in a period of transition鈥攂rought about by, among other factors, widespread dissatisfaction with conventional testing formats, advancements in technology, and the implementation of the Common Core State Standards.
To get some insight on what the near future of assessment might look like for teachers, we talked to James W. Pellegrino, a widely recognized expert on cognitive science, psychometrics, and educational technology. Pellegrino is co-director of the Learning Sciences Research Institute and a distinguished professor of education at the University of Illinois at Chicago. He was also co-chair of the Gordon Commission on the Future of Assessment in Education, which produced a high-profile report last year arguing that testing needs to be radically re-imagined and more closely integrated with classroom-based teaching and learning.
Pellegrino is on the technical advisory committees of Smarter Balanced and the Partnership for Assessment of Readiness for College and Career, the consortia charged with developing assessments aligned to the Common Core State Standards.
Teachers frequently complain that current standardized assessments are not adequate鈥攐r even very good鈥攎easures of their students鈥� learning or their own performance. Are they wrong about this?
I wouldn鈥檛 say they鈥檙e wrong. I think they have a legitimate concern. And their questioning of standardized tests really gets at some of the important aspects of what it is we want students to learn and be able to do and whether our assessments are doing a good job of measuring and fostering those things. So teachers are right to worry about this. Right now, we tend to rely on highly restrictive kinds of assessment formats. And we tend to focus on particular kinds of knowledge, like being able to retrieve facts or execute procedures, and not really on getting at deep understanding of important ideas and principles.
Are there reasons for teachers to be more positive about assessment in the near future?
Yes, I think that if you look at the efforts of the two consortia to develop the assessments that are aligned with the common core, what you see are attempts to represent more complex kinds of thinking, and reasoning, and performances. The simple fact that both Smarter Balanced and PARCC have performance tasks as a major part of their assessments is one indication that they鈥檙e trying to get at things that are represented in the new standards that aren鈥檛 easily measured in typical selected-response type questions.
What is your impression of how those assessments are playing out and how successful they鈥檒l be?
There鈥檚 been a lot of good work on item development, but there鈥檚 still a lot of field testing that needs to be done of the items and the tasks. In all honesty, I have to say that what the consortia were asked to do in four years鈥� time was a pretty herculean task鈥攖o try and develop assessments in grades 3-8 and high school in two subjects aligned to new standards and do it in a way that adequately got at the core concepts of what鈥檚 supposed to be mastered in the standards. So I think we鈥檒l find that assessments are less than what some people might have imagined was going to happen after four years of work. But those were unrealistic expectations. On the other hand, I think the new assessments will teach us a lot about building better assessments. They鈥檙e not going to be perfect, but hopefully they will move us forward in a way that enables us to more adequately measure student achievement and also teaches what we still need to think about.
What are the key issues in assessment that you think need to be addressed over the next several years?
I think we need to give a whole lot of thought and emphasis to the development of quality tools and materials that can support teachers in enacting assessment as part of a learning environment in the classroom. In other words, we need to spend less time being so preoccupied with the large-scale standardized accountability tests. We still need to have assessments that do that kind of monitoring, but I and a lot of other people in the assessment field think that we need to reverse the emphasis. We need to focus on assessment for learning at the classroom level. We are so hung up on assessment of learning for purposes of accountability or monitoring that we鈥檝e lost track of the investment we need in quality resources and materials to help teachers and help teachers help students.
The report by the Gordon Commission on the Future of Assessment in Education in which you were involved makes a similar point, specifically recommending 鈥渘ew assessment resources and tools that better integrate assessment with classroom teaching and learning and better represent current thinking on how students learn and on changes in the world at large.鈥� Any sense of what that would look like for teachers in the classroom?
Well, in some ways, really good assessment of the type we鈥檙e thinking about doesn鈥檛 look all that different from really good instruction鈥攐r the kinds of tasks we would want students to engage in the classroom as part of their learning. So there tends to be a misconception that we鈥檙e just talking about building a bunch of new tests. What we鈥檙e really talking about is building tasks and situations in which students engage in the kinds of thinking and reasoning we want them to do with texts or mathematical solutions or scientific concepts鈥攁nd engage in them in ways where they have to grapple with the key ideas. Some of this is about building better sets of tasks or scenarios that teachers can use as part of their instructional practice to help guide student learning. The tasks should allow students to show what they understand and do not understand鈥攁nd therefore help the teachers enact formative assessment practices. It鈥檚 complicated because it means that we in the assessment community have got to get much closer to the world of curriculum and instruction.
What鈥檚 the role of technology in this?
I for one believe that technology has a very big role to play, because I don鈥檛 think you can enact some of these more powerful uses of assessment at the instructional level without technology to help manage the process, to help manage the information, so that you don鈥檛 put the entire burden on teachers. If the teacher has to deliver the tasks, has to score the tasks, has to make sense of the evidence, and then figure out what to do about it, it鈥檚 not going to happen. The problem we鈥檝e always had in terms of teachers implementing formative-assessment practices is that it鈥檚 hard to do because there鈥檚 so much information to manage. Unless we can come up with technological tools and supports to help them manage that information鈥攁nd we do have systems like that now鈥攖hen I don鈥檛 think teachers have much of a chance to do the kinds of things we鈥檇 like them to do.
What are the specific systems available now that you would point to as models?
One good example is a program called 鈥淎SSISTments,鈥� which comes out of the Worcester Polytechnic Institute. This is a web-based platform where students can work on various kinds of items, and problems, and get feedback and help. The system keeps track of how students are doing and it can give information back to teachers so that teachers can know how their class is doing and take action accordingly. To me that鈥檚 a great example of how we reduce the burden on teachers and use technology in smart ways that don鈥檛 take the important decisions out of the hands of teachers, because the teachers still need to be able to make judgments as to how to pace and refine instruction.
Should teachers be worried about losing autonomy or responsibilities in the instructional scenario you鈥檙e envisioning, since a lot of the learning activities would be facilitated online?
Not at all. In fact, they have a great deal of control over how systems like this function. They select the types of tasks that students are asked to work on and then use the information from the system to decide where to spend their time in class. A lot of the learning that is facilitated in the online environment is focused on very specific topics or learning issues that teachers might be challenged to cover effectively for all students in the regular classroom.
Would you have any advice for teachers on adapting to or preparing for this kind of environment?
Well, I think my advice to teachers would be to implement these kinds of tools and technologies in a sort of incremental way. Take it a bit at a time. Don鈥檛 try to do everything at once. It takes professional development to use these kinds of tools and to change one鈥檚 teaching practice to incorporate them. School leaders and policymakers need to be respectful of teachers given the fact that they鈥檙e going to need time and support to do this. So I guess one piece of advice I鈥檇 give to teachers is, don鈥檛 try to go it alone. If you want to incorporate assessment tools like we鈥檙e talking about, do it as part of a community of learners so that you have a support structure.
Do you have any concerns about overtesting?
Oh, yeah! I think that in the United States, particularly when it comes to large-scale testing, especially since No Child Left Behind, we are testing too much. We test every child every year in math and English/language arts, and we鈥檙e doing that because we like to get scores on every kid so we can use those scores for all sorts of purposes, including teacher accountability. We need to break out of that mindset鈥攚e need to say, 鈥淟ook, we do need to monitor how well we鈥檙e doing in educating our children and how well they鈥檙e learning but it doesn鈥檛 necessarily mean that every kid, every year has to be assessed using large-scale, high-stakes tests.鈥�
As I鈥檝e been suggesting, a lot of what we want to know about student learning could be gleaned from information that could come out of the classroom. And if we want to monitor the system as a whole we could use more effective strategies like student sampling and matrix sampling of tasks. There are ways to do it that are more efficient to answer the kinds of questions that we want to have answered as a society, without requiring every kid, every year to take exactly the same test just so we ensure that Johnny鈥檚 score can be compared to Jimmy鈥檚 score.
What do you think testing will look like 20 years from now?
I think we will still have some legacy of what I call large-scale standardized testing in some form鈥擨 think it鈥檚 going to take a long time to shed that out of our educational DNA. You have to remember that the U.S. is the place that invented large-scale standardized testing, so it鈥檚 not something we can easily let go of. So I think we will still have some of that, but hopefully it will be used in more judicious ways, for monitoring purposes. Again, my sense鈥攎y hope鈥攊s that we will learn to use technology in more effective ways to gain the information we need about student learning out of classrooms and other contexts where students are doing their work, because much of the work will be mediated through technology. In this way, we鈥檒l hopefully be able to get better, more useful, and more timely information. So it won鈥檛 be that we鈥檙e constantly testing kids, but we鈥檒l be getting information from assessments of students鈥� performance that are more, you might say, ubiquitous as part of the work they鈥檙e already doing.
Here鈥檚 the metaphor I often use: It鈥檚 not perfect but I think it helps. Remember how, in the retail industry and major department stores, they used to have to close down stores once or twice a year to do inventory? That鈥檚 how they figured out what was selling and what they needed to restock. But then bar codes and scanners were invented. And now what happens is that every time you check out and buy something, inventory control is taken spontaneously. Stores are constantly accumulating information as to what鈥檚 being sold and what鈥檚 on the shelves, and they can do much better inventory control鈥攁nd they don鈥檛 have to shut down twice a year to have someone go in and count everything.
I think the analogy applies to the way we do assessment. I鈥檓 not advocating bar-coding kids, but in a world where much of the work that we do is mediated by technology and is done in technology-rich environments and learning systems, we could glean a lot of information spontaneously that would help us to track important things. And we could embed into those environments periodic assessments that students take to demonstrate their competence, instead of having to shut down the school once a year for multiple weeks while kids take large-scale tests.
That raises another question, especially in light of recent news: Do you have privacy concerns about that kind of framework?
Yes, absolutely. I think that when making that type of analogy, we need to be very careful about the issues of privacy and tact. One obvious concern is the potential for the sharing of information well beyond the bounds of those who need direct access to it. So checks and balances would need to be put in place to secure and monitor access to the data that鈥檚 collected.
But there鈥檚 also the danger of going overboard with respect to the collection of data. We certainly don鈥檛 want kids or teachers to feel like their work is constantly being used as evidence. This is not a Big Brother scenario we鈥檙e trying to create. We want kids to be comfortable doing school work online without being judged鈥攖hat鈥檚 part of the learning process. But then there are times when we want and need to monitor progress and collect evidence of learning to see if students have mastered important knowledge and skills. These wouldn鈥檛 be 鈥渟tealth鈥� assessments but rather ways for students to show they have mastered important standards in the context of their ongoing work. But we do need to be very clear about when and why we are collecting certain types of information.
