澳门跑狗论坛

Assessment

Software Said To Grade Essays for Content

By Debra Viadero 鈥 April 22, 1998 3 min read
  • Save to favorites
  • Print
Email Copy URL

Researchers at the University of Colorado at Boulder have developed a computer-software program that they say can grade student essays for content just as well as teachers can.

鈥淲e鈥檙e not seeing this as a replacement for teachers,鈥 said Peter W. Foltz, one of three developers of the program, known as Intelligent Essay Assessor. 鈥淏ut we think it may help teachers have more time to interact with students.鈥

Thomas K. Landauer

The program was created and tested over a period of 10 years by psychologists at the university, who initially sought to create a computer model that would mimic the way people learn about words and language. But, along the way, they said, they discovered that their theoretical model could also be used for more practical purposes, such as grading essays or giving students diagnostic practice in summarizing material.

Since then, the researchers have tested the program with almost 1,200 essays.

鈥淔rom 6th graders to first-year medical students, we get consistently good results,鈥 said Thomas K. Landauer, a University of Colorado psychology professor and the project鈥檚 lead investigator. 鈥淭he agreement between a human and the machine is about exactly the same as between two humans.鈥

But unlike humans, the computer will always give the same grade to the same essay, the researchers said.

鈥淚t doesn鈥檛 matter what else it was doing that day or if it was tired,鈥 said Darrell Laham, a University of Colorado doctoral student who鈥檚 also taking part in the project. 鈥淚t doesn鈥檛 ever have a bad day.鈥

The findings were presented here last week at the annual meeting of the American Educational Research Association.

Researchers have experimented for years on ways to use computers for grading essays. Ellis B. Page, a Duke University professor of psychology and education, has developed a program called Project Essay Grade, or peg, which assesses the quality of writing in students鈥 essays. It has shown similarly good results. (鈥淢aking the Grade,鈥 May 31, 1995.)

But the program unveiled last week may be among the first to grade essays for content.

The Colorado researchers have applied for a patent for the program, but say they may be a few years away from marketing it commercially. The software requires a computer with about 20 times the memory of an ordinary personal computer.

The researchers predict that in five years, however, the program will be able to run on most computers.

More Time for Teachers?

Ultimately, researchers say, such software could be a boon for professors and teachers as well as for commercial testing companies that grade thousands of exams.

鈥淭eachers in middle schools report that they spend six-and-a-half hours a week grading essays, and high school teachers spend more than ten-and-a-half hours,鈥 Gary B. Struck, a professor of educational psychology at the University of North Carolina at Chapel Hill, said during a conference session on the PEG project.

In addition, many teachers lack confidence in the grades they give on essays and would like some means of assessing their scoring, he said. 鈥淭hey鈥檇 like to have some kind of comparison.鈥

The developers of Intelligent Essay Assessor said their program might also be a useful supplement to textbooks published by commercial textbook companies. Students could use the program to test their knowledge of the textbook material.

Hard To Beat

The new software program grades essays by evaluating how closely the students鈥 words approximate content already fed into the program through textbooks and model essays that professors have already graded by hand.

鈥淵ou could write something like, 鈥楾he doctor operated on the patient,鈥 and if the text said, 鈥楾he physician did surgery,鈥 the computer would see those things as very similar,鈥 said Mr. Foltz, who is also an assistant professor of psychology at New Mexico State University.

Longer essays also get better grades because researchers have found that essay length correlates closely with the quality of the content.

Critics of the program have suggested that students could easily fool the computer by including the right content but putting it in the wrong order.

Students could still get a good grade that way, the researchers said, 鈥渂ut it鈥檚 hard to say good ideas without saying it well,鈥 Mr. Foltz said.

What鈥檚 more, the computer flags essays that are 鈥渢oo creative鈥 or that contain a lot of inconsistencies. It brings such essays to the attention of human graders. Plagiarized essays also get flagged.

The researchers have spent a lot of time trying to fool the system, Mr. Landauer said, but 鈥渢he easiest way to cheat this system is to study hard, know the material, and write a good essay.鈥

Events

Artificial Intelligence K-12 Essentials Forum Big AI Questions for Schools. How They Should Respond鈥
Join this free virtual event to unpack some of the big questions around the use of AI in K-12 education.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of 澳门跑狗论坛's editorial staff.
Sponsor
School & District Management Webinar
Harnessing AI to Address Chronic Absenteeism in Schools
Learn how AI can help your district improve student attendance and boost academic outcomes.
Content provided by 
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of 澳门跑狗论坛's editorial staff.
Sponsor
Science Webinar
Spark Minds, Reignite Students & Teachers: STEM鈥檚 Role in Supporting Presence and Engagement
Is your district struggling with chronic absenteeism? Discover how STEM can reignite students' and teachers' passion for learning.
Content provided by 

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide 鈥 elementary, middle, high school and more.
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.

Read Next

Assessment Massachusetts Voters Poised to Ditch High School Exit Exam
The support for nixing the testing requirement could foreshadow public opinion on state standardized testing in general.
3 min read
Tight cropped photograph of a bubble sheet test with  a pencil.
E+
Assessment This School Didn't Like Traditional Grades. So It Created Its Own System
Principals at this middle school said the transition to the new system took patience and time.
6 min read
Close-up of a teacher's hands grading papers in the classroom.
E+/Getty
Assessment Opinion 'Academic Rigor Is in Decline.' A College Professor Reflects on AP Scores
The College Board鈥檚 new tack on AP scoring means fewer students are prepared for college.
4 min read
The United States Capitol building as a bookcase filled with red, white, and blue policy books in a Washington DC landscape.
Luca D'Urbino for 澳门跑狗论坛
Assessment Opinion Students Shouldn't Have to Pass a State Test to Graduate High School
There are better ways than high-stakes tests to think about whether students are prepared for their next step, writes a former high school teacher.
Alex Green
4 min read
Reaching hands from The Creation of Adam of Michelangelo illustration representing the creation or origins of of high stakes testing.
Frances Coch/iStock + 澳门跑狗论坛