It’s important to use a consistent evaluation system when grading essay assessment items. This is important for several reasons. One reason is to ensure that all students are graded fairly and equally. Having clear and concise criteria for evaluation allows the grader to be certain this happens. Other ways to do this include grading without knowing the identity of the student, using a model answer, and grading all responses to a particular question at one time (Oosterhof, 2008).
The textbook provides 6 suggestions of criteria for evaluating essay items:
1. Does this item measure the specified skill?
2. Is the level of reading skill required by this item below the learner’s ability?
3. Will all or almost all students answer this item in less than 10 minutes?
4. Will the scoring plan result in different readers assigning similar scores to a given student’s response?
5. Does the scoring plan describe a correct and complete response?
6. Is the item written in such a way that the scoring plan will be obvious to knowledgeable learners?
The following will include a critique of the prompt and scoring article found on the ACT website (http://www.actstudent.org/writing/sample/textsamples.html ) using the 6 suggested criteria for evaluating essay items previously mentioned.
1. Does this item measure the specified skill? In this article a prompt is given but it does not include any indication of what the objective is. I am not sure how a grader would know if the objective of the essay was met or not. In this example the learner must answer the question given, but is allowed to choose a position on the subject. Knowledge of a particular subject is not needed to answer this question correctly. Based on the scoring explanation in the article, the learner’s writing ability and organizational skills are what is being evaluated.
2. Is the level of reading skill required by this item below the learner’s ability? Yes, I believe the required reading skill needed to understand this question is below the learner’s ability. The sentence structure is simple and does contain difficult or unusual words that may not be known by the learner.
3. Will all or almost all students answer this item in less than 10 minutes? Yes, this question should be able to be answered in less than 10 minutes. It is opinion based and only asks the learner to support their position using specific reasons and examples.
4. Will the scoring plan result in different readers assigning similar scores to a given student’s response? No, this example does not ensure that different readers will assign a similar score to a given student’s response. Three characteristics should be met to ensure reliable scoring. The first characteristic states that the number of points the item is worth should be known by the student at the time of the test. This is not the case in the example. The second characteristic states that the scoring plan should specify the attributes being evaluated. Again, this is not the case in the example. The third and final characteristic states that a scoring plan should indicate how points will be awarded. This is also not identified in the example. A good scoring plan should leave little doubt as to when a point should or should not be awarded.
5. Does the scoring plan describe a correct and complete response? No, a scoring plan was not used in this example. The learner has no indication of what is required to be considered a complete response other than including reasons and examples to support their position.
6. Is the item written in such a way that the scoring plan will be obvious to knowledgeable learners? No, again, no scoring plan was provided.
While it is possible that the ACT uses a scoring plan and provides it to the various readers grading the essay items, it is not made known to the student at the time of testing. Providing this information may make the learner more aware of what is expected of them and allow them to focus on the specific content.
Oosterhof, A., Conrad, R., & Ely, D.P. (2008). Assessing Learners Online. Upper Saddle River, NJ: Pearson Education Inc.
No comments:
Post a Comment