Examinations and Examiners
Walsh, James J.
EXAMINATIONS AND EXAMINERS By TAMES T. WALSH WE ARE all, at least recurrently, interested in examinations, their methods and results. And since they are upon us again, it might not be amiss for...
...No wonder that the investigators declared that examiners differ as much in one subject as in another: "They disagree as much in evaluating a paper in mathematics as in history...
...Four actual papers that had been handed in by pupils in the ordinary course of examination, and that represented about the average standing of the class, were sent to a group of teachers who had given grades in those subjects during the year...
...As the result of his failure, he never received a license to become a teacher, remained a substitute all his life and received something less than a dollar a day for his services...
...The custom of lettering examination papers from A to D, which represent a spread of thirty to forty points, is a much more satisfactory way of marking and much nearer the truth of comparative value, than the use of figures that would presumably make much finer distinctions...
...Five were marked 64, a tribute perhaps to that number's rounded quality of factorage...
...Geometry has been practically unchanged since Euclid's time...
...He was one of these human beings who know too well what they do not know...
...Manifestly the mathematicians felt that they could make nicer distinctions, so there was not the crowding so commonly noted in other examinations around cardinal points...
...Alas for human fallibility...
...The lowest mark given was 28 and the highest was 92...
...So far as their revelation of the talent, much less the possible genius, of those who are examined is concerned, they are quite hopeless...
...For some special mathematical reason that is perhaps a little difficult for the unmathematical mind to grasp, five papers were marked 53, and a dozen between 59 and 61...
...The geometry paper had a wider distribution of marks than any other of the examination papers...
...There can be no question of the facts of American history so far as the teachers' minds are concerned, and yet there was this striking difference in their evaluation of the responses...
...It may be thought that high school English is a somewhat indeterminate subject, and therefore a considerable difference in the marks given for that will be expected...
...It might be expected, however, that in a subject like American history where every detail is settled and definite, there would not be anything like the same variation in the marks given...
...We hear of men giving notes of 90 or 91 or sometimes of 98 or 99, and feel sure that here we have an example of discrimination of the acutest kind, only to be exemplified by those long accustomed to the examination of papers...
...And not only was the spread from first to last greatest but the marks were distributed more consistently over all the figures in between...
...We have had similar selections made for the Rhodes scholarships for some twenty years now, and so far no Rhodes scholar has set the world on fire...
...Most of these were bunched between 75 and 80...
...There have been other investigations which have made it quite clear that the marks depend very greatly on the examiners...
...He was not a good bluffer, and it takes some bluff to pass an examination with high marks...
...As a rule, unless papers are marked by the teacher who taught the subject and who knows what he has most emphasized and required during the course, there will be a very great diversity in the marks...
...They were asked to mark according to their usual standards...
...One of the June 25, 1930 THE COMMONWEAL 217 English papers was marked from 64 to 98...
...It is actually the examiner more than the examinee who is on trial...
...The lowest mark was 42, the highest was 90...
...The other English paper marked by the same English teachers had an even larger spread, namely from 50 to 98...
...Pupils, for instance, who have done excellent work and deservedly received high marks for years, may receive scarcely more than a passing mark...
...Their mark is not a question of mere chance...
...And since they are upon us again, it might not be amiss for the general public to be brought to the realization that examination ratings depend very largely on the persons who do the rating...
...His examination papers are still on file, and they show that he deserved to fail...
...Some teachers mark constantly lower than the average, others constantly higher...
...As a matter of fact, however, the longer men have been giving examination marks, the less likelihood there is of their presuming to make any very delicate distinction between papers...
...Later he tried again and failed again...
...The great majority of the marks were between 70 and 80, but there were nearly a dozen around 50 and another dozen around 60, and more than a dozen around 80 and 90...
...No wonder that teachers are not inclined to set very much store by examinations...
...The Edison examination of last year, for example, may or may not serve to reveal that certain individuals among the group of young men examined are of outstanding intellectual development...
...Apparently mathematical papers are not marked with mathematical precision any more than any other papers...
...On the other hand, as a little tribute to the weakness of even the mathematical mind, a dozen of papers were marked exactly 75...
...When some of these papers were resubmitted to the same teachers several years later for remarking, they did not differ from their previous ratings by more than five points...
...It was a good thing to pick not one but half a dozen candidates for the Edison scholarships...
...To single out an individual under such circumstances would be at least temerarious, if not quite unfair...
...But he had the genius to look right through the deep problem of heredity and solve it...
...The best illustration of that is that the Abbot Mendel, to whom we owe the revolution of our knowledge of heredity, and who must be considered one of the greatest geniuses in biology of the past hundred years, tried as a young man to pass an examination for teacher and failed...
...Young teachers will elaborately assess differences in marks of one or two points, but the man of long experience knows that it is as much as he can do to judge of differential values that correspond to six or seven...
...They may reveal the intellectual status of the individual student to some extent, but they are much more likely to reveal the intellectual interests of the examiner...
...Regents' examination marks in New York state are given at Albany...
...On the other hand, a few pupils every year who have been neglectful and slipshod in their studies, and whose papers have always revealed these qualities, will receive very high marks...
...Two of the papers were in first-year high-school English and were graded by 142 teachers of this subject in as many high schools...
...Besides, the examiners were themselves mathematicians and their mathematical minds would be expected to guide them in making absolute and not merely relative estimates of value...
...One final examination paper in American history was graded by seventy teachers of history, while the final examination paper in geometry was graded by 118 teachers of mathematics...
...Perhaps the most surprising thing in the development of pedagogy as a science has been the recognition of the fact that the same set of answers to a series of questions may have very different values in the minds of men accustomed to teach that subject...
...As I have said, there may be some excuse for these differences in the comparative indefiniteness of the subject...
...It is a good deal of a shock, however, to find how extreme the difference was...
...Ask teachers about them—and by that I mean teachers who have been many years teaching particular subjects—and they will tell you what surprising results the Regents' can produce...
...There is likely to be a very great difference of opinion among examiners with regard to the value to be placed on answers...
...Finally there is the examination paper in geometry marked by 114 teachers of mathematics...
...There were only about a dozen papers below 80, but there were a dozen above 95, and there were more than a few in the 6o's and 70's...
...Yet the seventy history teachers who marked the papers submitted to them differed at least as much as, perhaps a little more than, the teachers of English...
...Some years ago Starch and Elliot distributed certain examination papers which had been given in the finals of first year high school in Wisconsin...
...The choice between men who are very close together in marks is too illusory to be made with any assurance...
...Whatever influenced their estimate represented a constant factor in their judgment...
...What becomes very clear, however, as the result of such investigation is that examination marks do not mean much unless you know the person who gives them...
Vol. 12 • June 1930 • No. 8
