Part 7: Concluding statement

Chapter 20: Out of the fog
 

Before I commenced work on this thesis I had already worked on this particular aspect for two years, and had written about ten chapters for a book on the subject. Further work during the past two years at Flinders University has developed and enlarged the scope of the work. As well, I have traversed some side roads, taken some wrong turnings, and come to a few dead ends. For example, at one stage it seemed the whole focus of the work would be on competencies. At another point interviews with assessment experts, administrators, teachers and students loomed large on the agenda. So at various times I was diverted from the main topic but always returned to it, often with fresh insights.

Tying the focus to the concepts of validity and invalidity was a relatively late development, only possible after the literature on validity was reframed as an advocacy for the test taker. The centrality of comparability to the whole assessment issue was similarly a late discovery.

I am personally pleased at the outcome. I can now make some sense out of what seemed non-sense; I have shown how some of the fudging was accomplished, and why it was important, in terms of social stability, to do so. At the same time I have, I believe, forged a powerful tool for the analysis of invalidity of assessment, and hence of error in the categorisation of individual persons--a tool based on a shift in positioning from test giver to test taker.

In a rational world the thirteen sources of invalidity, developed in many cases by reframing and repositioning the accepted scholarship in the field of assessment, should be sufficient to halt the conceptual blindness, the blatant suppression of error, the subtle fudges, and the myth of certainty that permeates the "science" and expertise of categorising people. Full acceptance and individual specification of even one of these sources could revolutionise current practice. However, as the study indicates, the world in which assessment resides is far from that rational world to which much of the writing in this thesis appeals.

I have tried to be clear about some of the forces that work on all of us that will encourage the reader to react strongly and negatively to many of my arguments, to dismiss them as anathema. The work is immoral in that it conceptually threatens the inviolability of standards and their measurement, a lynch pin of the cultural production of the modern individual. And it is revolutionary in that action based on its conclusions would destabilise to a point of destruction many, probably most, educational and work practices that result in the categorisation of people.

On the other hand, the basic contentions of this project are not contentious at the top levels of evaluation in Education, Medicine, or Law: Ph D theses in Education are assessed by different examiners and it is expected that such assessors will often differ in their judgments of quality; when expert opinions are sought in medicine both diagnosis and treatment prescriptions may differ markedly; and the seven judges in the high court often give conflicting verdicts.

The work could be criticised as being unduly negative. Even if the claims of the thesis are true, or partially true, is its position not destructively unhelpful? We need to categorise people, so take away the standard and what remains? How can people live with the certainty of uncertainty? At the very least, give us an alternative. And whilst I have not developed the alternatives, I have certainly presented them. The Responsive frame has many developed modes of assessment within its boundaries. The chapter on quality clearly indicates one way to go. We live in a world of complexity and uncertainty, a fuzzy multi-dimensional world of immense variety and diverse interpretations. What is challenged in this work is the myth that this complexity can be reduced to simple linear dimension by some sort of examination, as a preliminary to comparing with some standard of adequacy somewhere defined.

This thesis does not contend that people cannot be pinpointed along such dimensions, butterflies permanently fixed on the board. It happens to millions every day. What is shown is that such categorisations are inevitably permeated with confusion, uncertainty and error, that genuine rather than fudged estimates of much of this error can be made, and that this particular violation of the human mind and spirit will continue until they are.