Some Abuses of “Science”, Logic, and Authority Illustrated from Research in Education
Abstract
In this paper, problems with the philosophy and research relating to various interpretations of “closing the gap” in educational achievement are used to open up a discussion of, and illustrate, the process whereby a narrow interpretation of “science” and neglect of systemic thinking result in the generation of huge amounts of dangerous and misleading misinformation and thence the generation of draconian and destructive policies. The paper opens by returning to an unfinished debate arising out of a summary of the unanticipated and counterintuitive effects of interventions designed to close the “attainment” gap between more and less advantaged pupils. This is used to illustrate the importance of studying the unintended as well as intended outcomes of interventions and the importance of considering whether those outcomes are desirable. More of the problems facing those who seek to contribute to evidence-based policy are then illustrated, via a discussion of an “illuminative” evaluation of competency-oriented, project-based, education conducted in the environment around a number of schools, to open a discussion of the need for comprehensive evaluation of educational—and other—projects and policies. “Comprehensive evaluation” implies the evaluation of all short and long term, personal and social, desired and desirable, and undesired and undesirable effects of the programmes and policies under investigation. When this criterion is applied to the vast number of published evaluations of school effectiveness it emerges that most fall well short of the mark. Worse than that, most of their conclusions are nothing less than seriously misleading and damaging. The generation of such misleading information is much more widespread and serious than that exposed by the “replication crisis.” It is argued that, in essence, it stems from the pervasive deployment of non-systemic (viz. “reductionist”) science. A range of serious deficits in the thinking and methodology of psychologists and educational researchers associated with this approach are then discussed. It is concluded that it is vital for social scientists to do what they can to rectify the situation.
References
Andersson, B-E. (2001). School is Good for Many, but Bad for Too Many. Voices from Students About Their School Situation. Mimeographed paper. Stockholm: Institute of Education, Department of Child and Youth Studies
Bailey, D. H., Duncan, G. J., Watts, T., Clements, D. H., & Sarama, J. (2018). Risky business: Correlation and causation in longitudinal studies of skill development. American Psychologist, 73, 81–94. http://dx.doi.org/10.1037/amp0000146
Bennett, N. (1976). Teaching Styles and Pupil Progress. London: Open Books.
Butler, P. (2015). Thousands have died after being found fit for work, DWP figures show. The Guardian (August 27). Retrieved June 6, 2015 from http://www.theguardian.com/society/2015/aug/27/thousands‐died‐after‐fit‐forwork‐assessment‐dwp‐figures
Campbell, D. T. (1979). Assessing the impact of planned social change. Evaluation and Program Planning, 2(1), 67–90.
Ceci, S. J. & Papierno, P. B (2005). The rhetoric and reality of gap closing: When the “have-nots” gain but the “haves” gain even more. American Psychologist, 60, 149–160.
Clark, M. M. (2015). The effect of the phonics check on young children’s ideas about written language. Why should we be concerned. The Psychology of Education Review, 39 (1, Spring ).
Elliott, J. G., & Grigorenko, E. L. (2014). The Dyslexia Debate. Cambridge, UK: Cambridge University Press.
Fink, B. (2016). How Neoliberalism got organized: A Usable history for resisters, With Special reference to education. The Good Society, 25, 158–171.
Flynn, J. R. (2000). How to Defend Humane Ideals. Nebraska: University of Nebraska Press.
Forrester, J. W. (1971/1995). Counterintuitive Behavior of Social System: An introduction to the concepts of system dynamics, discussing social policies and their derivation from incomplete understanding of complex systems. Original text appeared in the January, 1971, issue of the Technology Review published by the Alumni Association of the Massachusetts Institute of Technology. All figures are taken from World Dynamics by Jay W. Forrester, Pegasus Communications, Waltham MA. http://static.clexchange.org/ftp/documents/roadmaps/RM1/D-4468-2.pdf
Goodlad, J. (1983). A Place Called School. New York: McGraw Hill.
Graeber, D. (2018). Bullshit Jobs: A Theory. London: Allen Lane/Penguin Books.
Hamilton, D., Jenkins, D., King, C., MacDonald, B., & Parlett, M. (Eds.). (1977). Beyond the Numbers Game. London: MacMillan Education.
Harris, J. R. (2006). No Two Alike: Human Nature and Human Individuality. New York: W.W. Norton.
Hattie, J. A. C. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. London: Routledge, Taylor & Francis.
Hope, K. (1984). As Others See Us: Schooling and Social Mobility in Scotland and the United States. New York: Cambridge University Press.
Kanter, R. M. (1985). The Change Masters: Corporate Entrepreneurs at Work. Hemel Hempstead: Unwin Paperbacks.
Kapur, N. (2018). NHS disciplinary processes. The Psychologist, October, 31, 4.
Kazdin, A. E. (2006). Arbitrary metrics: Implications for identifying evidencebased treatments. American Psychologist, 61, 42–49.
Klein, N. (2007). The Shock Doctrine: The Rise of Disaster Capitalism. London: Penguin Books.
Klemp, G. O., Munger, M. T., & Spencer, L. M. (1977). An Analysis of Leadership and Management Competencies of Commissioned and Non-Commissioned Naval Officers in the Pacific and Atlantic Fleets. Boston: McBer.
Lees, S. (1996). Strategic Human Resource Management in Transition Economies. Proceedings of Conference: Human Resource Management: Strategy and Practice. Alma Atat Management School, Alma Atat, Khazaksthan.
Lester, S. (2001). Assessing the self-managing learner: A contradiction in terms? Chapter 26 in J. Raven & J. Stephenson (Eds.), Competence in the Learning Society. New York: Peter Lang. http://eyeonsociety.co.uk/resources/CILSchapter-26.pdf
Maxwell, J. N. (1969). Sixteen Years On. Edinburgh: Scottish Council for Research in Education.
McKnight, J. (1995). The Careless Society: Community and Its Counterfeits. New York: Basic Books (Perseus Group).
Milgram, S. (1974). Obedience to Authority. London: Tavistock.
Mill, J. S. (1859/1962). Representative Government. London: Dent.
Mottus, R., & Rozgonjuk, D. (2019). Development is in the details: Age differences in the big five domains, facets, and nuances. Journal of Personality and Social Psychology. Advance online publication. http://dx.doi.org/10.1037/pspp0000276.
Mulder, M. (Ed.). (2017). Competence-based Vocational and Professional Education: Bridging the Worlds of Work and Education. Basel, Switzerland: Springer International. http://www.springer.com/us/book/9783319417110
Nutt, D. (2012) .Drugs Without the Hot Air: Minimising the Harms of Legal and Illegal Drugs. Cambridge, UK. UIT Cambridge.
Ofsted (2017). Bold beginnings: The reception curriculum in a sample of good and outstanding primary schools. Piccadilly Gate, Store Street, Manchester M12WD. https://schoolsweek.co.uk/wp-content/uploads/2017/11/28933-Ofsted-Early-Years-Curriculum-Report.pdf
O’Reilly, D. (2001). Competence and incompetence in an institutional context. Chapter 22 in J. Raven & J. Stephenson (Eds.), Competence in the Learning Society. New York: Peter Lang.
O’Reilly, D., Cunningham, L., & Lester, S. (Eds.). (1999). Developing the Capable Practitioner. London: Kogan Page.
Papierno, P. B. and Ceci, S. J. (2005). Beyond the American context. American Psychologist, 60, 1042 –1043.
Prieler, J. A., & Raven, J. (2008). Problems in the measurement of change (with particular reference to individual change [gain] scores) and their potential solution using IRT. In J. Raven & J. Raven (Eds.) Uses and Abuses of Intelligence: Studies Advancing Spearman and Raven’s Quest for Non-Arbitrary Metrics. Unionville, New York: Royal Fireworks Press; Edinburgh, Scotland: Competency Motivation Project; Budapest, Hungary: EDGE 2000; Cluj Napoca, Romania: Romanian Psychological Testing Services SRL. (Chapter 7, pp. 173–210). Also available at http://eyeonsociety.co.uk/resources/UAIChapter7.pdf
Raven, J. (1980a). Parents, Teachers and Children: An Evaluation of an Educational Home Visiting Programme. Edinburgh: Scottish Council for Research in Education.
Raven, J. (1980b). Teetering on the brink of a totalitarian society? New Universities Quarterly, 34, 370–382. http://eyeonsociety.co.uk/resources/teeteringonthebrinkofatotalatariansociety.pdf
Raven, J. (1984). Some limitations of the standards. Evaluation and Program Planning, 7, 363–370.
Raven, J. (1991). The Tragic Illusion: Educational Testing. New York: Trillium Press. www.rfwp.com (now available from the author at 30, Great King Street, Edinburgh EH3 6QH.)
Raven, J. (1993). Managing society: The past belonged to economists: The future belongs to psychologists. In A. Biela, & L. Pawlowski, Protection of the Environment: Mental Changes and Social Integration Perspectives (pp. 531–563). Lublin, Poland: Central European Center for Behavioral Economics, Technical University of Lublin. Also published in: Journal for Mental Changes, (1995), 1–2, 7–46.
Raven, J. (1994). Managing Education for Effective Schooling: The Most Important Problem Is to Come to Terms with Values. Unionville, New York: Trillium Press. http://eyeonsociety.co.uk/resources/fulllist.html#managing_education
Raven, J. (1995). The New Wealth of Nations: A New Enquiry into the Nature and Origins of the Wealth of Nations and the Societal Learning Arrangements Needed for a Sustainable Society. Unionville, New York: Royal Fireworks Press Edinburgh, Scotland: Competency Motivation Project. http://eyeonsociety.co.uk/resources/fulllist.html#new_wealth
Raven, J. (1997a). Can we discuss policy if nothing is what it seems to be? Journal for Mental Changes, 3(1), 85–103. http://eyeonsociety.co.uk/resources/hcwdpall.pdf
Raven, J. (1984/1997b). Competence in Modern Society: Its Identification, Development and Release. Unionville, New York: Royal Fireworks Press. (First published in 1984 in London, England, by H. K. Lewis.)
Raven, J. (2001). Facilitating the development of competence (Chapter 18). In J. Raven & J. Stephenson (Eds.), Competence in the Learning Society. New York: Peter Lang.
Raven, J. (2005). More problems with Gap Closing Philosophy and Research. American Psychologist, 60, 1041–1042. Also available at: http://eyeonsociety.co.uk/resources/MPWCTGP.pdf
Raven, J. (2006a). Comment - Papierno and Ceci Miss the Point: A response to Papierno and Ceci’s response to Raven (2005) More Problems with Gap Closing Philosophy and Research. http://eyeonsociety.co.uk/resources/cpmtp.pdf
Raven, J. (2006b). Undesirable Human Traits? http://eyeonsociety.co.uk/resources/uhts.pdf
Raven, J. (2008). The emergence of hierarchy, domination and centralisation:Reflections on the work of Murray Bookchin. Journal for Perspectives of Economic, Political, and Social Integration, 14(1–2), 11–75. Also available at: http://www.eyeonsociety.co.uk/resources/Bookchin.pdf
Raven. J. (2011). What does it mean to be professional? The Psychologist, 24(8), 560.
Raven, J. (2012a). Competence, education, professional development, psychology, and socio-cybernetics. In G J. Neimeyer (Ed.), Continuing Education: Types, Roles, and Societal Impacts (Chapter 16). Hauppauge, New York: Nova Science Publishers, Inc. http://www.eyeonsociety.co.uk/resources/cpdapa_revised_full_version.pdf
Raven, J. (2012b). Emergence: A paper prepared for the Focolari Conference on Prospects for Research and Treatment in Psychology. Journal for Perspectives of Economic Political and Social Integration: Journal for Mental Changes, 19(1–2), 91–109. Also available at http://eyeonsociety.co.uk/resources/focolari.pdf and http://eyeonsociety.co.uk/resources/focolari_addendum.pdf
Raven, J. (2014). Our incompetent society (with a discussion of some of the competencies needed to transform it). http://eyeonsociety.co.uk/resources/Incompetent-society-v3.pdf Russian translation by Oleg Yarygin: Part I: Baltic Humanitarian Journal (2016) T.5. 4(17) 274–292. http://www.napravo.ru/pages/nauchnye_jurnaly/baltiiskii_gumanitarnyi_jurnal/nauchnye_jurnaly_/ Part II: Azimuth Of Scientific Researches / Pedagogy and Psychology, (2016) , 4(17), 198-205. http://www.napravo.ru/pages/nauchnye_jurnaly/jurnal_vektor_nauki/nomera_jurnala_2013/
Raven, J. (2016). The Pervasive and Pernicious Effects of Neglecting Systems Thinking (especially when combined with a disposition toward fascism). http://eyeonsociety.co.uk/resources/Unwillingness-to-engage-in-systemsthinking.pdf
Raven, J. (2017). Problems with Closing the Gap Philosophy and Research, Baltic Humanitarian Journal, 3, 252-275. Full version at: http://eyeonsociety.co.uk/resources/Closing-the-Gap-2017-V2.pdf
Raven, J. (2018a). Harnessing social processes for the common good. Journal for Perspectives of Economic, Political, and Social Integration, 24, 9–49. Also available at http://eyeonsociety.co.uk/resources/Harnessing-Social-Processes.pdf
Raven, J. (2018b). How to deal with atrocities invented and perpetrated by public servants? http://eyeonsociety.co.uk/resources/How-to-deal-with-atrocitiesperpetrated-by-Public-Servants-2.pdf
Raven, J. (2018c). Some criminal (if not yet criminalised) misapplications of “science,” logic, and power illustrated from the field of early childhood education. http://eyeonsociety.co.uk/resources/Criminal-misapplications-of-science.pdf
Raven, J. (2020). Recent Research Supporting a Specific-motive Based Model of Competence. Contribution to a symposium held in Prague, January 2020. http://eyeonsociety.co.uk/resources/Raven-Mottus-Prague-1-21-20.pdf
Raven, J., & Stephenson, J. (Eds.). (2001). Competence in the Learning Society. New York: Peter Lang. Many chapters also available at http://eyeonsociety.co.uk/resources/fulllist.html#competence_in_the_learning_society
Raven, J., Johnstone, J., & Varley, T. (1985). Opening the Primary Classroom. Edinburgh: Scottish Council for Research in Education. http://eyeonsociety.co.uk/resources/OTPC-complete.pdf
Revans, R. W. (1977). Action learning and the nature of knowledge. Education and Training, November/December.
Roberts, R. (2018a). Hopeful possibilities of resistance. The Psychologist, 31 (September, 7). https://thepsychologist.bps.org.uk/volume-31/september-2018/hopeful-possibilities-resistance
Roberts, R. (2018b) Social influence and malevolent authority: Obedience revisited. In Robinson, K., & Aronica, L. Creative Schools (Chapter 14). London, UK. Penguin Books.
Rothschild, Lord. (1982). An Enquiry into the Social Science Research Council. London: HMSO.
Sahlberg, P. (2015). Finnish Lessons: What can the world learn from educational change in Finland? New York, USA: Teachers College Press.
Schmidt, F. L., Oh, I.S. & Shaffer, J.A. (2016). The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 100 Years of Research Findings. Unpublished, available from the authors.
Schon, D. (2001). The crisis of professional knowledge and the pursuit of an epistemology of practice (Chapter 13). In J. Raven & J. Stephenson (Eds.), Competence in the Learning Society. New York: Peter Lang. Also available at http://eyeonsociety.co.uk/resources/CILS-chapter-13.pdf
Scottish Government. (2014). Children and Young People (Scotland) Act, 2014. Norwich, England: TSO (The Stationery Office). http://www.legislation.gov.uk/asp/2014/8/pdfs/asp_20140008_en.pdf
Seddon, J. (2008). Systems Thinking in the Public Sector: The Failure of the Reform Regime … and a Manifesto for a Better Way. Axminster, UK: Triarchy Press.
Shiva, V. (1998). Biopiracy: The Plunder of Nature and Knowledge. London: Green Books.
Spearman, C. (1927). The Abilities of Man: Their Nature and Measurement. London, England: MacMillan.
Spencer, L. M., & Spencer, S. M. (1993). Competence at Work. New York: Wiley.
Suggate, S. P. (2012). Watering the garden before a rainstorm: The case of early reading instruction. In S. Suggate, & E. Reese, Contemporary Debates in Childhood Education and Development (Chapter 17). London, England: Routledge.
Webster, D. (2014). Inquiry into benefit sanctions policy beyond the Oakley review. House of Commons Work and Pensions Committee. http://www.cpag.org.uk/sites/default/files/uploads/CPAG-HofC-Wk-Pens-Sanctions-DWevidence-Dec-2014.pdf
Weerts, D. J. (2016). From covenant to contract: Changing conceptions of public research universities in american society. The Good Society, 25, 182–208.
Wilkinson, L., & Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594–604.
Winter, D. G., McClelland, D. C., & Stewart, A. J. (1981). A New Case for the Liberal Arts. San Francisco, CA: Jossey-Bass.
Wolf, A. (1987). Work Based Learning: Trainee Assessment by Supervisors. Bradford, England: MSC Sales, ISCO.
Copyright (c) 2020 Journal for Perspectives of Economic Political and Social Integration
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
