Ronald GallimoreEveryone's a teacher to someone (John Wooden)

More on Education Reform and Campbell's Law

Yesterday I posted a bit on Campbell’s Law. To repeat:
"The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor." Wikipedia entry

My evaluator pals relate numerous seemingly innocent examples that might represent the Law at work. A generic (and hypothetical) example: imagine well intentioned reformers develop a program to increase the number of disadvantaged students applying to and matriculating at four year colleges. After one year, the program operators (and the funding source) were disappointed by the low rate of success. Counselors and teachers who worked with the students suggested re-examining admission criteria. They observed that many students admitted during the first year of operation were on paper academically qualified to enter four year colleges, but exhibited only superficial interest in higher education. Admission criteria the second year were refined to include interest level in college as well as include academic readiness–think of it as learning from experience. Outcomes for the second year of operation were dramatically better.

Did the adjustment of admission criteria represent a corruption of standards? If the original goals included increasing the number of students interested in attending college, unless the goals were also adjusted, then the change in admission process might be Campbell’s Law at work. If in the second year, the goals were explicitly revised to focus only on application and matriculation, then perhaps changing admission criteria was not as clear an example of corrupted standards. In practice, busy program personnel are so caught up in the day to day demands of their work that goals sometimes implicit shift without awareness. So perhaps there was in this instance a “slippery slope” kind of corruption in which admissions criteria were changed, but the operators never explicitly addressed the goals issue–and continued to perceive and describe the program as unchanged. But the Law makes no distinction between unintended and intended forms of standards corruption.

A second hypothetical example: a charter school operation within a public district was under close scrutiny by many community interests. For each of the first five years operation, standardized achievement test results in the charter schools were significantly higher than in non-charter schools in the district. After several years, it was discovered the charter operators gradually changed policies, by reducing the number of student in their program with learning disabilities and a history of behavior problems. Charter operators argued they were only trying to maximize the number of students benefiting from an academically challenging instructional program. Critics argue they had stacked the deck and bled desperately needed funds away from the rest of the schools in the district, who could not “screen out” students so easily; and that the charters school were duplicitous in claiming their program accounted for the superior achievement gains. The charter operators said they did not stack the deck deliberately, that they tried very carefully to screen out only a limited number of students ill suited to their program. A professional evaluator was consulted; she suggested that a careful review of the selection process might be yield useful information provided the charter operators had kept detailed records on each case screened or counseled out. The dispute continues.

Then there is out-right fraud. Fraudulently changing achievement scores to protect a program under scrutiny has happened, and likely will happen again. Making up and changing achievement scores to protect a program under scrutiny has happened, and likely will happen again. Fraud cases might be easier to recognize as examples of Campbell’s Law.

But it is more difficult to decide when the issue involves the sometimes complex and difficult job of selecting participants in or out of a high value program. How can we decide whether it is corruption or not when operators seek to improve the functioning of a program by selecting in those mostly likely to benefit?

Comments

Campbell's Law of Criterion Corruption

Campbell's law:
"The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor." Wikipedia entry

Recently, in an Education Week blog, Walt Gardner observed that Campbell’s Law struck the use of standardized achievement tests for high-stakes education decisions. He notes several high profile scandals in school testing programs. Examples of cheating by districts and schools he dates back to at least 1969 in Texas, in addition to more recent cases in Georgia, Indiana, Nevada, Virginia, Massachusetts, South Carolina, and California.

Education is not the only field in which Campbell’s Law applies, according to a comment in 2007 by Nichols and Berliner in the Harvard Education Letter. They claim there are in other fields numerous examples of “corruption, cheating, gaming the system, taking short cuts, and so forth whenever high stakes are attached to performance in athletics, academia, politics, government agencies, and the military.”

The nature and extent to which performance criteria inevitably lead to corruption is a matter of debate, and a tough problem to investigate. But there’s every reason Campbell’s Law operates to some extent in any field, and if nothing else is a warning to be humble about claims that setting quantitative criteria to meet is a easy way to improve performance.

If we don’t use standardized tests to assess student learning, what do we use? It is wise to remember standardized tests were once hailed as a democratic alternative to privilege and patronage that determined access to valued assets such as college entrance and employment opportunities. So what now?
Comments

More visitors to this site than I guessed.

This morning for the first time I checked Google Analytics to see if anyone was visiting this website. To my surprise, over the past few weeks there’s been 64 visits, by 38 different visitors. I didn’t realize I had so many friends and relatives.
Comments

Tharp reviews Life Dreams

Roland Tharp published a review of Price-Williams’ book Life Dreams in Perspective, the monthly newsletter of the Association for Humanistic Psychology (ahpweb.org).
To read the review, click here
Comments

Why more accountability might not improve achievement as much as hoped

A RAND study of a few years ago reported that a majority of teachers said new accountability pressures, such as measuring their effectiveness using standardized test scores, had increased their efforts to innovate instruction. However, 90% of the variance was in some districts was WITHIN school, meaning that accountability pressure was not leading to convergence on effective practices. Rather it was scattering teachers in all directions looking for fixes. This is the opposite of what the accountability reformers hoped for. They are operating from the simple feedback model: show teachers a standard to be met, provide feedback, and sit back and watch "market forces" do their magic. One reason of course there is so little convergence is the teacher isolation described in the article recently sent me and to which I responded in a previous posting. Here's another nice comment on this (see attached), and how unwittingly policy might shift focus from instruction.

Alexander Russo added an interesting post to his blog on April 17th summarizing a recent Brookings publication that critiqued the “accountability reform” as a “theoretical bank shot.”

“The contemporary data-driven "reform" movement, fundamentally, is a theoretical bank shot, where in the name of "output-based" accountability non-educators' change the subject away from teaching and learning in order to somehow improve teaching and learning. "Choosing Blindly," by the Brookings Foundation's Grover Whitehurst and Matthew Chingos, is a reminder that the best way to improve classroom outcomes is to concentrate on the real interactions in the classroom and not some statistical models. The better approach, all along, would have been to target the interactions between flesh and blood students, teachers, and the learning materials that they actually use. Whitehurst and Chingos write, "students learn principally through interactions with people (teachers and peers) and instructional materials (textbooks, workbooks, instructional software, web-based content, homework, projects, quizzes, and tests). But education policymakers focus primarily on factors removed from those interactions, such as academic standards, teacher evaluation systems, and school accountability policies." They then nail the essence of the contemporary accountability movement, "It’s as if the medical profession worried about the administration of hospitals and patient insurance but paid no attention to the treatments that doctors give their patients."

A caveat. Choosing Blindly focuses on the lack of information available on instructional materials used in the U.S. The authors argue that most teachers organize lessons and provide instruction that is influenced by instructional materials available and guidelines for the use of the materials provided by local education agencies. Others influences are acknowledged but not addressed. For example, teaching practices such as use of formative assessment, emphasis on conceptual connections as well as procedures, pedagogical content knowledge, etc.


Comments

Counting Horses' Teeth and Education R & D

There's an old story in science that might be one of the first urban legends.
In the Middle Ages, scholars met to settle questions unanswered in sacred and secular texts. One was the number of teeth in the mouth of a horse. After several days of prayer and re-examining ancient sources, the scholars were perplexed. In response to the elder’s distress, a young student volunteered to visit a local stable and count the teeth in a horse’s mouth. He was beaten with sticks, banished from the assembly, and cursed as an apostate. Somberly, the scholars concluded the answer would remain forever known only to God.

In education R&D, there comes a time to venture out, try ideas to see how they work. It’s safer to stay in air conditioned rooms debating theories and dreaming up interventions. But once all theory and evidence are thoroughly chewed, take to the field, learn from experience, count some teeth. Don’t be afraid to fail. You probably will, and might learn more from failure than more hours of debate and conjecture. Some grizzled veterans of the R&D wars might tell you this: out in the field with real people in actual settings, you often abandon what you thought would work in favor of discovering something that will.

Comments

Physics envy in social sciences

Economists, political scientists and sociologists have long suffered from an academic inferiority complex: physics envy. In a recent NY Times op-ed, Kevin Clarke and Primo follow this witty allusion to a Freudian concept with a more serious point: [Social scientists] often feel that their disciplines should be on a par with the “real” sciences and self-consciously model their work on them, using language (“theory,” “experiment,” “law”) evocative of physics and chemistry.


Another discipline recovering from physics envy is psychology which, to be fair, exhibits less symptoms today than 50 years ago. One reason being a diversification of psychology into many–some very large–sub-disciplines such as clinical, developmental, social, cognitive, evolutionary, genetic, biological. Many psychologists no longer belong to the same scientific organizations or share the same space on university campuses. There’s a second reason physics envy has diminished in psychology. Admiration of the cool detachment of physical sciences has been, is being, supplanted by more orientation to the biological sciences’ focus on adaptation over time of living organisms. For many psychologists this is a more agreeable frame given the dynamic nature of human cognition, emotion, and behavior. As the 20th century wound down, many psychologists diversified the research methods considered suited to study of humans.


For most of the 20th century, social, behavioral, and developmental science was governed by a Newtonian metaphor. It was a social physics..... patterned on that of celestial mechanics and the dynamics of nonliving material particles. The resultant image is of a mechanical universe governed by absolute and unalterable laws, made of indivisible and identical units, of a finite number of specifiable types, majestically floating along predetermined pathways in a limitless void. …[The Newtonian metaphor mentally prepared psychologists]....."for a bold exploration of the icy depths of interplanetary space. Instead, they found themselves completely unprepared for the tropical nightmare of a Darwinian jungle: A steaming green Hell, where everything is alive and keenly aware of you, most things are venomous or poisonous or otherwise dangerous, and nothing waits passively to be acted upon by an external force..... The Darwinian jungle manipulates and deceives the unwary wanderer into serving myriads of contrary and conflicting ends. The sweltering space suits just had to come off (Sechrest & Figuerdo, 1992, p. 647-48).

http://www.nytimes.com/2012/04/01/opinion/sunday/the-social-sciences-physics-envy.html?_r=1
Sechrest, L. & Figuereo, A.J. (1993). Program evaluation. Annual Review of Psychology, 44, 645-74.

Comments

John Wooden Taught Concepts Too

Many stories about Coach John Wooden emphasize his use of repetition as a key teaching approach. It’s true that practicing until certain actions were automatic was part of his approach. But it is not true that this was the foundation of his approach. He focused on development of players’ conceptual understanding that in combination with lots of practice led to a level of performance seldom equal in athletics.

A key goal of Wooden was the development of players who were creative, confident problem-solvers. As games progress, teams change tactics, presenting new problems by to force the opponent to play in a way it might not want to. In response to changing tactics by opponents, Wooden wanted …to be as surprised as our opponent at what my team came up with when confronted with an unexpected challenge (Nater & Gallimore, 2010, pps. 89-90).

Wooden’s goal was to teach the underlying concepts of basketball, so that when opponents surprised his players with new and different challenges they in turn surprised their coach and the other team with creative and effective solution methods.

To develop his players' capacities to “surprise” him with their solution methods during games, Wooden used a systematic pedagogical approach that he describes as the “whole-part” method.

I tried to teach according to the whole-part method. I would show them the whole thing to begin with. Then I’m going to break it down into the parts and work on the individual parts and then eventually bring them together.

(Wooden, personal communication, February 12, 2002).

There’s an interesting parallel between Coach Wooden’s pedagogy and contemporary views on teaching mathematics. It is not a perfect analogy because the subject matters are different in fundamental ways. Consider Coach Wooden’s hope “ …to be as surprised as our opponent at what my team came up with when confronted with an unexpected challenge.” The desire to be “surprised” by his players is surprisingly analogous to contemporary ideas on teaching mathematics. If students are only taught to memorize solution methods, any deviation in problem structure or form may stymie them. If they were taught to understand conceptually the underlying mathematics, they are typically better prepared to devise solution methods as the need arises.

Learning the “basics” is important; however, students who memorize facts or procedures without understanding often are not sure when or how to use what they know. In contrast, conceptual understanding enables students to deal with novel problems and settings. They can solve problems that they have not encountered before.

(National Council of Teachers of Mathematics, 2005) ).

Coach Wooden emphasized repetition of fundamentals so that his players would be resourceful, imaginative, and creative, not because he wanted them to be robots mindlessly relying on rote memory. For him, repetition is a means to an end; he firmly believed that when students understand what they are doing and can connect the ideas they are taught, they are better prepared to solve new problems as they arise in the future. He teaches that understanding and conceptual knowledge, supported by automatic mastery of fundamentals, prepares students to tackle problems of all kinds, like those they had encountered before, and novel ones, too.

For more how Coach Wooden taught concepts and used repetition to build automaticity, see Nater and Gallimore (2010), Chapter 6.

Comments

Studying Coach Wooden's Teaching

During the 1974-1975 basketball season, Roland Tharp and I spent many afternoons observing Coach John Wooden as he conducted practice on the UCLA campus. Using an observational coding scheme tailored to Coach’s teaching practices, we analyzed and published the results of our observations in Psychology Today (Tharp & Gallimore, 1976).
rg:jw:rt


In 2004, The Sports Psychologist published a followup to the original study including new materials and interpretation of 1970s investigation (Gallimore & Tharp, 2004). PDF

Swen Nater and I summarized the results of 30 years of investigating Coach Wooden’s teaching principles and practices in You Haven’t Taught Until They Have Learned (Nater & Gallimore, 2005). Some of his key approaches are summarized on this page. Read More.
swen, coach, & ron cropped 7.03
Comments

John Wooden's Timeless Lessons

Pasted Graphic


In 2003, Hank Bias called me asking about films of Coach Wooden teaching on the practice court. Hank was and still is head basketball coach. He coaches the Fairmont High’s Fairmont Firebirds of Kettering, Ohio. When he called, the team had lost 17 games, winning only 3, and Hank was thinking of leaving coaching. He called because he’d read an article Roland Tharp and I wrote about John Wooden’s teaching practices. Hank wanted films so he could learn to be a better teacher of basketball. I had made digital copies of old films Coach Wooden gave me, but I told Hank he’d have to call Coach Wooden to get permission to get copies. After some hesitation, Hank phoned Wooden who invited him out to California to talk about teaching. That began a story that some describe as “too Hollywood” to be true. But it is true, and what happened in the following years is quite a story. For ten years, beginning in 2003, I tracked Hank’s progress, visiting him twice, interviewing him in person and on the phone, and keeping up an email correspondence. Our research team recently published the first of several articles on Hank’s story. This first article describes how Hank learned to apply Wooden’s approach to steadily, continually improving his teaching, and the effects it had on him and his team. Read more about Hank and Coach Wooden: click here.
Comments

Coach John Wooden talks about teaching

John Robert Wooden (1910 – 2010) was Head Basketball Coach at UCLA from 1948 until his retirement in 1975. Wooden’s UCLA teams earned 10 NCAA titles in 12 years, reeled off an 88-game win streak, and won 38 consecutive tournament games.
NCAA coach of the year 6 times, in 2000 he was named Men’s College Coach of the 20th Century by the Naismith Hall of Fame and ESPN. In 2003 Coach was awarded the Presidential Medal of Freedom.

When he spoke to groups, Coach Wooden often began remarks this way: “When I was teaching at UCLA…..” He believed that coaching is teaching, and that the foundation of his success as a coach was the years he spent as a high school English teacher. The Academy of Achievement site includes a video of Coach Wooden in which he describes himself as a teacher.
Comments