education philosopher

“Research Based” Rhetoric and “Research Based” Reality

Posted in Teaching by KevinCK on July 16, 2009

Once, my mentor teacher told me about the “10:2 theory.” It states that for every ten minutes of meaningful learning a child does, she should have 2 minutes of a “brain break.” I was open minded, but she could see a bit of skepticism in my face, so she assured me that it was a “research based” practice. I asked her if she had any literature on the theory, and she showed me a book article that also assured readers that “10:2” was “research based.”

researcherLater that night, I decided to do a bit of internet research just to see if I could find any studies that had validated the “research based” 10:2 idea. What I found was that a certain article (telling us the practice was “research based”) cited another article (which told us that same) which cited another, which cited another. Each article cited the other as proof of the 10:2 theory, but none actually had any proof of the 10:2 theory.

My point: “research based” is practically an empty term in the field of education. Obviously, teachers are in no position to find out what research actually says, and (as the example should illustrate) most of those who tell us what the “best practices” are seem also to be quite gullible when it comes to figuring out what the research says. (I think my mentor teacher, a very smart woman, just heard enough people say that 10:2 was research based that she had no reason to check for herself, and they probably heard it from their friends, and so on.)

Here is another example of such an unfortunate trend from psychologist Daniel Willingham’s review of Mel Levine’s wildly popular (in education circles) book One Mind at a Time:

How did Levine come to his particular theory of the mind?

Since A Mind at a Time contains few references to the scientific literature, I telephoned All Kinds of Minds and asked the associate director of research if there was a more research-oriented publication that I might read. She directed me to the web site of Schools Attuned, the teacher training program Levine established to promote his prescriptions for handling learning-disabled students, which lists the “research base” for the program. This research base consists of eight works, all by Levine and coauthors, none of which appeared in a peer-reviewed journal.

A review of these works reveals that they do not marshal research evidence to support their conclusions. Instead, they present the same ideas contained in A Mind at a Time, citing a few references that support well-accepted ideas—for example, that attention capacity is limited—but none to shore up Levine’s particular views.

So what is a good teacher to do? Obviously we all want to do what works, and following the “research based” findings seems the best way to do that. But, as we can see, we can’t always trust what is recommended to us as “research based.” My advice, for what it’s worth?  Try any suggestion that sounds good to you, and do your own research in the classroom. Use what works there and don’t listen to what the experts say…or what people say the experts say…or what people say that people say the experts say.


4 Responses

Subscribe to comments with RSS.

  1. David said, on July 17, 2009 at 3:54 pm

    Try any suggestion that sounds good to you, and do your own research in the classroom.
    But of course, without the proper experimental controls and such, it’s hard to tell what’s helping in your classroom.

    My dad used to teach college courses, and he did a great job of keeping track of things with statistical methods. One thing he would do is cross-correlate each test question with overall test performance so that he could throw out bad questions (where the students who did well overall didn’t seem to have an advantage).

    It’s easy to do, but almost nobody does it, probably because of a “I’m the expert, I don’t make mistakes” mentality. I wonder how it would affect schools to have a full-time “stats master” run through all the data for each classroom and watch for problem areas, cross-checking the teachers’ intuitions and calling BS when necessary.

  2. KevinCK said, on July 17, 2009 at 8:41 pm


    Actually, the Baltimore County School System had a program that allowed us to do that. Every quiz and test we gave that was BCPS issued – and most were – required that we ‘scan the test’ in to AssessTrax. From there, you could see class or individual printouts of how students did on each question, etc.

    Part of the reason teachers don’t keep stats like this on their own is time constraints. When we need to plan lessons, grade papers and input grades – outside of teaching our classes and doing any other paperwork necessary – it is quite hard to find time to keep stats that thoroughly.

    But it is not always about test results either. Much of the ‘strategy’ we are taught – like 10:2 – has more to do with improving time on task and behavior than about improving whether the students’ academic scores in prove (or at least as much). And that stuff is generally stuff teachers can recognize even without stats. When my goal is to find the strategy that keeps Lamont and Sarah on task and not disrupting the rest of the class, I could keep statistics, but I could (very unscientifically, but more easily) informally observe the results.

    But certainly any strategy reccomended as research-based is supposed to have data behind it, and the reality is that I have seen too many instances where the ‘data’ is no more than the latest ed guru saying that it works (and that he alleges evidence, but didn’t cite it).

  3. Lem Usita said, on March 17, 2010 at 3:54 am

    great explanation. you are so right. theory and research aren’t truth claims. they are theoretical claims. theoretical constructs – at best – tell us what seems to be. it cannot tell us that – these claims are absolutely true.

    read any research and none of the research will make truth claims like that. as researchers, we come up with the best possibilities and the best observations.

    i love your posts. thanks for sharing your ideas. great title as well.


  4. KevinCK said, on March 19, 2010 at 12:28 am

    Thanks, Lem.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: