How do you figure out what works in your own classroom? Say you have come up with a new way to teach phonics or fractions – how could you tell whether the new method you use is better than current practice? Here are some suggestions of things to keep in mind when you want to evaluate your new evidence-based practice:

• Comparison – you should use the new technique on some kids but not others, and see which group improves the most

• Random allocation – the children should be put at random into the groups who get the new technique versus the current practice. If the choice isn’t random, effects can get confused (say the kids in one group happen to be older, or brighter, then this would mess up whether it’s the new technique or the group choice that’s producing any difference)

• Active agent – if your technique works and you want other people to use it, you need to know why it works. Let’s say you think teaching fractions by pretending to run a local pizza restaurant. How much pizza does each person get on a table of six? After work, three waiters and waitresses split tips of 18 pounds, how much does each get? Let’s say you’re confident this approach works better than current practice. Other teachers are likely to want to adjust their method to their kids. You need to be able to identify the key parts of your new method. Is it important that it’s a restaurant? Would a casino or a farm setting work, too? Is it important that you are dividing circles? Or (more likely) is it important that you are combining division-into-groups with splitting-a-whole-into-portions, embedded in a familiar context? You need to identify the active agent so that other teachers can adapt your technique always retaining the active agent.

• Control group – if you tell kids they are getting a special new technique, they might do better because they are more excited and feel special. To check that your technique is working for the reason you think it is, you may need a control group. This would be a group of children doing something similar to your new technique, differing only in whether the ‘active agent’ is present or not. Perhaps the ‘control’ group could do role-play of working in a restaurant, but not using division problems. You may end up with three groups to compare, then: new technique, control, and current practice.

• Outcome measure – decide in advance what you are going to measure to check whether your technique has worked (e.g., a fractions test). Give it to the children before you try your technique (a ‘pre-test’). Give it again afterwards (a ‘post-test’). See which group improves the most.[i] You might also include another test, where you wouldn’t expect the new technique to improve performance (e.g., a spelling test).

• Be blind if you can – there is a long history of researchers inadvertently encouraging the children using the new technique to do better (perhaps smiling at them more during testing!). If at all possible, when testing the children, it’s much better not to know which group a given child is in. Perhaps you might buddy up with another teacher in the same year group, to test each other’s children, both ‘blind’ to which child is in which group?

• Be objective – when we want something to work, we are all at risk of just looking for evidence that we are right (trying to ‘confirm’ our idea). Try to be objective, and look for what else is going on. Are other skills improving? Are other skills getting worse, compared to the current practice? Are you focusing only on the odd child who shows a big improvement rather than considering the whole group?

• Kids differ – the new technique may work for some children but not for others. If so, how would you decide whether it’s worth pursuing or not? Can you tell who it works for?

• Ethics – make sure beforehand whether there are any risks in what you’re proposing to do, and also, whether you are being fair to all the children. For example, if you are very confident your technique will work, you still need a group of children who don’t get the technique to compare against, but you could decide to give those children the technique later on. So all children get the new technique, just at different times.

• What’s true? Evidence-based practice is about working out what really works in the classroom. Folk law or the Internet may claim something works, but does it really? How would you and your class find out? Get into the habit of checking what’s really true. Does a claim seem likely? How much do other people (or even you) want it to be true? Where is the evidence? Who found it? Is the evidence any good?

The suggestions here may seem a little cumbersome. But here’s the problem. To improve classroom practice is not just a question of finding out what works one time in one classroom. Unless evaluations of techniques identify the crucial components of techniques and who they will work for, we end up with ‘innovations’ that don’t work when other people use them and sometimes don’t work when we use them again ourselves with other children!

Here are two leaflets we have prepared, summarising these tips:

Download For teachers: What works in my classroom

Download For clinicians: What works in my practice

Written by Prof. Michael Thomas and Dr. Jo van Herwegen

©CEN

[i] How to evaluate your new practice: In order to know whether your new technique worked you can now compare the children’s post-test scores to their pre-test scores. As you will see not all children might improve to a similar degree and not all children may have had the same pre-test scores. So a way to compare like for like is to calculate an improvement score that represents the child’s pre-test score as well. To do this you work out: (post-test score minus pre-test score) divided by pre-test score.

You now have a score that shows how much they improved from their pre-test score. Put the scores from small to large. Not all children may have improved equally. You may want to look at who are the children that improved the most and benefitted the most from the new technique and who did not improve at all as this will help you to decide whether the new technique is worth pursuing.