Tuesday 2 October 2012

Potential selection bias problems

Oh bugger. I've stumbled across a problem that I hadn't anticipated. One that will introduce selection bias if I'm not careful.

The baseline assessment of spatial ability is being done via 2 tests: The Vandenburg and Kuse Mental Rotation Test and the CEEB Mental Cutting Test. Both are reliable, valid and well suited to the study. Example problems from both tests are here:

The top one is an example from the MRT (A) battery. The bottom one is an example from the MCT.

I got a couple of large groups of participants to do these tests immediately after consent to participate. The rationale for asking them to do this was as follows:
  1. They need to be timed and invigilated.
  2. I need the data to add to the minimisation algorithm before allocation.
  3. It saves a lot of time on the main data collection days that they attend.
Both tests are quite 'tough' and yesterday I noted about 5 participants give up after a couple of minutes because they couldn't do them (easily?) and deciding to then not participate in the research because of this. Clearly these are those who are particularly low on the spatial ability (SA) continuum and ones who I would ideally like to include in the research.  Through trying to be organised I have introduced some selection bias. I'm not sure that it would be ethical to follow these individuals up and encourage them to have another go and participate.

Possible solutions?
  • Do these tests after other aspects of data collection. This would mean using simple randomisation  instead of minimisation (actually someone from the research centre wondered why I wasn't doing this anyway, suggesting that it would be perfectly acceptable) but I could still end up with attrition later in the study if they give up on the SA tests later rather than sooner?
  • Explain to other prospective participants that the SA tests are quite tough and that low scores are not something to worry about. Essentially try to encourage them to give them a go and continue with participation.
  • Accept that attrition will occur as a result of this, accept it and discuss the limitations in the thesis.
Not sure what to do yet.
Posted by at

3 comments:

  1. Rob2 October 2012 at 11:50

    This comment has been removed by the author.

    ReplyDelete
  2. Anonymous2 October 2012 at 14:40

    Attrition is a problem, you can do some sensitivity analysis to look at the impact of missing data after the event but it is preferable to avoid attrition if possible. Which of the two tests did participants give up on, was it the same one? Does there need to be two tests?
    My preference would be to suggest to participants that the tests are tricky but to give it a go and not to worry about low test scores and explain the relevance in relation to this study.
    I'm not sure I agree with the person that said simple randomisation is likely to be sufficient, we've discussed this before I'd need to know the context of their recommendation for simple randomisation being sufficient.
    H

    ReplyDelete
  3. Rob3 October 2012 at 02:31

    They gave up on the first one (possibly the easier of the 2)!

    I like the suggestion re how to proceed. Was my preference.

    Will chat to you later about the randomisation. Have a meeting with Karen K on Monday.

    ReplyDelete

Subscribe to: Post Comments (Atom)