The aim of all research is to answer questions
(Robson, 2011); but unfortunately the processes, as well as the answers, are
not always that simple. There are different issues that can work to impede
and/or complicate research within and between the hard sciences (biology, chemistry,
physics, psychology and medicine) and the social sciences (criminology,
sociology, social work and psychology[i]). Within social science
orientated research, we are using social science in its broadest capacity here to
discuss any and all research that examines human participants as well as the
relationship between the self and society; and there is a vast array of
different methodologies to choose from. I would firmly place sex offender
research within this bracket. All research is about using tried and tested
methodologies to answer specific questions, which means that you need to use
the right methodology to answer the research question that you have to answer;
this can sometimes be a bone of contention (Robson, 2011; Chambers, 1999).
Within any given research project there are
often a diaspora of “interested” parties (for instance, funders, government
bodies, other stakeholders) who directly or indirectly impact upon the
research, when these “interested” parties also have a vested interest in the
research (they are funding it, supplying data or access) they will get more
vocal in promoting their viewpoint. All of these “interested” parties want the
research to say and do different things, which can be quite frustrating
(Robson, 2011; Davies, Francis & Jupp, 2011). Within sexual abuse research,
these viewpoints can become difficult to navigate. There are a lot of
“interested” parties; sexual abuse research does not exist in isolation and its
outcomes can impact a swath of professions (police, probation, prisons,
therapists, counsellors, community groups) and a number of issues (prevention,
public protection, criminal justice, treatment, risk management), all of which
can confine research practices.
In the main, when we talk about academic rigour
and research with impact, especially in the Criminal Justice field, we are
talking about inferential statistics and quantitative methodologies (NESTA). The prevailing view among both the academy and
stakeholders, especially in government, internationally is that empirical,
quantitative methodologies are the always the most appropriate approach to take
when conducting research. Interestingly, there seems to be a step away from the
traditional international governmental attitude to quantitative research and
statistics within branches of government in Canada, where the emerging attitude
seems to be that inferential statistics should not be used at all!
This international disparity throws up the real
issue, that is that judgments are being made on the viability and impact of
research based upon attitudes to methodology NOT, as it should be, attitudes to
the whole research process and how the methodology sits in respect to the
question. We should not play favorites with some methodologies, nor should we
blacklist or ostracize others out of context. We need to make sure that the
full methodological toolkit is available to us so that we can better develop
and execute appropriate research; especially given that sexual abuse is a
multi-disciplinary area and needs representation from different disciplines and
methodologies. Ultimately, we should applaud all attempts to improve our
knowledge rather than criticizing existing research because it couldn’t be of
the highest quality. This is particularly true in an era when policy-makers are
often unwilling to accept guidance from scientific findings. For instance, in
the UK the Scottish (Chan et al, 2010) and English (Kemshall et al, 2010) of
the limited disclosure scheme indicated a very low take up of the schemes, a
lack of engagement with the schemes by the public and low levels of real world
impact in policing terms all of which would indicate that the disclosure scheme
was not doing what it was supposed to did not stop the government implementing
it. A case of policy based evidence over evidence based policy maybe? There is
no question that we should want the best methodologies; and we should want to
best overall processes and options as well.
Not all methodologies suit all research
questions, regardless of what some interested parties think, and using an
inappropriate methodology will compromise the research study, impact upon the
quality of the research as well as the outcomes (Robson, 2011). For instance,
- Randomized
Control Trials
are often seen as the gold
standard in evaluation research (Robson, 2011) and have been historically, and successfully used
in social
science research,
and are now being used more often in criminal justice research (Duwe, 2012;
Singer & Cooper, 2009), as criminology is starting to become more
quantitative and experimental. However, from Larry
Sherman and Richard Berk’s domestic
violence RCT,
the first criminal justice RCT, to Grant Duwe’s (2012), both in Minnesota and
both related to interpersonal violence interestingly, the use of RCTs is better
able to polarize a group of researchers above and beyond any other methodology.
The main concern with RCT’s is that they are inferentially powerful tools and
should only be used accordingly, namely with a strong, variable and disperse
sample. Therefore using an RCT to evaluate a well-established mainstream sex
offender programme would be appropriate, relevant and useful; whereas using it
to evaluate a pilot, specialized programme with 5 participants would not. This is not to say that RCT’s should not be
used, but rather there are limits to the functionality, reliability, validity
and results that it would produce; which would problematise the results and
give a potential false negative. In
addition to the practical issues with RCTs there are also moral dilemmas as
well as ethical issues relating to alternatives to treatment, public protection
and offender care which although not limited to the sex offender field are very
pertinent to it.
- Using a purely quantitative approach to test the
effectiveness and impact of an emerging paradigm. An example would be Circles
of Support and Accountability (CoSA), where one would be better placed to use a
multi-strategy design, a case study design, or even a purely qualitative
approach to understand the processes and analysis the impact of CoSA, which,
far from a simple treatment program is a complex social phenomenon (McCartan et
al, 2014). That is to say, an RCT of CoSA should take place further down the
line when a quantitative study with an appropriate (and sizeable) sample, but
doing it with an unrepresentative or inappropriate sample would skew the
results and problematise the research. Criticizing the methodology of existing
studies too early in the research process (as has been the case in many
spheres) misses the point and risks sending the wrong message to stakeholders.
The important thing to realize is that the
choice to use a non-inferential methodology should not be seen as weakening the
coherence of the research, its outcomes or impact just because you cannot give
a level of significance to 0.05 or 0.01. If, through qualitative of case study
research, you can demonstrate that a majority of your participants experienced
a positive outcomes you are indicating that your research is having an impact,
regardless of not having a level of significance. Instead you are showing that
you are making an informed decision to use the correct methodology to answer
your research question in the most appropriate light and that your results
reflect the reality of what you are investigating. This means that the fringe
benefits of research, like contributing to evidence based policy and practice,
are realistic and grounded.
It is important to stay open minded to all forms
of research methodology and use the one best suited to your research question,
there are many ways to show success and impact so why decide to limit them.
Kieran McCartan, PhD
David S. Prescott, LICSW
References
Chambers,
A. F. (1999). What is this thing called
Science? 3rd Edition. Open University Press: Buckingham.
Chan, V.,
Homes, A., Murray, L., & Treanor, S. (2010). Evaluation of the Sex Offender
Community Disclosure Pilot. Scottish Government Social Research. Retrieved
December 6, 2010, from http://www.scotland.gov.uk/Resource/Doc/328113/0106001.pdf.
Davies,
P., Francis, P., & Jupp, V. (2011). Doing
Criminological Research, 2nd Edition. London: Sage.
Duwe, G. (2012) Can Circles of Support and Accountability (COSA) Work in
the United States? Preliminary Results
From a Randomized Experiment in Minnesota.
Sexual Abuse: A Journal of Research
and Treatment, online access August 1st 2012 at: http://sax.sagepub.com/content/early/2012/07/26/1079063212453942.abstract
Kemshall,
H. & Wood, J., with Westwood, S., Stout, B., Wilkinson, B., Kelly, G.,
& Mackenzie, G. (2010). Child Sex Offender Review (CSOR) public disclosure
pilots: A process evaluation. London: Home Office. Retrieved March 31, 2011,
from http://rds.homeoffice.gov.uk/rds/pdfs10/horr32c.pdf.
Robson,
C. (2011). Real world research, 3rd
Ed. John Wiley.
McCartan,
K., Kemshall, H., Westwood, S., Solle, J., MacKenize, G., Cattell, J., &
Pollard, A. (2014) Circles of Support and Accountability (CoSA): A case file
review of two pilots. Project Report. Ministry of Justice.
Singer,
L., & Cooper, S. (2009). Improving Public Confidence in the Criminal
Justice System: An Evaluation of a Communication Activity. Howard Journal of Criminal Justice, 48 (5), 485 -500.
[i] Please note the ambiguous labeling of psychology as “hard” (i.e.,
cognitive psychology, vision and perception) and “soft” science (i.e., social
psychology), this is because psychology is a wide and varied discipline that in
different topic areas it is both.
No comments:
Post a Comment