Evolution does not mandate revolution: the role of evaluation and developing an evidence base.

One of the biggest debates that we have in the field of social science research is the role of evaluation (e.g., evaluation of a policy, the completion of treatment goals or of the program itself) and where independence sits within it. Should organisations and governments conduct their own evaluations, or does that create a conflict of interest? Is all independent research really independent, or is it just independent of the organization/provider? How independent is it of the researcher’s own biases? It is important to consider these factors when deciding to do research, validate programmes, conduct evaluation and implement change.

The place to start is asking the right questions: What is the purpose of evaluation, research, and development of an evidence base; is it to justify what is being done, a required part of a larger grant or project, to see if what is being done is working, to justify an existing programme/venture or to implement something new in a broader fashion? Is it about “evidence based policy” [research driven] or “policy based evidence” [ideologically driven]? On the other hand, is an organisation implementing an evidence-based practice or developing practice-based evidence in earnest?

The reasoning behind the purpose of the research is as important as the research being conducted, especially if the research is being used to justify or implement something; therefore, is the research question trying to see if something is working or if something is going to work in a different setting. This means that evaluation can often usher in change to policies and programmes, which can be minor or major but often seen as significant by people on the ground. This can, in turn, bring about new questions: Should an organisation implement a top-down approach or method, or integrate it in accordance with the unique characteristics and contributions of those who work and benefit from it? It is important that everyone involved in the process recognizes the importance of evaluation in the evolution of our research and working; it’s not a threat but rather an opportunity to see if something works, to identify good practice and identify bad practice. Therefore, who should conduct evaluation research; organisations themselves or external organisations/researchers?

It is generally viewed that good research is independent research as organisations are often considered to be biased in how they evaluate their own programmes and practices. The notion that organisations cannot conduct their own critical, methodologically sound and unbiased research is outdated, problematic and questionable. The issue is not with research or methodology, but rather with how organisations understand, priorities and train their staff in research. If staff do not fully understand research and its impact, then their ability to develop, conduct, justify and roll out research becomes problematic. This is a training issue and investment issue, not an independence issue. This is reinforced by the fact that not all research and researchers are independent with the same researchers being used by the same organisations frequently and/or with research tending processes being subjective.  Which means that so called independent research can be as subjective and biased as research conducted by the organisation itself; particularly, if the organisations do not have their own in house understanding of research, their own critical reflection of research and how to process it.

The processes involved in research involve interaction between the researcher and the organisation, regardless of whether the research is independent or not, with both needing to be able to work together to collect the data and implement change. A decision not to engage in the research process or implement changes is problematic regardless of who conducts the evaluation and has nothing to do with independence, but rather it’s about quality control.

The most important part of doing evaluation research is the process and methodology, it is about accepting the good and the bad on both fronts [researcher and organisation]; it’s about us and us, not them and us. Evaluation is about change and evolution, not revolution and condemnation. Evaluation is about maintaining funding, generating funding, maintaining good practice and eliminating bad practice. It’s a process, not an end point.

Kieran McCartan, PhD, & David Prescott, LICSW 

The Adam Johnson Trial: Talking healthy and appropriate sexual relationships with youth.

UPDATED: This blog ties to two previous blogs Age of consent in the UK &  Celebrity, Sexual Abuse and Societal Reaction. Kieran 

The recent Adam Johnson trial in the UK has raised lots of questions about how we think and talk about sexual relationships with youths and teenagers, especially what they perceive an appropriate sexual relationships to be and why. Are we having the correct conversations with the correct people and the right time? Also, how do we react to youths when they ask about sex, talk about relationships or disclose sexually harmful behaviours?

Adam Johnson [28 years of age] is a premier League footballer who played for Sunderland and the English national team, he was a local and national celebrity who was held in high regard; yesterday he was sentenced to 6 years for grooming, kissing and having sexual activity with a 15 year old girl. The victim was a supporter of Sunderland Football Club and a fan of Johnson, they came into contact via social media where she friended him via Facebook and conversed via whatsapp. During the time that the abuse was occurring Johnson and his victim exchanged over 800 whatsapp messages [on a second whatsapp account that he created to converse with the victim only and keep these conversations a secret] which clearly demonstrated Johnson’s grooming behaviour, including attempts to silence the victim, cover up the abuse and a conversation about her age. In addition, Johnson’s phone records and internet records showed that he searched for the age of consent in the UK, checked teen sites [nice young teens] and looked at extreme pornography [mainly linked to bestiality]. A psychologist involved in the case stated that they did not think that Johnson was a paedophile or sexually interested in children, but rather that his offence was an extension of a sexually promiscuous lifestyle and the opportunities, as well as attitudes that accompanied being a high profile footballer.

The Adam Johnson, while problematic, is not the only high profile one relating to celebrity that has emerged in recent months and years; the surprising aspect of the case is our [societies] reaction to it and its acceptability in some quarters. In respect to the Johnson trial we have seen;

1.     Victim blaming and abuse, with the internet and social media targeting the victim

2.     The story being about his football career and his careless in throwing it away

3.     The reaction to the story by the club, which was highly criticised  

4.    A misunderstanding of the nature of what sexual abuse is, its impact and the terminology [especially the phase paedophile] by the public and media

5.     Public misperceptions of how judges sentence in these cases and what it means in real terms.

The real stories at the heart of the Johnson trial are [1] how we talk to our children and teenagers about sex, healthy sexual relationships and appropriate behaviour; [2] abuse by people in positions of respect, trust and the safeguards in place to prevent the abuse occurring; and [3] how we understand and respond to inappropriate sexual behaviour between youths and adults.

The real story here is about a teenager who got to meet, was groomed by and taken advantage of by a celebrity that she idolised. The real story here is about how youths [children and teenagers] know how to identify sexually problematic situations, how they react in these situations and where they can go for help. The real story here is about an adult who made inappropriate and problematic decisions without any thought for the victim, only for themselves.

In the Johnson case the court of public opinion was split, but vocal on both sides with some members of the public supporting him, stating that the teenager lead him on and that she knew what she was doing, that she was “old enough to know better” [they created a facebook page “Justice for Johnson” which has since been removed because of inappropriate comments and content]; but given the victims statement, that is not true – it was a perfect storm. It is clear that Johnson groomed his victim and people surrounding him. The public discussion of the Johnson case does clearly demonstrate the need for greater public education and discussion regarding sexual activity in and with youths; it has become clear that sections of the public did not see the abuse as ‘abuse’ or even problematic ignoring the abuse of trust and the responsibility of the perpetrator to act appropriately. As we know abuse of any kind can have a lasting and significant effect upon the victim, as is demonstrated in this case, and that there are not degrees of acceptability in sexual harm.

What this case does reinforce is the need for better sexual education, better relationship advice, better safeguarding advice and a need for a range of responsible citizens (parents, teachers, etc) to deliver a consistent message. The conversation about healthily sexual relationships is a difficult for families and the state to navigate, who is responsible for having the conversation in the first place [especially when we have a paradoxical mind-set to talking to our youths about sex, in that we try to protect them by not really discussing sex and relationships but this may result in them being in risky situations and making poorer decisions], the parents, the school, peers, the internet, all or none? We do know that in the UK the state, via schools, are not providing a consistent response as they believe that sexual education [including what is an appropriate sexual relationship and behaviours] is a not compulsory part of the national curriculum that should be taught the same in all schools nationally; therefore sexual education is inconsistent and incomplete nationally. Hence reinforcing an air of confusion and ambiguity, for whose responsibility is it [schools, parents, peers, the media or all?], are we surprised that our youths look elsewhere for answers [media, celebrity, pornography] and do not feel comfortable disclosing sexual issues that they face with families as well as friends if they receive no response or a negative backlash? The learning curve in the Adam Johnson trial is not just limited to him, his behaviour and celebrity culture; but it is also linked to us as a society and our responses to victims of sexual harm, their disclosures and how we discuss healthily sexual relationships with youths. The Adam Johnson trial makes us question how far we have moved forward in discussing sexual harm and society’s response it; do we fully understand sexual harm, what it looks like, its causes and consequences?  

Kieran McCartan, PhD 

Talking prevention with Maia Christopher – from prison to prevention in 12 easy badges

Maia Christopher is ATSA’s executive director, most people will recognise her from conference, those who have had the chance to speak and meet with her will recognise that the majority of the work she does on a daily basis supports ATSA’s mission in the field of sexual harm prevention. Maia has many roles which include sitting on national and international committees, speaking at conferences and other events, as well as helping to frame the sexual harm agenda. I talked to Maia about her work, where she came from, how she got to where she is today and where she sees herself in the future. We also spoke about the field of sexual harm moving into the future.

Given Maia’s expertise and experiences from her criminal justice and treatment days in Canada, to taking over and managing ATSA, to the high level collaborative work she is involved in now, it was hard to identify how to ground this narrative. Then she gave me an example, an unconventional piece of prevention work that she had recently been involved with and everything slotted together.  Maia told me of a recent experience that she and Kelly McGrath, Associate Director of ATSA, had when they attended a conference.

Every year ATSA has a conference and every year Maia and the ATSA staff assist in the development of the programme around it. From arranging book sellers to sorting out finances and planning everything else that needs to happen over the main 5 days of the conference.

Maia told me that herself and Kelly recently attended an event planning conference, the aim of which was to give conference organisers insights, ideas and new approaches to better plan and streamline large conferences. The night before they left for the conference they were deciding what to bring with them to promote ATSA and the work that, as an organisation, is presented at ATSA’s annual conference.  They items they took included  twenty-four prevention badges emblazoned with slogans promoting ATSA’s work. Maia and Kelly distributed these badges across the event planning conference venue the night before the conference, out of the sight of other attendees. The simple distribution of the prevention badges had a massive impact during and beyond the life of the conference. Attendees began having conversations with each other and with Maia about sexual abuse, victims, policy, prevention, how people are impacted by sexual harm, what treatment looks like, whether offenders are mad or bad and related news coverage.

People picked up the badges and thought about how sexual harm had affected their lives and/or the lives of people close to them. Seeing the badges promoted people to talk, some of whom were uncomfortable about the conversation, but they all felt the conversation was one that needed to be had. The reason why this is important is because these individuals where not representatives from the arena of sexual harm or related professions like police, probation or prisons— “outsiders” not ”insiders”. This meant that Maia, Kelly and the badges were not preaching to the converted, but people from a range of different, unrelated and everyday professions who were more representative of the public at large.

Although, the debate and discussion among the delegates was important it was not the only significant thing that happened. Across the life of the conference the attendees started to think about how conferences, as an entity, could be more productively used to highlight and discuss issues of sexual harm; including information about safeguarding attendees, local support service on sexual harm matters and how an increase in awareness of sexual harm issues for hotel staff could be provided. Maia felt that a door had been opened on to the topic of sexual harm by people not working in the field who wanted to discuss and promote it in the areas in which they worked.

Maia felt that the badges had a massive impact, that they were an ice breaker, a mechanism through which people could discuss sexual harm and relate it back to their lives and experiences while also thinking about how they could incorporate it into their daily workings. Interestingly, Maia felt that the real success here was in stepping out of the field, going to non-sexual harm related conferences and talking about the issue. She feels that this is part of her, and every ATSA member’s job. Although, she did recognise that she would have felt it was a massive leap of faith when she was working in the criminal justice system as opposed to the smaller leap of faith that she has to take in the role she has now. Maia closed the conversation with some pertinent advice to us all when talking to people who do not work in the sexual harm field about the main issues – “keep it simple, keep it to the point, listen and respond”. 

Kieran McCartan, PhD

Q & A with Kevin L. Nunes and Chantal A. Hermann on call for papers for a special edition of SAJRT on “Connecting Theory With Research: Testing Hypotheses About the Causes of Sexual Offending”

This is the second of two Q & A posts, over the next couple of weeks, on upcoming special editions of SAJRT. There is currently a call for papers out on a special edition related to “Connecting Theory With Research: Testing Hypotheses About the Causes of Sexual Offending”, please read the blog below and if you are interested in submitting an article follow the instructions on the SAJRT website (http://sax.sagepub.com/content/28/1/73.full.pdf+html)  – Kieran 

What is the topic of the special issue?

We are very happy to be guest editors of this special issue of Sexual Abuse: A Journal of Research and Treatment! (Follow this link to see the call for papers: (http://sax.sagepub.com/content/28/1/73.full.pdf+html.) This special issue is meant to take stock of the state of the available evidence regarding theoretical assertions about the causes of sexual offending and to provide guidance for future research. We are hoping to receive manuscripts that focus on questions such as the following: What evidence do we have regarding assertions about the causes of sexual offending made in theories/models of sexual offending? What evidence is missing? How credible is the available evidence? What methodological approaches (e.g., design, measurement, analysis, etc.) will yield more conclusive evidence?

What is the story behind this special issue?

We wanted to facilitate discussion in the field about research on the causes of sexual offending. We got things started by conducting a pre-conference seminar with Michael Seto, Tony Beech, and Patrick Lussier at last year’s ATSA conference in Montreal. We received generous funding and other support from the Social Sciences and Humanities Research Council (Connection grant), ATSA, and the Carleton University Psychology Department for our pre-conference seminar—thank you very much! (The handouts from our pre-conference seminar are available on Kevin’s lab website: www.carleton.ca/acbrlab)  We are now aiming to take the discussion further with this special issue of Sexual Abuse: A Journal of Research and Treatment. We thank Michael Seto for his support and SAGE for generously providing complimentary open access publishing through toll-free linking for articles in this special issue—thank you very much!

Why is this topic (communicated through the pre-conference seminar and special issue) important for researchers and practitioners who work to prevent sexual abuse?

Identifying causes of sexual offending is the foundation of effective and efficient assessment and intervention aimed at managing and reducing sexual offending. Practice is often guided by implicit or explicit assumptions about the causes of sexual offending. For example, if you believe that changing attitudes (or any other factor you think is important) through treatment reduces the likelihood of sexual offending, then you believe that attitudes are a cause of sexual offending. Thanks to the efforts of pioneering researchers and practitioners, impressive advances have been made in our field. Numerous theories and models provide carefully considered and well-informed hypotheses about the causes of sexual offending. However, credible tests of these hypotheses are remarkably rare. Even more remarkable is that this important gap in scientific knowledge seems to be recognized or acknowledged by so few people in our area, researchers and non-researchers alike. We know that testing such hypotheses is very difficult, but methodological rigor is a matter of degree and there is certainly room for improvement. In part, the scarcity of rigorous tests may be due to uncertainty about the relevant empirical evidence available and the optimal methodological approaches required. Our hope for this special issue is that it will raise awareness about important gaps in knowledge regarding the causes of sexual offending and identify ways to narrow those gaps. We believe this will help researchers to do more rigorous and informative studies on the causes of sexual offending, which, ultimately, will help practitioners and policy-makers to more effectively and efficiently reduce sexual offending.

Kevin L. Nunes, PhD, and Chantal A. Hermann, PhD.