Online CSAM offending is a challenge for law enforcement, policymakers and child welfare organisations alike. The use of online warning messages to prevent or deter an individual when they actively search for CSAM is gaining traction as a response to some types of CSAM offending. Yet, to date, the technical question of how warning messages can be implemented, and who can implement them, has been largely unexplored. To address this, we use a case study to analyse the actions individuals and organisations within the technology, government, nongovernment and private sectors could take to implement warning messages. We find that, from a technical perspective, there is considerable opportunity to implement warning messages, although further research into efficacy and cost is needed.
References
URLs correct as at November 2022
Abbott Т, Lai K, Lieberman M & Price E 2007. Browser-based attacks on Tor. In N Borisov & P Golle (eds), Privacy enhancing technologies. Lecture Notes in Computer Science vol 4776. Springer: 184–199. https://doi.org/10.1007/978-3-540-75551-7_12
Bailey A, Squire T & Thornhill L 2018. The Lucy Faithfull Foundation: Twenty-five years of child protection and preventing child sexual abuse. In R Lievesley, K Hocken, H Elliott, B Winder, N Blagden & P Banyard (eds), Sexual crime and prevention. Cham: Palgrave Macmillan: 57–82. https://doi.org/10.1007/978-3-319-98243-4_3
Belbeze C et al. 2009. Automatic identification of paedophile keywords. Measurements and Analysis of P2P Activity Against Paedophile Content Project
Bissias G et al. 2016. Characterization of contact offenders and child exploitation material trafficking on five peer-to-peer networks. Child Abuse & Neglect 52: 185–199. https://doi.org/10.1016/j.chiabu.2015.10.022
Broadhurst R 2019. Child sex abuse images and exploitation materials. In R Leukfeldt & T Holt (eds), Cybercrime: The human factor. Routledge: 310–336. https://doi.org/10.4324/9780429460593-14
Carr J 2017. A brief history of child safety online: Child abuse images on the internet. In J Brown (ed), Online risk to children: Impact, protection and prevention. Wiley: 5–21. https://doi.org/10.1002/9781118977545.ch1
Conroy S 2012. Child abuse material blocked online, removing need for legislation. Media release, 9 November. https://parlinfo.aph.gov.au/parlInfo/search/summary/summary.w3p;adv=yes;orderBy=customrank;page=0;query=%E2%80%9CChild%20abuse%20material%20blocked%20online,%20removing%20need%20for%20legislation%E2%80%9D
Coole M, Corkill J & Woodward A 2012. Defence in depth, protection in depth and security in depth: A comparative analysis towards a common usage language. Proceedings of the 5th Australian Security and Intelligence Conference. https://ro.ecu.edu.au/asi/
Europol 2022. Child sexual exploitation. https://www.europol.europa.eu/crime-areas-and-trends/crime-areas/child-sexual-exploitation
Europol 2020. Internet organised crime threat assessment 2020. https://www.europol.europa.eu/activities-services/main-reports/internet-organised-crime-threat-assessment-iocta-2020
Google 2020. Fighting child sexual abuse online. https://protectingchildren.google/intl/en/
Green MD & Stamos A 2021. Apple wants to protect children. But it’s creating serious privacy risks. New York Times, 11 August. https://www.nytimes.com/2021/08/11/opinion/apple-iphones-privacy.html
Hammond D 2011. Health warning messages on tobacco products: A review. Tobacco Control 20(5): 327–337. https://doi.org/10.1136/tc.2010.037630
Holt TJ 2018. Regulating cybercrime through law enforcement and industry mechanisms. Annals of the American Academy of Political and Social Science 679(1): 140–157. https://doi.org/10.1177/0002716218783679
Holt TJ, Cale J, Leclerc B & Drew J 2020. Assessing the challenges affecting the investigative methods to combat online child exploitation material offenses. Aggression and Violent Behavior 55: 101464. https://doi.org/10.1016/j.avb.2020.101464
Internet Watch Foundation (IWF) 2020. Keywords list. https://annualreport2020.iwf.org.uk/tech/keyservices/keywords
Internet Watch Foundation (IWF) 2013. New study reveals child sexual abuse content as top online concern and potentially 1.5m adults have stumbled upon it. https://www.iwf.org.uk/news/new-study-reveals-child-sexual-abuse-content-as-top-online-concern-and-potentially-1-5m-adults
Interpol 2020. Threats and trends child sexual exploitation and abuse: COVID-19 impact. https://www.interpol.int/en/News-and-Events/News/2020/Interpol-report-highlights-impact-of-COVID-19-on-child-sexual-abuse
Loesing K, Murdoch SJ & Dingledine R 2010. A case study on measuring statistical data in the Tor anonymity network. In Financial cryptography and data security. Lecture Notes in Computer Science vol 6054. Springer: 203–215. https://doi.org/10.1007/978-3-642-14992-4_19
Maimon D, Alper M, Sobesto B & Cukier M 2014. Restrictive deterrent effects of a warning banner in an attacked computer system. Criminology 52(1): 33–59. https://doi.org/10.1111/1745-9125.12028
Microsoft 2022. PhotoDNA. https://www.microsoft.com/en-us/photodna
Morgan S & Lambie I 2019. Understanding men who access sexualised images of children: Exploratory interviews with offenders. Journal of Sexual Aggression 25(1): 60–73. https://doi.org/10.1080/13552600.2018.1551502
National Center for Missing and Exploited Children 2022. 2021 CyberTipline reports by electronic service providers (ESP). https://www.missingkids.org/gethelpnow/cybertipline/cybertiplinedata
Price C 2021. 20 great search engines you can use instead of Google. https://www.searchenginejournal.com/alternative-search-engines/271409
Prichard J et al. 2022a. Online messages to reduce users’ engagement with child sexual abuse material: A review of relevant literature for the reThink chatbot. Report for the Lucy Faithfull Foundation. Hobart: University of Tasmania. https://eprints.utas.edu.au/46223/
Prichard J et al. 2022b. Warning messages to prevent illegal sharing of sexual images: Results of a randomised controlled experiment. Trends & issues in crime and criminal justice no. 647. Canberra: Australian Institute of Criminology. https://doi.org/10.52922/ti78559
Prichard J et al. T 2021. Effects of automated messages on internet users attempting to access “barely legal” pornography. Sexual Abuse 34(1): 106–124. https://doi.org/10.1177/10790632211013809
Prichard J, Krone T, Spiranovic C & Watters P 2019. Transdisciplinary research in virtual space: Can online warning messages reduce engagement with child exploitation material? In R Wortley, A Sidebottom, N Tilley & G Laycock (eds), Routledge handbook of crime science. London: Routledge: 309–319. https://doi.org/10.4324/9780203431405
Protect Children 2021. ReDirection launch: ‘Protecting children through prevention’. https://suojellaanlapsia.fi/2021/09/27/redirection-launch-protecting-children-through-prevention/
Quayle E 2013. Organizational issues and new technologies. In M Erooga (ed), Creating safer organizations: Practical steps to prevent the abuse of children by those working with them. John Wiley & Sons: 99–121. https://doi.org/10.1002/9781119943228.ch6
Quayle E & Koukopoulos N 2019. Deterrence of online child sexual abuse and exploitation. Policing: A Journal of Policy and Practice 13(3): 345–362. https://doi.org/10.1093/police/pay028
Sanchez L, Grajeda C, Baggili I & Hall C 2019. A practitioner survey exploring the value of forensic tools, AI, filtering, & safer presentation for investigating child sexual abuse material (CSAM). Digital Investigation 29: S124–S142. https://doi.org/10.1016/j.diin.2019.04.005
Schneider J 2021a. Apple confirms it will scan iPhone photo libraries to protect children. PetaPixel, 5 August. https://petapixel.com/2021/08/05/apple-confirms-it-will-scan-iphone-photo-libraries-to-protect-children/
Schneider J 2021b. Apple wipes controversial child abuse photo scanning from its site. PetaPixel, 16 December. https://petapixel.com/2021/12/16/apple-removes-all-references-to-csam-photo-scanning-from-its-site/
Seto MC & Ahmed AG 2014. Treatment and management of child pornography use. Psychiatric Clinics of North America 37(2): 207–214. https://doi.org/10.1016/j.psc.2014.03.004
Smallbone S & Wortley R 2017. Preventing child sexual abuse online. In J Brown (ed), Online risk to children: Impact, protection and prevention. Wiley: 143. https://doi.org/10.1002/9781118977545.ch8
Steel CM 2015. Web-based child pornography: The global impact of deterrence efforts and its consumption on mobile platforms. Child Abuse & Neglect 44: 150–158. https://doi.org/10.1016/j.chiabu.2014.12.009
Thorn 2020. The road to Safer: Equipping industry to end CSAM. https://www.thorn.org/blog/announcing-safer-built-by-thorn-eliminate-csam/
WePROTECT Global Alliance 2021. Global threat assessment 2021: Working together to end the sexual exploitation of children online. https://www.weprotect.org/global-threat-assessment-21/#report
WePROTECT Global Alliance 2020. Voluntary principles to counter online child sexual exploitation and abuse. https://www.weprotect.org/library/voluntary-principles-to-counter-online-child-sexual-exploitation-and-abuse/
WePROTECT Global Alliance 2019. Global Threat Assessment 2019: Working together to end the sexual exploitation of children online. https://www.weprotect.org/issue/global-threat-assessment/
Westlake B 2020. The past, present, and future of online child sexual exploitation: Summarizing the evolution of production, distribution, and detection. In T Holt & A Bossler (eds), The Palgrave handbook of international cybercrime and cyberdeviance. Cham: Palgrave Macmillan: 1225–1253. https://doi.org/10.1007/978-3-319-78440-3_52
Westlake B, Bouchard M & Frank R 2017. Assessing the validity of automated webcrawlers as data collection tools to investigate online child sexual exploitation. Sexual Abuse 29(7): 685–708. https://doi.org/10.1177/1079063215616818
Westlake B, Bouchard M & Girodat A 2017. How obvious is it? The content of child sexual exploitation websites. Deviant Behavior 38(3): 282–293. https://doi.org/10.1080/01639625.2016.1197001
Williams KS 2005. Facilitating safer choices: Use of warnings to dissuade viewing of pornography on the internet. Child Abuse Review: Journal of the British Association for the Study and Prevention of Child Abuse and Neglect 14(6): 415–429. https://doi.org/10.1002/car.920
Wolak J, Liberatore M & Levine BN 2014. Measuring a year of child pornography trafficking by US computers on a peer-to-peer network. Child Abuse & Neglect 38(2): 347–356. https://doi.org/10.1016/j.chiabu.2013.10.018
World Health Organization 2020. Joint leaders’ statement: Violence against children: A hidden crisis of the COVID-19 pandemic. https://www.who.int/news/item/08-04-2020-joint-leader-s-statement---violence-against-children-a-hidden-crisis-of-the-covid-19-pandemic
Wortley R & Smallbone S 2012. Internet child pornography: Causes, investigation, and prevention. Santa Barbara, CA: Praeger