The AISC-GP SimED study is the first study of clinicians’ use of the simulation-based educational tool for skin and mole cancer
Why only learn from future patients when you can educate yourself on previous?
Skin cancer diagnostics is challenging and mastery requires years of practice, with continuous exposure to hundreds of lesions. Treatment is delayed in 25-28% of melanomas because they are referred from general practice (GP) to dermatologists outside of the cancer pathway, and the vast majority of pigmented lesions referred in cancer pathways are benign. This is costly for the society, leads to patient anxiety and longer referral time at the dermatologists’ offices.
We’ve recently shown that simulation-based education can improve medical students’ diagnostic accuracy of skin and mole cancer significantly after just 3 hours of training (publication in review).
With this study we aim to test the efficiency this newly developed digital patient-case-based pattern recognition training system on the diagnostic accuracy of Danish doctors employed in General Practice.
Randomized Trial
The trial is a randomized controlled trial with an intervention and a control group, but with an allocation ratio of 3:1. This allocation allows for further subgroup analysis.
GPs were invited at skin cancer sessions at “Lægedage” 2021. Participants filled out a questionnaire and answered a skin
cancer multiple choice quiz (MCQ) with 12 patient-cases with prior validity evidence.
Group A (Intervention) was given access to the DermLoop Learn education platform for 8 days, and then waited without using the platform for another 8 days before answering the final MCQ test.
The educational platform contains 2376 anonymized patient-cases with a
clinical and dermoscopic image, along with educational material on the 35 most common benign and malignant skin lesions.
Group B (Control) waited for 16 days before answering the final MCQ test with no intervention or education.
After the trail period has ended all participants were granted free access to the educational platform for 3 months.
Results
304 GPs and GP residents applied for more information about the study, 135 accepted the invitation and finished the initial questionnaire and MCQ and 101 participants were randomized to the intervention and 34 participants to the control group. 120 (89%) finished the final MCQ.
The intervention group participants spend on average spend 2,5 hours quizzing themselves in digital patient-cases and one hour reading the written educational modules during their 8 days of access to the mobile education app. They completed 466 cases and 55 participants (65 % of the 84) completed 500 digital patient-cases (as per protocol) or more. The time from initial to final MCQ was 21 and 19 days for the Intervention and Control group, respectively.
Intervention Group MCQ score on average improved by 27% from 6.3 to 7.9 (1.7 points, 95% CI 1.1 to 2.2, p<0.001) with Intention-To-Treat analysis and by 30% from 6.5 to 8.5 (2 points, 95% CI 1.3 to 2.6, p<0.001) with Per-Protocol analysis.
The control group did not improve their MCQ score after their waiting period of 16 days.
In the table below you can see what the different available diagnosis in the system was diagnosed as by the participants. As an explanatory example, the Superficial Spreading Melanomas 61.45% was diagnosed as melanoma.
| Actual Diagnosis | Doctors Diagnosis | ||||||||
| Group | True Diagnosis | Basal cell carcinoma | Dermatofibroma | Hemangioma | Melanoma | Nevus | Seb. keratosis/ Lentigo solaris | Squamous cell carcinoma | Grand Total |
| Benign | Blue nevus | 3.74% | 6.91% | 8.35% | 22.01% | 51.51% | 7.05% | 0.43% | 100.00% |
| Compound nevus | 2.41% | 5.66% | 1.05% | 25.91% | 48.39% | 15.87% | 0.71% | 100.00% | |
| Dermal nevus | 9.27% | 12.36% | 8.11% | 7.72% | 34.17% | 25.68% | 2.70% | 100.00% | |
| Dermatofibroma | 8.51% | 50.91% | 2.61% | 11.12% | 12.12% | 10.23% | 4.50% | 100.00% | |
| Hemangioma | 7.59% | 6.37% | 65.09% | 7.29% | 5.91% | 1.65% | 6.09% | 100.00% | |
| Junctional nevus | 0.84% | 2.51% | 0.42% | 27.23% | 56.31% | 12.36% | 0.33% | 100.00% | |
| Lentigo solaris | 4.97% | 5.21% | 17.75% | 17.57% | 52.27% | 2.24% | 100.00% | ||
| Seborrheic keratosis | 8.85% | 5.91% | 3.25% | 15.62% | 9.61% | 48.70% | 8.06% | 100.00% | |
| Spitz nevus | 3.08% | 19.49% | 6.15% | 24.10% | 43.59% | 3.59% | 100.00% | ||
| Benign Total | 6.28% | 13.53% | 12.72% | 16.64% | 23.10% | 23.45% | 4.29% | 100.00% | |
| Malignant | Basal cell carcinoma | 40.00% | 8.42% | 4.44% | 10.95% | 3.04% | 11.48% | 21.66% | 100.00% |
| Lentigo maligna | 9.25% | 1.78% | 0.71% | 38.08% | 7.83% | 39.15% | 3.20% | 100.00% | |
| Lentigo maligna melanoma | 5.80% | 1.45% | 56.52% | 11.59% | 21.74% | 2.90% | 100.00% | ||
| Melanoma in situ | 2.89% | 2.44% | 0.20% | 53.88% | 25.55% | 13.84% | 1.20% | 100.00% | |
| Nodular melanoma | 10.94% | 0.94% | 20.31% | 35.94% | 16.56% | 4.06% | 11.25% | 100.00% | |
| Squamous cell carcinoma | 27.27% | 6.42% | 2.63% | 2.80% | 0.54% | 5.00% | 55.34% | 100.00% | |
| Superficial spreading melanoma | 5.65% | 3.80% | 1.95% | 61.45% | 11.93% | 12.29% | 2.93% | 100.00% | |
| Malignant Total | 19.64% | 5.35% | 2.86% | 32.06% | 8.53% | 10.85% | 20.72% | 100.00% | |
| Grand Total | 11.63% | 10.25% | 8.77% | 22.82% | 17.26% | 18.40% | 10.87% | 100.00% |
They diagnosed 40484 patient-cases in the mobile education app during the study period. In-system diagnostic accuracy increased 11% from 42% to 53% on diagnosis level accuracy and around 6% from 67% to 73% on malignant/benign level, with the majority of the increase happening over the first 100 digital patient-cases.
Conclusion
Dermoscopy is commonly used among Danish GPs, but formal training in use of dermoscopy is not. Digital patient-case based educational smartphone applications can improve doctors’ diagnostic accuracy on skin and mole cancer.
Seborrheic keratosis is the most commonly misdiagnosed benign skin lesion and melanoma is the most commonly misdiagnosed malignant skin lesion in our case-library.
Whats next?
The next study will focus on the clinical implications of improving clinicians’ diagnostic accuracy. GPs, Dermatologists, Plastic Surgeons and Pathologists.
Can we avoid spending time and resources on benign lesions? Can we find malignancy earlier?
Follow our Twitter for updates on our current research projects.
Gustav Gede Nervil
MD, Ph.D. student
Gustav is full time researcher and Ph.D. student in the AISC consortium, and focuses on the clinical validation and implementation of the educational platform.
“This is the first trial that validates the use of simulation-based training on doctors’ ability to correctly diagnose skin and mole cancer.
I’m quite excited!“
Gustav Gede Nervil
Contact Us
Email:
contact@aiscresearch.com
Phone:
+45 3868 9236
Find Us
The Research Unit at the Department of Plastic Surgery
Ground Floor, Entrance 7
Herlev Hospital
Borgmester Ib Juuls Vej 7
2730 Herlev
Denmark