How to Minimize Subjectivity in Dermatology – Part 1: Principal Investigators & Clinical Staff

The success of any clinical trial depends on unbiased data. However, the subjective nature of many dermatology assessments makes unbiased data harder to come by. Inconsistent reports based on subjective findings can skew data, which may obscure the true effects of treatment.

In-depth training of investigators and staff helps minimize subjectivity, but even traditional training can fall short. Here are a few solutions.

Objective measurements of subjective characteristics

Physicians measure certain indications and treatment responses with established scales such as the Psoriasis Area and Severity Index (PASI), the Eczema Area and Severity Index (EASI), and the SCORing Atopic Dermatitis (SCORAD). These tests allow for subjective interpretation and some standardization of results, but they still lack objectivity. Without highly specific training, measuring characteristics such as redness, which physicians assess visually, becomes biased.

Subjective sensations, such as itching and pain, are even more challenging to measure objectively, as each patient perceives sensations differently. Take itching, for example.

Itch intensity scales (NRS, VRS, and VAS, for example) provide simple, reliable, and valid itch-intensity measurements that researchers have used in large-scale clinical trials. Scratching activity, an objective measurement, can also help evaluate itch. To more holistically measure the impact of chronic itching, investigators may combine these scales with the Dermatology Life Quality Index (DLQI) or the ItchyQoL survey.

Even more clearly defined characteristics run the risk of subjective interpretation. For example, one dermatologist may count 20 pustules on a patient’s face, while another may count 30. The two physicians have different definitions of what constitutes a pustule for purposes of the study.

Mitigating Subjectivity of Assessments in Dermatology Trials

The need for quality training

To ensure all physicians involved in a study operate under the same parameters, a CRO must conduct ongoing, study-specific training which goes beyond a single two- to three-hour session.

Investigators should consider meeting in person with medical monitors to discuss the study and how to assess certain indications. The medical monitor can also share photos and discuss examples of specific indications with physicians.

As new-hires join the study, it’s important they receive the same training as existing staff. If a new study assistant starts working with physicians without the same precise training, their work could cause data anomalies. And considering clinical trials continue for years, study sites should periodically retrain all staff. To ensure study physicians stay on the same page, Biorasi uses continual training throughout the course of a study.

The benefit of early intervention

If one study site shows significantly higher or lower scores for a particular assessment than others, there may be a scoring issue. Biorasi uses big data-derived data science processes to identify possible anomalies in rating issues early on in the trial. After analyzing the data and meeting with the principal investigator, we attempt to uncover the source of the anomaly and correct it when possible.

New frontiers in minimizing subjectivity

Artificial intelligence and other technology and tools help limit subjectivity, or turn subjective data into objective data. For example, redness is one parameter in PASI that lends itself to subjective results when measuring the severity of a psoriasis lesion. Researchers found that by using a chromameter to first obtain the color of lesions, doctors can more objectively assess redness for PASI scoring.¹

New technology has the potential to help investigators and physicians more objectively assess dermatological indications. In 2017, Stanford researchers developed a diagnosis algorithm using 130,000 skin disease images to diagnose potential skin cancer. The algorithm came within 91 percent of matching the diagnosis of 21 dermatologists. In time, we may also have an algorithm to measure surface area or indications associated with eczema, psoriasis, acne, and other conditions.²

The Center for Human Drug Research, in the Netherlands, developed a tool that measures subjective symptoms using objective measurements and instruments. Called DermaToolbox, the tool uses conventional clinical photography and 3D photography to measure lesions. It also uses measurements such as transepidermal water loss, lipid profiling, colorimetry, and laser Doppler imaging to measure a treatment’s effects on the skin.

Specific, ongoing training and regular data monitoring help to minimize subjectivity in dermatology studies. As technology advances and becomes more widespread among study sites, we’ll have even more tools to help deliver objective results.

To learn more about Biorasi’s data-driven approach, talk to one of our experts.

References

1. M. H. Ahmad Fadzil, Dani Ihtatho, Azura Mohd Affandi & S. H. Hussein (2009) Objective assessment of psoriasis erythema for PASI scoring, Journal of Medical Engineering & Technology, 33:7, 516-524, DOI: 10.1080/07434610902744074

2. Esteva, Andre, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature volume542, pages115–118 (02 February 2017)