Quantifying device type and handedness biases in a remote Parkinson’s disease AI-powered assessment

In recent years, the use of artificial intelligence (AI) in healthcare has significantly advanced, especially in remote assessments of neurological conditions like Parkinson’s Disease (PD). However, one critical aspect that researchers must address is the potential for biases related to device type and handedness during these assessments. In this article, we will explore the implications of such biases, the methods used to quantify them, and the study design implemented in one recent research effort aimed at assessing PD remotely.

Study Design and Participant Recruitment

The study focused on recruiting participants aged 50 and older, targeting both individuals diagnosed with PD and healthy controls. This diverse recruitment strategy aimed to gather a comprehensive dataset and was facilitated through partnerships with local PD support organizations. Notably, recruitment efforts were also enhanced by participation in notable events like the Hawaii Parkinson’s Association Symposiums in 2023 and 2024.

Participants accessed the study through a web application which provided detailed information about the research. Consent was obtained electronically, ensuring compliance with ethical standards as mandated by the University of Hawaii Institutional Review Board. This systematic approach helped to create an inclusive environment for various demographics, which is essential when evaluating the impact of factors like handedness and device type.

Data Collection Procedure

The assessment comprised a series of digital tasks assessing motor control and working memory. Participants utilized a mouse or keyboard to complete these tasks, which included:

1. Mouse-based Assessments: Tasks such as straight-line and curved-line tracing evaluated hand stability, precision, and coordination.
2. Keyboard-based Assessments: These involved single-letter pressing to measure speed and accuracy, and tasks requiring participants to replicate random sequences to assess cognitive flexibility.

Dealing with Biases

A crucial aspect of this research was recognizing and quantifying biases linked to device type, handedness, race, and sex. The study initially revealed demographic imbalances, notably in racial representation, with a predominance of white participants. Efforts were made to balance these demographics using techniques like synthetic minority over-sampling (SMOTE) to enhance sensitivity in detecting PD through the AI models.

Technical Approaches

To address the issues of dataset imbalance and enhance model performance, the study employed several technical strategies:

• SMOTE: It increased representation among PD diagnosed participants, ultimately improving model sensitivity, which is critically important in healthcare applications where early detection is vital.
• Bootstrapping: This was used to ensure that each sampled dataset contained balanced representation across demographic categories during model evaluation.

Evaluation Metrics

The study adopted several performance and fairness metrics to ensure models were both accurate and equitable. Key metrics included:

• F1 Score: This metric combines precision and recall, presenting a holistic view of model performance.
• Sensitivity and Specificity: These metrics help determine the model’s ability to correctly identify PD cases versus healthy controls.

Fairness metrics like Demographic Index (DI) and Equal Opportunity (EO) were also employed to monitor disparities across different demographic groups. An ideal DI value of 1 would indicate that the model treats all groups equally, while values deviating from this may flag potential biases.

Importance of Understanding Device Biases

Understanding biases related to device type is particularly crucial in the context of remote assessments:

1. Device Variability: Different devices may exhibit variability in input sensitivity, screen size, and user interface layouts, which can lead to differences in assessment performance. For example, users of touchscreen devices might display different fine motor skills compared to those using traditional mouse-and-keyboard setups.

2. Handedness Implications: The dominant hand used during assessments can impact performance and efficacy in tasks requiring precision and coordination, further complicating analyses.

Conclusions and Future Directions

Addressing device type and handedness biases in remote assessments for PD is paramount for creating a more equitable AI-powered health assessment framework. By carefully designing studies to account for these factors and employing rigorous analytical approaches, researchers can enhance the accuracy of AI models, ultimately leading to improved diagnostic modalities for neurological conditions.

As healthcare technology progresses toward a more personalized framework, future studies should continue to refine techniques for mitigating biases. This requires ongoing collaborations across diverse demographic groups, innovative technical adjustments, and continuous evaluation of the performance of AI models in real-world applications.

In conclusion, while AI offers groundbreaking potential in remote health assessments, precautions must be taken to ensure these systems are equitable and accurate across diverse populations. This ongoing commitment to fairness and inclusivity will be essential in leveraging AI to enhance health outcomes for individuals living with Parkinson’s Disease and other neurological conditions.

Source link