While the potential of artificial intelligence (AI) to revolutionize healthcare is undeniable, concerns regarding data bias, transparency, and accountability demand careful consideration [1]. The rapid evolution of AI, particularly in healthcare, necessitates critically examining its ethical implications and the measures needed to ensure its safe and responsible application. As Dr. Kim addressed in his article in a timely manner [2], the importance of patient and public involvement and engagement (PPIE) in healthcare AI is increasingly recognized.
While participatory design, healthcare AI literacy, and citizen science models offer promising pathways for PPIE, it is crucial to acknowledge the challenges and potential risks associated with increased public involvement in the development and application of healthcare AI: Data privacy and security concerns need to be carefully addressed to prevent data breaches and misuse. In addition, the potential for the blurring of boundaries between professional healthcare practice and public involvement requires clear guidelines and regulatory frameworks.
This is particularly relevant in the context of external validation, a key component of ethical frameworks for healthcare AI. The dominant paradigm in AI, particularly with large language models (LLMs), relies heavily on machine learning trained on vast amounts of real-world data (RWD). The inherent limitations of RWD, often riddled with inaccuracies, prejudices, and misunderstandings originating from the general public and non-professionals, pose significant challenges. As the volume of training data grows exponentially, manually scrutinizing and validating this information becomes increasingly challenging and impractical. This commentary advocates for the integration of a knowledge-based approach, specifically through the use of ontologies, as a crucial step towards refining RWD and mitigating these concerns.
Ontologies, as structured representations of domain knowledge, offer a powerful tool for verifying and validating the information contained in RWD. This approach leverages the expertise of domain experts or interested parties to ensure the accuracy and reliability of the knowledge base [3]. The automated reasoning capabilities of ontologies enable the identification and rectification of inconsistencies, errors, and biases within RWD. One of the key advantages of ontologies is their ability to enable automated verification and checking of knowledge in RWD. By defining relationships, constraints, and rules within a domain, ontologies can help identify inconsistencies, errors, and biases in large datasets. This automated process can significantly reduce human bias and error in the data used to train AI systems.
The integration of machine learning-based approaches and knowledge-based approaches presents a powerful strategy for developing safer and more accurate AI systems. This harmonization allows for enhanced data quality, with ontologies acting as a filter, improving the quality of data used in machine learning models. Knowledge-based systems can also provide explanations for AI decisions, addressing the “black box” problem often associated with complex machine learning models. By incorporating expert knowledge, AI systems can make more accurate predictions and decisions in specialized fields, identifying and mitigating biases present in RWD.
The synergy between machine learning and knowledge-based approaches has shown promising results in various domains. In medical diagnosis, machine learning models trained on large datasets can be complemented by ontologies that encode medical knowledge, leading to more accurate and explainable diagnostic systems [4]. In finance, fraud detection systems can benefit from the combination of pattern recognition capabilities of machine learning and the rule-based logic of knowledge-based systems [5].
The citizen science model of design can be incorporated when developing ontologies. Consumer health ontology is a structured system that helps consumers make informed decisions about their health by providing consumer-level meaning to health information [6]. It helps consumers understand and act on health information by providing a formal dataset that can be linked to other resources. Ontology of Consumer Health Vocabulary, one of these accomplishments, helps people describe their conditions and search the internet using medical nomenclature [7]. When developing the ontology, consumer terms from social media, discussion forums, and patient diaries were gathered to address the imbalance of domain knowledge between professionals and laypersons, and social network mining was used to expand the terminology [8].
The harmonization of machine learning and knowledge-based approaches in AI represents a crucial step towards safer and more reliable AI. By leveraging the strengths of both methodologies, we can create AI systems that are not only powerful but also trustworthy and aligned with human expertise. The participation of citizens and patients in developing ontologies can further ensure inclusive models. As we continue to advance in this field, it is essential to prioritize this integrated approach to ensure that AI technologies benefit society while minimizing potential risks and biases.
