2Department of Medical Informatics, Başkent University Faculty of Medicine, Ankara-Türkiye DOI : 10.5505/tjo.2025.4563 OBJECTIVE
Cancer drug reimbursement policies play a crucial role in regulating access to novel therapies. In Türkiye, the Social Security Institution's Health Practice Regulation (SUT) defines the reimbursement criteria for cancer treatments. Despite frequent updates, oncologists face challenges in interpreting and applying these regulations. Large language models (LLMs) have shown potential in processing complex medical texts, yet their utility in regulatory compliance remains underexplored.
METHODS
We evaluated the effectiveness of GOOGLE Gemini 2.5 Pro Preview 03-25 in analyzing drug reimbursement
policies within the SUT. A structured prompt was developed to ensure responses
strictly adhered to regulatory text. A total of 80 oncology-related test questions, covering multiple
cancer types, were used to assess the model"s accuracy. Responses were categorized as correct and
complete, correct but incomplete, or incorrect. Performance metrics, including precision, recall,
and F1-score, were calculated. An iterative prompt engineering process was employed to optimize
model performance.
RESULTS
The LLM provided completely correct responses to 77 (96.3%) of 80 test cases and correct but incomplete
responses to 3 (3.7%), with no incorrect answers. Performance metrics demonstrated high accuracy
(precision: 1.00, recall: 0.96, F1-score: 0.98). The model successfully processed medical terminology
variations but showed limitations in synthesizing implicit reimbursement rules.
CONCLUSION
LLMs demonstrate strong potential for interpreting cancer drug reimbursement regulations, reducing
administrative burden for oncologists. Future refinements should address inference limitations to enhance
regulatory compliance support in clinical practice.