AI-based products, particularly in medical diagnosis, have become increasingly popular. However, with the rise of AI technologies, there is a critical need for quality assurance and risk assessment to ensure the reliability and impartiality of these systems. One crucial application of AI in the medical field is the diagnosis of diseases through imaging techniques, such as chest X-rays. Chest X-rays are commonly used by physicians to diagnose respiratory diseases quickly and cost-effectively. Yet, interpreting chest X-rays can be challenging, and errors in diagnosis can have severe consequences, especially for life-threatening conditions like pneumonia. Given the high mortality rate associated with pneumonia, accurate and timely diagnosis is essential. It is vital to prioritize quality assurance and risk assessment in the development and implementation of AI-based products, particularly in critical areas like medical diagnosis.
In this paper, we utilized a deep CNN network to diagnose pneumonia from chest X-ray images. We also introduced two criteria, bias and transparency, to evaluate these products. For assessing these criteria, we provided methods based on checklists and quantitative assessment approaches for our data. We successfully implemented these solutions on our data and even achieved a robust model by applying data augmentation techniques, raising accuracy above 90 percent. Additionally, to validate our data, we used two tests: the pressure test and the crystal test, which yielded an accuracy of over 70 percent. We also completed all the checklist-based methods and were able to obtain validation for these data in medical products.