Journal of International Society of Preventive and Community Dentistry

: 2019  |  Volume : 9  |  Issue : 5  |  Page : 464--469

Psychometric analysis of the european medical risk-related history questionnaire within Indonesian-speaking population

Yuniardini Septorini Wimardhani1, Anzany Tania Dwi Putri Baringbing2, Gus Permana Subita3, Luzi Abraham-Inpijn4,  
1 Department of Oral Medicine, Faculty of Dentistry, Universitas Indonesia; Cluster of Epidemiology and Clinical Studies in Dentistry, Faculty of Dentistry, Universitas Indonesia; Center of Ageing Studies, Universitas Indonesia, Jakarta, Indonesia
2 Oral Medicine Residency Program, Faculty of Dentistry, Universitas Indonesia; Drug Rehabilitation Center, National Narcotic Board of Republic of Indonesia, Jakarta, Indonesia
3 Department of Oral Medicine, Faculty of Dentistry, Universitas Indonesia, Jakarta, Indonesia
4 Department of Internal Medicine, Academic Medical Centre, University of Amsterdam, Amsterdam, The Netherlands

Correspondence Address:
Dr. Yuniardini Septorini Wimardhani
Department of Oral Medicine, Faculty of Dentistry, Universitas Indonesia, 2nd Floor A Building, Salemba Campus, Jl. Salemba Raya No. 4, Jakarta 10430.


Objectives: Medical complications during dental treatment are increasingly anticipated because advances in medical treatment have prolonged life expectancy. Therefore, a thorough analysis of medical history data to assess the medical risks that may occur before, during, and after dental procedures is required. The European Medical Risk-Related History (EMRRH) questionnaire has been used in 10 European countries to detect medical problems and determine the risks of complications. However, no similar questionnaire has been developed for Indonesian patients. This study aimed to develop an Indonesian version of the EMRRH questionnaire and analyze its psychometric properties. Materials and Methods: The EMRRH questionnaire was utilized and processed for cross-cultural adaptation according to the Beaton guidelines. The final version was used after completing all of the steps of cross-cultural adaptation. The psychometric property analysis was performed by measuring the validity, reliability, sensitivity, and specificity of the questionnaire in 172 dental patients using cross-sectional survey at the National Drug Rehabilitation Center. Results: The content validity was first measured by an internist and anesthesiologist, and the coefficient was 0.91. The construct validity assessment revealed significant associations for five of six global questions (P < 0.05). Cronbach’s alpha coefficient for internal consistency was 0.790. The test–retest reliability results were excellent based on reevaluation in 17 patients (intraclass correlation coefficient = 0.846), including sensitivity and specificity values of 69.31% and 92.2%, respectively. Conclusion: The Indonesian version of the EMRRH questionnaire is valid, reliable, sensitive, and specific for this population. Further study to explore its use in larger Indonesian populations is needed.

How to cite this article:
Wimardhani YS, Baringbing AT, Subita GP, Abraham-Inpijn L. Psychometric analysis of the european medical risk-related history questionnaire within Indonesian-speaking population.J Int Soc Prevent Communit Dent 2019;9:464-469

How to cite this URL:
Wimardhani YS, Baringbing AT, Subita GP, Abraham-Inpijn L. Psychometric analysis of the european medical risk-related history questionnaire within Indonesian-speaking population. J Int Soc Prevent Communit Dent [serial online] 2019 [cited 2021 Nov 30 ];9:464-469
Available from:

Full Text


The demand for patients to undergo invasive dental treatment, for example, implantology or periodontal surgery, has increased in recent years. Some patients have at least one systemic disease, increasing the risk of medical emergencies in dental practice.[1],[2] A study by  Coplan and Curson in UK uncovered 120 deaths in dental practices over the last 10 years.[3] Similarly, in the Netherlands, of 288 dentists examined, 208 emergencies were reported over a 1-year period, including sudden death due to myocardial infarction.[4]

According to Indonesia Basic Health Research 2013 published by the country’s Ministry of Health, individuals with systemic diseases can be divided into two major groups: infectious and noninfectious diseases. In the infectious disease group, the highest rank of illness was acute respiratory infection, followed by hepatitis, diarrhea, and malaria. In the noninfectious disease group, asthma was the most prevalent disease, followed by malignancy, diabetes mellitus, hyperthyroid, hypertension, coronary heart disease, heart failure, kidney disease, and rheumatic diseases.

Based on this fact, dentists in Indonesia must identify medical risks patients could face before, during, and after dental procedures by comprehensively reviewing their medical histories.[5],[6] This is necessary to determine whether the dental treatment plan should be modified to prevent further complications.[1],[7],[8] The most widely used classification for assessing medical risk is the American Society of Anesthesiology (ASA) physical status classification. Although it was more commonly used for patients undergoing anesthesia, it has also been used for patients undergoing various dental procedures to assess potential medical risks because patients with chronic or complex conditions can appear physically healthy.[9],[10] ASA scores are commonly obtained through self-administered questionnaires; thus, developing questionnaires that can accurately predict risk has recently become a focus of study in many countries.[11]

In Europe, Abraham-Inpijn et al. designed the European Medical Risk-Related History (EMRRH) questionnaire that has been validated for use in dental practice not only for detecting medical variables and conditions that can possibly lead to medical emergencies, including cardiovascular problems, gastrointestinal disorders, hepatitis, asthma, epilepsy, pulmonary disturbances, renal problems, and medication use, but also clarifying the degree of risk, including preventive measures, by determining ASA scores. The questionnaire has been translated into ten languages (Dutch, English, French, Spanish, Swedish, Hungarian, German, Turkish, Arabic, and Greece) to date, and its validity and reliability have been confirmed.[7],[12],[13],[14],[15],[16]

Thus, this study aimed to develop an Indonesian version of the EMRRH questionnaire and analyze its psychometric properties, including its validity, reliability, sensitivity, and specificity, because no standardized questionnaires have been developed for screening medical risks in dental practice in Indonesia.

 Materials and Methods

Cross adaptation of the European Medical Risk-Related History questionnaire

The original English version of the EMRRH questionnaire was obtained from a previous publication.[12] The questionnaire was translated into Indonesian using the Beaton guidelines for cross-cultural adaptation.[17] First, the EMRRH questionnaire was initially translated separately by two translators with a medical (T1) or nonmedical (T2) background. These two versions were then synthesized into one common translation (T-12). Working from the T-12 version, a separate translator with a nonmedical background translated the questionnaire back into English (BT) to ensure that the translated version matched the original questionnaire.

An expert panel consisting of an internist, anesthesiologist, oral medicine specialist, and public health professional then reviewed and revised the translations (T1, T2, T-12, BT) regarding item equivalence between the original and Indonesian versions to develop a consensus version. The Indonesian consensus version was pilot-tested via administration to 46 dental patients to probe their interpretations of each item in the questionnaire. No patients cited any meaningful ambiguities. Therefore, the final version of the Indonesian version of the EMRRH questionnaire was ready for use. After completing cross-cultural adaptation, psychometric property analysis was conducted by evaluating validity, reliability, sensitivity, and specificity.

This was a cross-sectional study to test the EMRRH Indonesian version to 182 dental patients consisting of recovering drug users who were undergoing rehabilitation program in the Drug Rehabilitation Center, National Narcotics Board, Republic of Indonesia. They were the residents of the Rehabilitation Center during the period January–February 2018. The inclusion criteria were as follows: being individuals aged at least 18 years, completing all standard medical tests in the Drug Rehabilitation Center for evaluating medical conditions within the last 6 months, able to read and write independently, and willing to participate in the study by providing written informed consent. The exclusion criteria included individuals who had a history of severe mental disorders and were being undersupervised by medical team. There were 10 participants were excluded in the study because of their unstable mental condition. Each respondent was self-administered the Indonesian version of the EMRRH questionnaire and assured that the provided data were only used for statistical analysis. This study was approved by the Ethics Committee of the Faculty of Dentistry, Universitas Indonesia, No. 140/Ethical Approval/FKGUI/XII/2017, with Protocol No. 051481217.

The EMRRH questionnaire consisted of 22 items assessing various systemic conditions related to medical emergencies, with each question having a yes/no answer. Some items had primary and secondary questions, with the main question printed in bold. The answer to each item was recorded on the basis of the modified-ASA score, an ASA Physical Status scale that has been modified according to dental needs, which ranged from 1 to 4 based on the patient’s reply. The highest ASA score of the questionnaire was recorded for each patient. In addition, demographic data, including age, gender, province of residence, occupation, and educational background, and the responses to six general questions (“are you satisfied with your health,” “have you ever had any problems or complications during surgical or dental procedures,” “have you ever had any adverse reactions due to the use of certain drugs,” “have you visited a general practitioner or a specialist during the past year,” “have you noticed any changes in your health status recently,” and “has your physician recently made any modifications to the drugs you take”) were also recorded.

After the questionnaire was fully completed, the answers provided by the patients were reviewed by two physicians with at least 5 years of experience via comparisons to their medical test results over the last 6 months. The evaluation criteria were as follows:

True positive: The ASA score was 2–4 and the physician examination and test results confirmed that the patient was “not healthy”

False positive: The ASA score was 2–4 but the physician examination and test results confirmed that the patient was “healthy”

True negative: The ASA score was 1 and the physician examination and test results confirmed that the patient was “healthy”

False negative: The ASA score was 1 but the physician examination and test results confirmed that the patient was “not healthy.”

Psychometric analysis of the European Medical Risk-Related History questionnaire

To ensure reproducibility, the questionnaire was resubmitted to 17 patients, 7–14 days after the first administration. The number fulfilled the requirement of 5%–10% of patients for the reproducibility testing. Reliability was tested using Cronbach’s alpha, the intra- and inter-physician kappa, and the intraclass correlation coefficient (ICC). Content validity was assessed by an experienced internist and anesthesiologist; construct validity was examined by comparing the total score of the Indonesian version of the EMRRH questionnaire for six general questions, and discriminant validity was confirmed by comparing the score of each item in the EMRRH questionnaire regarding the presence of complications during surgical or dental procedures.


In total, 172 dental patients completed all items of the Indonesian version of the EMRRH questionnaire, and no items were excluded from the data analysis due to missing data. The mean patient age was 27.66 ± 7.22 years (range, 17–50 years), including ages of 27.47 ± 6.94 and 29.28 ± 9.40 years for 154 males and 18 females patients, respectively.

The mean Cronbach’s alpha for the reliability of the EMRRH questionnaire was 0.79, with a range of 0.77–0.79 for various questions, indicating acceptable internal consistency [Table 1]. The inter- and intra-physician kappa values were 0.86 and 1, respectively, indicating almost perfect agreement. The ICC was 0.85, indicating excellent reproducibility.[18]{Table 1}

The coefficient for content validity was determined using the viewpoints of internist (I) and anesthesiologist (A) by dividing the answers of each item into 2 × 2 table: low relevance (lr) and high relevance (hr), resulting a score of 0.91 that indicated good agreement in each items of this questionnaire.[19] The assessment of construct validity illustrated that the total score of the Indonesia version of the EMRRH questionnaire was significantly different for five of six general questions based on the binary response [Table 2].{Table 2}

The discriminant validity analysis illustrated that the 21 patients who experienced complications during surgical or dental procedures had a significantly higher total EMRRH questionnaire score than the 152 patients with no histories of complications [Table 3].{Table 3}

[Table 4] shows the true-positive, false-positive, true-negative, and false-negative results together with data for sensitivity and specificity for each of the questions of the questionnaire. The overall sensitivity and specificity were 69.31% and 92.20%, respectively.{Table 4}


Medical complications during dental treatment are increasingly expected because life expectancy has been prolonged through medication use and fewer patients with systemic conditions require hospitalization. Consequently, it has become increasing difficult for dental practitioners to differentiate healthy and unhealthy patients.[1],[9],[16] The EMRRH questionnaire is used to identify systemic conditions before dental procedures. Through this questionnaire, a dental practitioner can identify systemic diseases in patients and assess their risks based on the ASA score.[12],[13],[14]

In Indonesia, patients’ medical histories and risks are usually obtained through verbal confirmation before anesthesia, but this process highly depends on the dental practitioner’s ability to recognize and identify medical problems and histories of drug use as well as patients’ willingness to provide truthful answers.[20] Therefore, a standardized instrument for this purpose has been desired to avoid medical emergencies in dental practice. This study was the first study to explore the usage of standardized instrument to assess possible systemic condition and medical risks in dental patients in a sample of Indonesian population.

In this study, the EMRRH questionnaire was cross-culturally adapted to the Indonesian language, and its psychometric properties were confirmed through evaluation of its validity, reliability, sensitivity, and specificity. One important consideration in this evaluation was that patients’ medical histories were reviewed by an experienced physician and then cross-checked with their medical test results within the past 6 months, which is considered the gold standard method.[21] The addition of cross-checking medical test results become a strength of this study because most respondents involved in the study were recovering drug users with varied mental health statuses, which could affect the truthfulness of their responses.[22],[23]

The results of cross-cultural adaptation that were performed in this study indicated that the Indonesian and English versions of the EMRRH questionnaire were semantically similar. Moreover, the validity and reliability of the Indonesian version were also confirmed and it showed valid and reliable. The reliability of internal and test–retest consistency was established with acceptable result but excellent reproducibility. Regarding validity, not all general questions were associated. It was lack of a significant association for the question “have you ever had any adverse reactions due to the use of certain drugs” based on the response. This may be attributable to the histories of drug use among the respondents, who may have confused their adverse reactions to medications with those to illicit drugs.

The sensitivity of the Indonesian version of the EMRRH questionnaire was 69.3%. The value was lower than the sensitivity test of the questionnaire in other countries. In 10 countries in Europe, the mean sensitivity was 93.50% (range, 87%–98%).[12] Similarly, in a Persian-speaking population, the overall value was 94%.[11] The lower sensitivity value may be related to differences in the confirmatory methods used between Indonesia and other countries. Specifically, the patients’ medical histories were crosschecked by a physician, whereas in other countries, only verbal confirmation by a physician was performed. The results could also be attributable to differences in the cognitive function status of respondents.[21],[22] We know that drug dependence both affects the systemic and mental status of patients and decreases cognitive function, which could affect respondents’ abilities to understand each question in this questionnaire.[24] However, the specificity of the Indonesian version was 92.20% that was quite similar to other studies. Therefore, both results show that the Indonesian version of the EMRRH questionnaire has sufficient sensitivity and specificity for wider use.

The limitations of the study include short age range, drug dependence background, and modest amounts of participant. This could be controversy because it potentially limits the generalization of the results to overall population in Indonesia. However, the result could still be beneficial as a pilot study for developing a tool to measure patient’s medical risk before all dental treatments in Indonesia. At this stage, there was no available systematic reviews related to the medical risk assessment of dental patients using EMRRH that could be referred to be compared with the results of the study.

As a conclusion, we produced an Indonesian version of the EMRRH questionnaire using Beaton’s cross-cultural adaptation guidelines. Psychometric property analysis confirmed that this version of the questionnaire has sufficient validity, reliability, sensitivity, and specificity for use in Indonesia. More collaborative studies with researchers in many dental healthcare providers could add more values of the usage of this questionnaire. The results would be the basis before it could be administered as standard procedure for dental patients in Indonesian general population.

Financial support and sponsorship

Universitas Indonesia Research Grant 2018.

Conflicts of interest

There are no conflicts of interest.


1Monteserín-Matesanz M, Esparza-Gómez GC, García-Chías B, Gasco-García C, Cerero-Lapiedra R. Descriptive study of the patients treated at the clinic “integrated dentistry for patients with special needs” at complutense university of Madrid (2003-2012). Med Oral Patol Oral Cir Bucal 2015;20:e211-7.
2Vaughan M, Park A, Sholapurkar A, Esterman A. Medical emergencies in dental practice – Management requirements and international practitioner proficiency. A scoping review. Aust Dent J 2018;63:455-66.
3Coplans MP, Curson I. Deaths associated with dentistry. Br Dent J 1982;153:357-62.
4de Jong KJ, Oosting J, Peters GJ, Abraham-Inpijn L. Detecting medical problems in dentistry: A survey of 4,087 patients in The Netherlands. Eur J Med 1992;1:23-9.
5Al-Hassan M, AlQahtani S. Preparedness of dental clinics for medical emergencies in Riyadh, Saudi Arabia. Saudi Dent J 2019;31:115-21.
6Albelaihi HF, Alweneen AI, Ettish A, Alshahrani FA. Knowledge, attitude, and perceived confidence in the management of medical emergencies in the dental office: A survey among the dental students and interns. J Int Soc Prev Community Dent 2017;7:364-9.
7Chandler-Gutiérrez L, Martínez-Sahuquillo A, Bullón-Fernández P. Evaluation of medical risk in dental practice through using the EMRRH questionnaire. Med Oral 2004;9:309-20.
8Shetty A, Nagaraj T, Santosh HN, Shetty N. Determination of medical condition afflicting patients visiting dental clinics: A population-based survey study. J Adv Clin Res Insights 2015;2:62-6.
9José H, Esteves M, Suảrez M. Identification of medically compromised dental. Oral Health Prev Dent 2013;11:315-22.
10Clough S, Shehabi Z, Morgan C. Medical risk assessment in dentistry: Use of the American Society of Anesthesiologists physical status classification. Br Dent J 2016;220:103-8.
11Maloney WJ, Weinberg MA. Implementation of the American Society of Anesthesiologists physical status classification system in periodontal practice. J Periodontol 2008;79:1124-6.
12Abraham-Inpijn L, Russell G, Abraham DA, Bäckman N, Baum E, Bullón-Fernández P, et al. A patient-administered medical risk related history questionnaire (EMRRH) for use in 10 European countries (multicenter trial). Oral Surg Oral Med Oral Pathol Oral Radiol Endod 2008;105:597-605.
13de Jong KJ, Borgmeijer-Hoelen A, Abraham-Inpijn L. Validity of a risk-related patient-administered medical questionnaire for dental patients. Oral Surg Oral Med Oral Pathol Oral Radiol J 1991;72:527-33.
14de Jong KJ, Abraham-Inpijn L, Vinckier F, Declerck D. The validity of a medical risk-related history for dental patients in Belgium. Int Dent J 1997;47:16-20.
15Maryam A, Atessa P, Mozafari Pegah M, Zahra S, Hanieh G, Davood A, et al. Medical risk assessment in patients referred to dental clinics, Mashhad, Iran (2011-2012). Open Dent J 2015;9:420-5.
16Pooyafared A, Hashemipour MA, Baharloey K, Shafiei L, Montajeb F. Standardization of European medical risk related history questionnaire for use with Persian-speaking population. J Educ Health Promot 2015;4:41.
17Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976) 2000;25:3186-91.
18Kimberlin CL, Winterstein AG. Validity and reliability of measurement instruments used in research. Am J Health Syst Pharm 2008;65:2276-84.
19Zamanzadeh V, Ghahramanian A, Rassouli M, Abbaszadeh A, Alavi-Majd H, Nikanfar AR, et al. Design and implementation content validity study: Development of an instrument for measuring patient-centered communication. J Caring Sci 2015;4:165-78.
20Al Hibshi SM, Al-Raddadi RM, Assery MK. Discrepancies between dental and medical records of cardiac patients in alHada Armed forces hospital, Taif, Saudi Arabia. J Int Soc Prev Community Dent 2016;6:568-74.
21Ramos LW, Souza CF, Dias IW, Oliveira RG, Cristina B, Calil M, et al. Validity time of normal results of preoperative tests for surgical reintervention and the impact on postoperative outcomes. Rev Bras Anestesiol 2018;68:154-61.
22Squeglia LM, Jacobus J, Tapert SF. The influence of substance use on adolescent brain development. Clin EEG Neurosci 2009;40:31-8.
23Carey B, Stassen L. An audit comparing the discrepancies between a verbal enquiry, a written history, and an electronic medical history questionnaire: A suggested medical history/social history form for clinical practice. J Ir Dent Assoc 2011;57:54-9.
24Chukhlovina ML. Cognitive impairments in psychoactive drug addicts. Zh Nevrol Psikhiatr Im S S Korsakova 2015;115:83-5.