Home | About us | Editorial board | Search | Ahead of print | Current issue | Archives | Submit article | Instructions| Reviewers

  Home Print this page Email this page Small font sizeDefault font sizeIncrease font size Users Online: 6182    

Table of Contents   
Year : 2022  |  Volume : 12  |  Issue : 2  |  Page : 210-215
The implementation of PROMs/PREMs in the assessment of orthodontic treatment outcomes: A questionnaire survey

1 Department of Orthodontics, Faculty of Dentistry, Mahidol University, Bangkok, Thailand
2 Faculty of Dentistry, Mahidol University, Bangkok, Thailand

Date of Submission23-Sep-2021
Date of Decision14-Oct-2021
Date of Acceptance12-Nov-2021
Date of Web Publication07-Apr-2022

Correspondence Address:
Dr. Kawin Sipiyaruk
Department of Orthodontics, Faculty of Dentistry, Mahidol University, 6 Yothi Road, Ratchathewi, Bangkok 10400.
Login to access the Email id

Source of Support: None, Conflict of Interest: None

DOI: 10.4103/jispcd.JISPCD_275_21

Rights and Permissions


Objectives: The aim of this study was to explore the impact of self-perceived treatment outcomes and experiences on psychological aspects, reflecting the importance of patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) in orthodontic practice. Materials and Methods: A questionnaire was constructed using the PROMs/PREMs concepts. It contained five parts: “Demographic data”, “Self-perceived esthetics”, “Self-perceived functional concern”, “Satisfaction of treatment procedures”, and “Psychological aspect”. The questionnaire was piloted in five experts and 50 orthodontic patients to ensure its quality, using content validity, test-retest reliability, and Cronbach’s alpha. The validated version was provided to patients who had a complete fixed appliance orthodontic treatment. These data were analyzed using descriptive statistics, Mann-Whitney U test, and Spearman correlation. In addition, a multiple linear regression was used to analyze whether there were any influential factors on a psychological aspect. Statistical significance was taken at P < 0.05. Results: A total of 271 respondents completed the questionnaire. They tended to have positive perceptions toward the outcomes of orthodontic treatment. There was a significant difference in self-perceived esthetics between the male and female groups. Self-perceived esthetic and functional concerns as well as satisfaction of treatment procedures were found to have significant correlations with the psychological aspect. Sex and self-perceived esthetics were also found to be significant predictors of the psychological aspect. Conclusion: PROMs/PREMs should be considered as an important tool to assess treatment outcomes in orthodontic practice. There appeared to be significant impact of self-perceived esthetics on psychological aspects. Consequently, PROMs and PREMs should be introduced in orthodontic postgraduate programs to enhance the concept of patient-centered care in orthodontic practice.

Keywords: Esthetics, orthodontics, patient-centered care, treatment outcome

How to cite this article:
Santiwong P, Sommaluan K, Mokkasak S, Rachuratchata C, Rattanaopas T, Sipiyaruk K. The implementation of PROMs/PREMs in the assessment of orthodontic treatment outcomes: A questionnaire survey. J Int Soc Prevent Communit Dent 2022;12:210-5

How to cite this URL:
Santiwong P, Sommaluan K, Mokkasak S, Rachuratchata C, Rattanaopas T, Sipiyaruk K. The implementation of PROMs/PREMs in the assessment of orthodontic treatment outcomes: A questionnaire survey. J Int Soc Prevent Communit Dent [serial online] 2022 [cited 2022 May 17];12:210-5. Available from: https://www.jispcd.org/text.asp?2022/12/2/210/342725

   Introduction Top

The consideration of treatment outcomes of orthodontic practice has been widely discussed. Although there appear to be several concepts, its main goal is to provide patients with a normal occlusion, well function, and good arrangement of teeth after treatment.[1],[2] It appears that those evaluations of orthodontic treatment outcomes have been evaluated by only dental professionals. However, perceptions toward tooth alignment and expectations of the treatment could be varied amongst orthodontic patients,[3] resulting that the evaluation of self-perceived treatment outcomes in orthodontic patients seems to be insufficient with the traditional outcome measurement.

Patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) appear to be increasingly used in orthodontic practice since 2018. PROMs can be used to evaluate the outcomes of orthodontic treatment.[4],[5] In addition, the evaluation of perceived pain and discomfort during dental treatment seems to be possible with PROMs.[6] Health status, primary concerns and experiences of orthodontic care, and oral health-related quality of life (OHRQoL) could also be measured to evaluate self-perceived treatment outcomes through the concept of PROMs.[6],[7] These arguments support the use of PROMs in orthodontic practice, as a patient-centered approach can be enhanced.

Although PROMs have been increasingly used in orthodontic practice, there appeared to be no study exploring relationships between self-perceived treatment outcomes and psychological impact. Dental esthetics seem to have impact on psychosocial aspects,[8] and this issue should also be considered as a necessary component in PROMs, as it can reflect whether or not the outcomes of orthodontic treatment meet the expectation of patients. This research aimed to construct a questionnaire using the concept of PROMs and PREMs, as well as to explore whether or not self-perceived treatment outcomes and experiences could have impact on psychological aspects.

   Materials and Methods Top

This research employed a quantitative research selected using a survey design. A paper-based questionnaire was used as a data collection tool to gather quantitative data of PROMs, PREMs, and OHRQoL. This research protocol was approved by the Faculty of Dentistry and the Faculty of Pharmacy, Mahidol University, Institutional Review Board (MU-DT/PY-IRB), reference number: MU-DT/PY-IRB 2018/058.0010.

As there appeared to be no research of PROMs or PREMs conducted in Thailand, the questionnaire in this study was constructed based on literature review and group discussion. Items for the questionnaire were developed from previous studies;[7],[8],[9],[10] however, some of them were adapted to suit the research objectives. All items were first constructed in English. They were then translated into Thai by the researchers in order to suit the respondents with the validity process of translation.[11] The process required the Thai version to be blindly translated back into English. The two English versions were then compared, and any content mismatch was revised to ensure the validity of translation.

The questionnaire consisted of five parts, which were “Demographic data”, “Self-perceived esthetics”, “Self-perceived functional concern”, “Satisfaction of treatment procedures”, and “Psychological aspect”. The questionnaire contained both positive and negative items to minimize a response bias. Dichotomous questions and checklists were used for the “demographic data” part, while 5-point Likert scales were selected for the other parts, in which “1” refers to “Strongly disagree”; “2” refers to “Disagree”; “3” refers to “Neither agree nor disagree”; “4” refers to “Agree”; and “5” refers to “Strongly agree”. A choice of Likert scale was selected, as it could measure opinions and attitudes of respondents. In addition, comparing with a Visual Analog Scale, Likert scale was easy to understand, requiring less total training time for teaching respondents.

The questionnaire was piloted in five experts and 50 orthodontic patients to confirm its validity and reliability, as well as to assure that all questions were clear and understandable. Content validity was conducted at both item-level and scaled-level, where the items were revised or excluded until their index become 1.0 at both levels. In addition, test-retest reliability was employed to evaluate whether the questionnaire was reliable or not; the items with a correlation coefficient were deleted from the questionnaire. Cronbach’s alpha was also calculated to measure internal consistency, and the value was greater than 0.70 for each construct.

Patients who had a complete fixed appliance orthodontic treatment from the Orthodontic Clinic, Faculty of Dentistry, Mahidol University were eligible for this research if they were 15 years old or above. However, they were excluded if they received an orthognathic surgery, or a further retreatment was required. To achieve the confidence level at 95% and margin of error at 5%, a total of 271 patients were recruited for the survey using a convenient sampling. The questionnaire was distributed between 1st October 2018 and 31st March 2019.

Descriptive statistics were used to display an overview of research data. As the data were not normally distributed according to the Kolmogorov-Smirnov test, a Mann-Whitney U test was used to compare means between two groups. In addition, a Spearman correlation was conducted to explore relationships amongst the variables, and a multiple linear regression was used to analyze whether or not the independent variables (“Demographic data”, “Self-perceived esthetics”, “Self-perceived functional concern”, and “Satisfaction of treatment procedures”) had influence on the dependent variable (“Psychological aspect”). Statistical significance was taken at P < 0.05.

   Results Top

The questionnaires were completed by 271 respondents, which were 200 (73.8%) females and 71 (26.2%) males. Nearly 90% of the respondents were over 18-years old.

The respondents seemed to have quite positive perceptions toward the esthetic aspect after the orthodontic treatment (4.3 from 5). The most positive perception was the better alignment of their anterior teeth (4.6 from 5). However, their perceptions were reported less positive when they compared their teeth with their friends (3.7 from 5). These data are presented in [Table 1].
Table 1: Self-perceptions of an esthetic aspect

Click here to view

[Table 2] demonstrates that a function aspect was perceived less positively when compared with the esthetic concern, with the overall score was 3.8 from 5. Oral hygiene was perceived as the most positive outcome, as the respondents reported they could brush your teeth easily after treatment (4.2 from 5). They also felt that, after treatment they felt more comfortable during a meal (3.9 from 5), and they could pronounce any words better after treatment (3.7 from 5).
Table 2: Self-perceptions of a function aspect

Click here to view

In terms of satisfaction during treatment procedure [Table 3], the respondents perceived this aspect as very positive (4.6 from 5). They felt that their dentists were polite (4.7 from 5) and kept informing their treatment progress (4.5 from 5). Also, the treatment area was perceived very clean (4.6 from 5).
Table 3: Patient’s satisfaction towards a treatment procedure

Click here to view

The psychological aspect appeared to be improved after orthodontic treatment, as reported by respondents (4.1 from 5), as shown in [Table 4]. They tended to feel less embarrassed after treatment due to problems with their teeth (4.4 from 5). On the contrary, they reported they felt proud (4.3 from 5) and had improved self-esteem (4.2 from 5) after the treatment.
Table 4: Patient’s perceptions toward psychological aspects

Click here to view

The findings showed significant difference in the esthetic aspect between the female and male groups, and the score of the latter one was significantly higher (P < 0.05). However, no significant differences were found between both sexes in the function and psychological aspects as well as satisfaction toward treatment procedure. These data are presented in [Table 5].
Table 5: A comparison of overall mean scores when considering sexes

Click here to view

According to [Table 6], the findings demonstrated the positive and significant correlation between the psychological aspect and self-perceived esthetics (r = 0.533, P < 0.01). The psychological aspect also significantly correlated with the functional consideration (r = 0.404, P < 0.01) and satisfactions toward a treatment procedure (r = 0.376, P < 0.01), although they were weaker than the correlation with the esthetic concern.
Table 6: Correlations between psychological aspects and influencing factors

Click here to view

The regression analysis demonstrated that the psychological aspect could be significantly affected by sex (β = −0.240, P < 0.01) and self-perceived esthetics (β = −0.572, P < 0.001). However, the other factors were not found to be significant predictors of the psychological aspect [Table 7]. When considering overall fit of the model, the R2 value for the regression model was 0.397, which indicated that sex and esthetic aspect accounted for 40% of the psychological aspect. In terms of collinearity, variance inflation factor and tolerance statistics of all variables in this model were less than 10 and more than 0.2, respectively, meaning that the influencing factors were not strongly correlated.
Table 7: Multiple regression analysis of psychological aspect and its influential factors

Click here to view

   Discussion Top

Most of the respondents in this study were female; this trend was also found in other studies, which found that a number of female orthodontic patients were higher than that of males.[12],[13] This could be explained by an argument that females tended to being concerned with their dental and facial appearances more than males.[12],[13],[14] Therefore, orthodontic treatment was perceived more important in females, compared to males. This argument was concurrent to another finding of this study, and the score of perceptions toward the esthetic aspect was lower in the female group. This could imply that females might have expectation on this aspect higher than males.

The respondents in this study perceived positively toward functional improvement after orthodontic treatment. Orthodontic patients tend to be highly satisfied with dental functions particularly with eating and chewing as well as oral comfort, which were significantly correlated with the total satisfaction score.[12] However, this study found that the score of this topic seemed lower than other aspects. Not all of the respondents reported that they were satisfied with a food impaction problem after orthodontic treatment. It could be suggested that this issue was hardly detected by orthodontists, and therefore a question regarding food impaction may be required before orthodontic treatment completion.

Our respondents tended to perceive their psychosocial concerns improved than before treatment. This similar trend was also found in previous research, where there were significant improvements in psychological well-beings of patients after orthodontic treatment.[15],[16],[17] Likewise, orthognathic patients experienced a reduction in bullying and had self-confidence improvement.[14] This could be a result from the improvement of self-perceived appearance after a complete orthodontic treatment, as they were significantly correlated.

Self-perceived aesthetics and sexes were significant predictors of psychological improvement. Facial appearance seems to have impact on OHRQoL.[18] However, a functional concern appeared to be no significant impact on psychological aspects. Patients tended to seek for orthodontic treatment with expectations to improve their aesthetics and social acceptance, rather than oral function improvement.[19],[20] In addition, as males tended to perceive their appearances more positively when compared with females, sex was another factor to significantly affect psychological improvement. These arguments support the use of PROMs in measurement of orthodontic treatment outcomes, as they seemed to have impact on psychological aspects.

The concept of PROMs appears to be necessary for orthodontic practice, especially in the comprehensive assessment of treatment outcomes. However, psychological aspects have not been widely studied in orthodontic practice.[21] This research support the use of PROMs/PREMs as a tool for assessing orthodontic outcomes, as self-perceived esthetic and functional improvement as well as satisfactions of treatment procedures appear to have impact on psychological perspectives, which cannot be measured by professional judgement or clinical assessment. They have been perceived as a promising tool in dentistry including orthodontics.[6],[22],[23] Consequently, PROMs/PREMs should be emphasized in postgraduate orthodontic programs. This should be set as a learning outcome requiring orthodontists to comprehensively evaluate the treatment outcomes, enabling the convergence of orthodontic practice and patient-centered care in the future.

There could be a limitation in the application of results retrieved from this single-site study to other settings. The findings of this research could be generalized to a group of patients who have received orthodontic treatment at the Orthodontic Clinic, Faculty of Dentistry, Mahidol University, presuming a 95% confidence level. However, when considering whether or not the results of this study is transferable to another setting, although the data could be selectively applied to another population with a similar context, research in other sites should be required to ensure the transferability. In addition, further research may emphasize how PROMs/PREMs can be implemented as an assessment tool in postgraduate orthodontic programs.

   Conclusion Top

PROMs/PREMs should be considered for orthodontic practice to enhance the concept of patient-centered care. The questionnaire constructed in this research can be used to evaluate treatment outcomes by exploring how patients perceived their dental appearance and functional concerns, as well as their satisfaction of treatment procedures. Self-perceived esthetics can be considered as very important, as they tended to have impact on psychological aspects. However, both PROMs and PREMs have not been widely used in orthodontic practice. Consequently, they should be considered as a promising tool for evaluating treatment outcomes in postgraduate orthodontic programs.


The authors would like to thank all respondents for their contribution to this study.

Financial support and sponsorship

This research was supported by the Faculty of Dentistry, Mahidol University, Thailand.

Conflicts of interest

The authors declare no conflict of interest.

Authors contributions

Conceptualization, P.W. and K.Si.; methodology, P.W. and K.Si.; investigation, P.W., K.So., S.M., C.R., T.R., and K.Si.; data analysis, P.W., K.So., S.M., C.R., T.R., and K.Si.; validation, P.W. and K.Si.; manuscript writing, P.W. and K.Si.; manuscript review, P.W. and K.Si. All authors have read and agreed to the published version of the manuscript.

Ethical policy and institutional review board statement

This research protocol was approved by the Faculty of Dentistry and the Faculty of Pharmacy, Mahidol University, Institutional Review Board (MU-DT/PY-IRB), reference number: MUDT/PY-IRB 2018/058.0010.

Patient declaration of consent

The participant information sheet was distributed with the questionnaire for patients aged 18 years and older; the return of a completed questionnaire was considered as implied consent. Patients aged 15-17 years were provided with a participant information sheet and consent form, requiring consent from them and their parents.

Data availability statement

The data are available on request from the corresponding author.

   References Top

Wylie WL The mandibular incisor--Its role in facial esthetics. Angle Orthod 1955;25:32-41.  Back to cited text no. 1
Bondemark L, Holm AK, Hansen K, Axelsson S, Mohlin B, Brattstrom V, et al. Long-term stability of orthodontic treatment and patient satisfaction. A systematic review. Angle Orthod 2007;77:181-91.  Back to cited text no. 2
Isiekwe GI, Sofola OO, Onigbogi OO, Utomi IL, Sanu OO, daCosta OO Dental esthetics and oral health-related quality of life in young adults. Am J Orthod Dentofacial Orthop 2016;150:627-36.  Back to cited text no. 3
Cunningham SJ The sheldon friel memorial lecture 2020 it’s not just about the teeth: Patient-centred orthodontics. Eur J Orthod 2020;42:472-7.  Back to cited text no. 4
Bradley E, Shelton A, Hodge T, Morris D, Bekker H, Fletcher S, et al. Patient-reported experience and outcomes from orthodontic treatment. J Orthod 2020;47:107-15.  Back to cited text no. 5
Perazzo MF, Serra-Negra JM, Firmino RT, Pordeus IA, Martins-JÚnior PA, Paiva SM Patient-centered assessments: How can they be used in dental clinical trials? Braz Oral Res 2020;34 Suppl 2:e075.  Back to cited text no. 6
Okayasu M, Susami T, Asahito T, Sakamoto T, Kurata K, Ishiwata Y, et al. Questionnaire investigation after orthodontic treatment in patients with unilateral cleft lip and palate: An inter-clinic collaborative study. Orthodontic Waves 2018;77:90-100.  Back to cited text no. 7
Klages U, Claus N, Wehrbein H, Zentner A Development of a questionnaire for assessment of the psychosocial impact of dental aesthetics in young adults. Eur J Orthod 2006;28:103-11.  Back to cited text no. 8
de Oliveira CM, Sheiham A Orthodontic treatment and its impact on oral health-related quality of life in Brazilian adolescents. J Orthod 2004;31:20-7; discussion 15.  Back to cited text no. 9
Bennett ME, Tulloch JF, Vig KW, Phillips CL Measuring orthodontic treatment satisfaction: Questionnaire development and preliminary validation. J Public Health Dent 2001;61:155-60.  Back to cited text no. 10
Brislin RW Back-translation for cross-cultural research. J Cross Cult Psychol 1970;1:185-216.  Back to cited text no. 11
Aljughaiman A, Alshammari A, Althumairi A, Alshammari A, Almasoud N, Nazir MA Patient satisfaction with orthodontic treatment received in public and private hospitals in dammam, saudi arabia. Open Access Maced J Med Sci 2018;6:1492-7.  Back to cited text no. 12
Al-Omiri MK, Abu Alhaija ES Factors affecting patient satisfaction after orthodontic treatment. Angle Orthod 2006;76:422-31.  Back to cited text no. 13
Schilbred Eriksen E, Moen K, Wisth PJ, Løes S, Klock KS Patient satisfaction and oral health-related quality of life 10-15 years after orthodontic-surgical treatment of mandibular prognathism. Int J Oral Maxillofac Surg 2018;47:1015-21.  Back to cited text no. 14
Gazit-Rappaport T, Haisraeli-Shalish M, Gazit E Psychosocial reward of orthodontic treatment in adult patients. Eur J Orthod 2010;32:441-6.  Back to cited text no. 15
Chen M, Feng ZC, Liu X, Li ZM, Cai B, Wang DW Impact of malocclusion on oral health-related quality of life in young adults. Angle Orthod 2015;85:986-91.  Back to cited text no. 16
Neely ML, Miller R, Rich SE, Will LA, Wright WG, Jones JA Effect of malocclusion on adults seeking orthodontic treatment. Am J Orthod Dentofacial Orthop 2017;152:778-87.  Back to cited text no. 17
Larsson P, Bondemark L, Häggman-Henrikson B The impact of oro-facial appearance on oral health-related quality of life: A systematic review. J Oral Rehabil 2021;48:271-81.  Back to cited text no. 18
Kiyak HA Does orthodontic treatment affect patients’ quality of life? J Dent Educ 2008;72:886-94.  Back to cited text no. 19
Santiwong P, Phenphasit P, Chatthanavej R, Sonjon W, Patthanapuapun S, Anantapanyagul S, et al. Self-perceived needs for orthodontic retreatment among dental students: A qualitative study. J Int Soc Prev Community Dent 2021;11:402-7.  Back to cited text no. 20
Peter E, Baiju R, Varughese J, Varghese N Patient-reported outcome measures in orthodontics. Dent Med Res 2019;7:3-11.  Back to cited text no. 21
Rosen EB, Donoff RB, Riedy CA U.S. Dental school deans’ views on the value of patient-reported outcome measures in dentistry. J Dent Educ 2016;80:721-5.  Back to cited text no. 22
Barber S, Shelton A Patient questionnaires in orthodontics. Orthodontic Update 2021;14:14-20.  Back to cited text no. 23


  [Table 1], [Table 2], [Table 3], [Table 4], [Table 5], [Table 6], [Table 7]


Print this article  Email this article
    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Article in PDF (199 KB)
    Citation Manager
    Access Statistics
    Reader Comments
    Email Alert *
    Add to My List *
* Registration required (free)  

    Materials and Me...
    Article Tables

 Article Access Statistics
    PDF Downloaded51    
    Comments [Add]    

Recommend this journal