A Rasch Analysis of Perceived Stigma of Covid-19 among Nurses in Indonesia Questionnaire

This study is the first to investigate and validate the psychometric properties of the perceived stigma of the Covid-19 questionnaire among nurses in Indonesia. The respondents in the study were 2156 nurses from various regions in Indonesia (34 provinces), consisting of 613 males (28.4%) and 1543 females (71.6%). The number of nurses consisted of 300 respondents (13.9%) from the western region of Indonesia, 1825 respondents (84.6%) from the central region of Indonesia, and 31 respondents (1.4%) from the eastern region of Indonesia. The Rating Scale Model used to investigate the questionnaire's psychometric properties. The analysis results showed that the perceived stigma questionnaire met all the psychometric aspects (unidimensionality, item fit analysis, item validity, diagnostic rating scale, and reliability). Rasch Model analysis showed a value that meets all aspects measured criterion limit. Therefore, this perception stigma questionnaire can measure the perceived stigma of Covid-19 among nurses in Indonesia. A Novel Coronavirus Disease 2019 (Covid-19) has become a global problem. The of released The data shows that the number of patients exposed to Covid-19 increases rapidly. In Indonesia, as of July 14, 2020, the number of positive cases of Covid-19 was 78.572, the number of recovered cases was 37.636, and the number of deaths was 3710.

Health workers may experience avoidance by their family or community because of stigma.
That difficult situation can become more complex (World Health Organization, 2020). Therefore, it is necessary to take appropriate steps to help health workers face psycho-social problems, especially mental disorders and stigma. Research exploring the mental health and stigma of Covid-19 vanguard nurses has been carried out by the Nursing Faculty of the University of Indonesia. The study used a test instrument to identify mental health and nurses' stigma as the vanguard of Covid-19 in Indonesia.
Accurate measurement requires a valid and reliable instrument. An accurate measurement can produce data that can describe actual conditions related to mental health problems and stigma. Furthermore, the data can be used by the government or related parties as a consideration in formulating policies to more effectively address the problem of mental health disorders and the stigma faced by nurses in Indonesia. One of the essential points is that an accurate measurement can be considered in preparing mental health care and psychological counselling programs for Covid-19 nurses in Indonesia. Therefore, it is crucial to conduct indepth studies related to developing instruments that meet the criteria of validity and reliability.
The study aims to examine the instrument of the perceived stigma of Covid-19 among nurses in Indonesia through a psychometric approach. This research has never been done before, especially in Indonesia. According to Kean et al. (2018), psychometric evaluation in psychology and measurement in education has undergone a shift, which usually uses classical theory tests, now developing using modern test theory, such as item response theory and the Rasch measurement model. The Rasch model allows for in-depth investigations of various aspects of measurement, including dimensionality, local independence, item and test taker suitability, and the use of Wright maps (Rasch, 1960;Andrich & Marais, 2019;Rahayu et al., 2020). Wright and Mok (2004) stated some of the advantages of using the Rasch Model approach as it can: provide linear measures with the same intervals; carry out an accurate estimation process; find inappropriate (misfit) or uncommon (outliers) items; address missing data; yield replicable measurements (independence of the parameters studied). Considering those advantages, psychometric evaluation using the Rasch model is expected to produce the same quality measurements as measurements in the specific field. Therefore, the psychometric evaluation of the items in nurses' mental health and stigma identification tests in Indonesia used the Rasch model.

Method Participants
The participants in this study consisted of 2,156 nurses from various regions in Indonesia (34 provinces). The nurses come from various health service units that handle  (government hospitals, private hospitals, community health centers, and integrated service units). The following is a table of the characteristics of the respondents used.

Rasch Model
One of the revolutions in psychometrics is the probabilistic model introduced by George Rasch in 1960. According to Wright and Masters (1982), the limitations of classical theory can be overcome by using the Rasch Model approach. It is used in constructing a measurement scale more accurately. The Rasch model is a model that has been used widely in the development of measurement instruments in various fields, including mental health and psychology research (Lusczakoski et al., 2016). The Rasch model has many advantages so that the measurements are more accurate. Therefore, this study used the Rasch Model approach in evaluating the validity and reliability criteria of the developed instrument.
The questionnaire used to measure perceived stigma was an ordinal scale of answer choices. The Rasch model's choice in analyzing data with a rating scale is very appropriate because the rating scale response data produces ordinal data that needs to be transformed into an interval scale to increase the measurements' accuracy (Boone et al., 2014). Rating Scale Model (RSM; Andrich, 1978) is a polytomous IRT model. Let X be an ordinal item data matrix of size nxm. K is an item that has the same category. The RSM mathematical equation for the probability of a v test taker getting an h score on item i is as follows,  Based on the table above, the total raw variance data is 91.04%. This value is not much different from the expected value. It shows that the unidimensional conditions have been met. Linacre (2016) states that the minimum value of total raw variance to meet unidimensional requirements is at least 40%. It is supported by the table's information that all other variances that the test instrument cannot explain are below 10%. It meant that the level of independence of the test instrument items is in the item category. The unidimensional assumption based on the opinion of Linacre (1998) states that if the value of raw variance explained by Measures > 30%, then an instrument meets the unidimensional criteria. From the analysis results, the raw variance explained by measures value is more than 30%. In 2016, Linacre stated that the eigenvalue of unexplained variance in first contrast is quite small (2.76) with an observed percentage value of 3%. It also shows the dimension of an instrument. contrast, item B2 with a logit value of -1.84 indicates the easiest item for respondents to answer. The total count column contains the same value (2156). It shows that no data is missing for each item analyzed.

Item Fit
To determine how well the items are in an instrument, the MNSQ outfit criteria were used from 0.5 to 1.5 (Bond & Fox, 2015). From the table above, all MNSQ outfit values are in this range. It shows that all items in the questionnaire meet the criteria for item validity. Thus, the questionnaire items have the decent quality to be used in data collection related to mental health problems especially the perceived stigma experienced by Covid-19 nurses in Indonesia. The table above describes the instrument's character, including the respondent's ability to its reliability. The item reliability value showed the number .98, which means that the value is very good. Thus, value indicates that the item will consistently provide the same measurement value for various respondents (Abdullah et al., 2012). The separation section shows that the instrument can divide respondents into several groups (Othman et al., 2012).

Rating Scale Diagnostic
The higher the separation value, the better the instrument.

Discussions
The measurement of the perceived stigma of Covid-19 among nurses is needed in the current pandemic era since nurses are at the forefront of tackling Covid-19. How can one trust that questionnaires provide meaningful information about the condition of nurses' perceived stigma? It is needed to have a scientific justification for the response data given to the questionnaire items (Clinton et al., 2014). The scientific justification for this research is done by using the Rasch model approach. Quantitative social research often does not engage with these issues and takes questionnaire (and other) data at 'face value,' ignoring Psychological Thought South-West University "Neofit Rilski" 2022, Vol. 15(1), 12-28 https://doi.org/10.37708/psyct.v15i1.530 potential problems of reliability and validity (Shryane et al., 2020). The Rasch model provides a combination analysis technique between statistical models and measurement theory to justify the validity and reliability of multivariate data (Baker, 1998).
This study used RSM to assess and validate the questionnaire on the perceived stigma of nurses for Covid-19 among nurses. The results of the analysis of the questionnaire indicate that the questionnaire items are generally adequate for measuring the nursing stigma perception in Indonesia. A unidimensional assumption is a prerequisite that needs to be met to justify the quality of the questionnaire using the Rasch Model approach (Mair, 2018). In Winstep, testing this unidimensional assumption is carried out using principal component analysis; this can be seen as a limitation in this study. Testing the prerequisite for this unidimensional assumption is important because more than one method must be used to ensure that unidimensional assumptions are met. One of the methods is Confirmatory Factor Analysis (CFA) (Flora & Curran, 2004). The results show that the questionnaire used has a high level of reliability.
Furthermore, the RSM analysis offers additional analysis of each item's performance and response category of the perceived stigma questionnaire. The diagnostic rating scale indicates that nurses have no difficulty confirming their answers to the answer choices provided on the questionnaire. The rating scale diagnostics show that respondents inconsistently select several categories; the solution to the problem is to combine the categories that are close together and re-analyzed. It increases variable clarity and eliminates noise (Fox et al., 1994;Linacre, 1999;Wright & Linacre, 1992). RSM shows that most of the stigma questionnaire items are perceived to have outfits and infits. In addition, MNSQ statistics are still within the standard criteria limit. This shows that, in general, these questionnaire items consistently measure one defined construct (Tennant & Conaghan, 2007).
The results of the fit item analysis have shown that the items usually function to measure the perceived stigma. An inappropriate question indicates a misconception between the item and the respondent. It will be useful for those who take measurements to improve the quality of these items to produce accurate data. According to Boone et al. (2004), and Bond & Fox (2015), treating an inappropriate item is by reviewing and then considering whether the item will be repaired or replaced with the new one. It ensures that the measurement carried out related to the perceived stigma is done by using qualified items in the future.
One of the revolutions in the item response theory is Wright Maps. This map illustrates the Psychological Thought South-West University "Neofit Rilski" 2022, Vol. 15(1), 12-28 https://doi.org/10.37708/psyct.v15i1.530 diagnostic results for rating scale data, even for high complexity. Theoretical and conceptual definitions of latent variables need to be done properly. Well-defined definitions can improve test quality and avoid too many test items so that respondents do not waste time doing the test. It makes the responses more accurate. The Wright map allows researchers to evaluate how questionnaire items define a latent variable by following three steps: assessing the strengths and weaknesses of an instrument documenting the questionnaire item hierarchy, and comparing theory with the results of observational data (Boone et al., 2014). The Wright map above shows that the average item is one logit above the respondent's average ability.
According to Boone et al. (2014), based on the literature review, the optimal test instrument is achieved when the average items are equal to the average person. However, other references state that the optimal test instrument is psychologically achieved when the average item is one logit higher than the average person (Linacre, 2010). The Wright map indicates that items B11a, B11b, B13, and B4 have a difficulty level far above the respondent's ability. The difficulty level in items B11a and B11b is likely caused by questions that unmarried nurses can only answer for item B11a, while married nurses can only answer b11b. Thus, in the results of the WINSTEP analysis, most of the nurses could not answer these questions.
Therefore, on the Wright map, B11a and B11b are detected as questions with difficulty levels above the test-taker's ability. This matter needs to be considered by the test developer to review the analysis results before deciding to dispose of the item (Downing, 2006;Haladyna & Rodriguez, 2013). The level of difficulty of this difficult question does not mean that the item in question must be discarded. Instead, it will be better if it is reviewed and corrected.
Meanwhile, items B13 and B4 have different cases. In terms of editorial, the questions used are concise. However, these two items have a difficulty level above the respondent's ability. It is most likely caused by unfocused questions so that respondents cannot ensure the choice of response they provide.
The limitation in this study is that the unidimensional test used is the principal component analysis method. To strengthen further the testing of unidimensionality assumptions in the questionnaire used, there is a need for a more in-depth study that applies other methods to test the unidimensionality of this questionnaire, for example, by using Confirmatory Factor Analysis (CFA) (Flora & Curran, 2004). Besides, the respondents in this study are from various regions throughout Indonesia. This regional difference certainly affects the cultural variations of the multiple respondents in this study. It may cause bias in the items used.
Therefore, it is necessary to have further research regarding the differential item function test Psychological Thought South-West University "Neofit Rilski" 2022, Vol. 15(1), 12-28 https://doi.org/10.37708/psyct.v15i1.530 for each item to further strengthen the psychometric evaluation so that the data obtained can honestly describe the condition in the field.

Conclusion
Based on the results of the response analysis that has been carried out on the psychometric properties of the perceived stigma of Covid-19 among nurses in Indonesia questionnaire, most of the items in the questionnaire used are good in terms of psychometric aspects (unidimensionality, item fit analysis, item validity, diagnostic rating scale, and reliability).
Furthermore, this questionnaire can collect data related to mental health problems and perceived stigma faced by Covid-19 nurses. The data can be used to consider the government and related parties in formulating programs to follow up on these problems.
Further research can increase measurement accuracy, including by testing the assumption of unidimensionality using other methods such as factor analysis. In addition, further can also be done to improve the quality of the measurement by considering the function of differential information on the questionnaire items.

Funding
The authors have no funding to report.

Other Support/Acknowledgement
The authors have no support to report.
For those of you who are married, do you feel your current condition is causing marriage problems?
Do you feel that your current condition makes it difficult for your family or relatives to get married?
Have you ever been asked to stay away from your environment? 14 Pernahkah anda memutuskan sendiri untuk menjauh dari komunitas di lingkungan anda?
Because of your condition treating COVID-19 patients, do other people think you also have health problems like COVID-19 patients?