EXAMINING UNIDIMENSIONALITY OF PSYCHOMETRIC PROPERTIES VIA RASCH MODEL

Transcription

1 International Journal of Civil Engineering and Technology (IJCIET) Volume 9, Issue 9, September 2018, pp , Article ID: IJCIET_09_09_141 Available online at ISSN Print: and ISSN Online: IAEME Publication Scopus Indexed EXAMINING UNIDIMENSIONALITY OF PSYCHOMETRIC PROPERTIES VIA RASCH MODEL Amal Hayati Ishak Academy of Contemporary Islamic Studies, Universiti Teknologi MARA Shah Alam, Selangor, Malaysia Mohd Hapiz Mahaiyadin Academy of Contemporary Islamic Studies, Universiti Teknologi MARA, Pulau Pinang Branch, Bertam Campus, Pulau Pinang, Malaysia Mohd Amzari Tumiran Academy of Contemporary Islamic Studies, Universiti Teknologi MARA Shah Alam, Selangor, Malaysia ABSTRACT Rasch Model supports the validation of psychometric properties, through a series of indicators such as item polarity, fit statistics, local item dependency and unidimensionality. Nevertheless, this article specifically presents the analysis on unidimensionality of a newly developed instrument, the Islamic Quality Management Scale (IQMS). The IQMS measures the application of values in the context of quality management. The scale is proposed due to the weakness of existing scales which merely highlighting the material content of quality management, marginalizing the values embedded in. This article validates on the IQMS unidimensionality to approve on its usability in future studies. The results assured on the IQMS unidimensionality and reliability. Key words: IQMS, unidimensionality, Rasch Model, reliability, quality management Cite this Article: Amal Hayati Ishak, Mohd Hapiz Mahaiyadin and Mohd Amzari Tumiran, Examining Unidimensionality of Psychometric Properties Via Rasch Model. International Journal of Civil Engineering and Technology, 9(9), 2018, pp INTRODUCTION The Rasch Measurement Model defines how a scale measures latent variables (Singh, 2004; DeVellis, 2012). It contains methods to link observed elements with latent constructs and subsequently generate meaningful results. As explained by the advocates of Rasch Model (Bond & Fox, 2007), Rasch is a probabilistic model which uses logit as measurement units editor@iaeme.com

2 Amal Hayati Ishak, Mohd Hapiz Mahaiyadin and Mohd Amzari Tumiran The logits are obtained by transforming ordinal data into interval data where the data can be mapped into a linear scale. The proponents of Rasch Model (Singh, 2004; Bond & Fox, 2007; Tennant & Conaghan, 2007; Aziz et al., 2013) have agreed that Rasch Model provides sufficient parameters for a good measurement with the advantage of; providing a linear scale, transforming ordinal data into interval data, proposing suggestion for missing data and assessing items quality. Thus, it is commonly suitable for validating the reliability of a developed measurement scale. This article demonstrates the unidimensionality analysis of a newly developed scale. The scale, Islamic Quality Management Scale (IQMS) was developed following the meticulous procedures of scale development as proposed in by (Spector, 1992). The development process starts with systematic literature review, followed by expert review analysis. Later, the set of items was appraised empirically via Fuzzy Delphi analysis and Rasch Model. Although Rasch Model offers a variety of reliability indicators, this articles reports on the unidimensionaility analysis which reflects that all items are measuring a similar latent trait. 2. METHODOLOGY Rasch Measurement Model has been increasingly popular in scale development studies (DeVellis, 2012). It has been widely applied in testing and validating instruments in various areas including education, business, marketing and behavioural studies. For instance, (Bechtel, 1985) used Rasch for a consumer rating scale, (Albano Jr., 2009) for individual happiness scale and (Salzberger & Koller, 2013) for their marketing scale. Rasch model performs the assessment based on the response of a sample of respondents to a set of measurement scale. In Rasch, each person is categorized based on ability, while items are categorized based on difficulty. The categorization is resulted from the interaction between person ability and item difficulty, which utilizes log odd values. Rasch transform ordinal response (the Likert scale) into log odd values based on the probability of success, which depends on the differences between person ability and item difficulty. These logit values enable the person ability and item difficulty to be mapped in a log ruler. The mapping is based on two principle assumptions; 1. A more developed (or able) person has greater likelihood of endorsing all items, and 2. Easier items have greater likelihood to be endorsed by all respondents. Based on these two assumptions, Rasch model predicts the location of items and persons in a map. Rasch also provides empirical evidences indicating unidimensionality, problematic items and scale reliability, among others, which require the researcher to scrutinize items prior to item deletion (Bond & Fox, 2007). Unidimensionality is the focus in this article. As explained by (Chou & Wang, 2010), dimensionality, which is the number of latent traits determining item responses, is a key assumption in Item Response Theory (IRT). The assumption of unidmensionality is crucial in order to ascertain that all items in a test measure the same latent trait (Apple, 2013). Similarly, the unidimensionality will indicate the reproducibility or reliability of a test. Generally, Rasch Model indicates unidimensionality via Principal Component Analysis (PCA) and local independence analysis. However, this article only reports the PCA. According to (Chou & Wang, 2010), the PCA on residual score is defined as the difference between an observed score and its expected score. Unidimensionality and local independence are related because, by definition, a data set is unidimensional when item responses are locally independent based on a single latent variable. The view has been agreed by (Embretson & Reise, 2000). Thus, Wright (1996) puts forward that PCA on residuals from a unidimensional data set is expected to extract no principal components. Similarly, Linacre editor@iaeme.com

3 Examining Unidimensionality of Psychometric Properties Via Rasch Model (1998) observes that PCA on standardized residuals provides sufficient diagnostic information about dimensionality. The instrument tested for PCA in this article is the IQMS (Islamic Quality Management Scale); meant for measuring the application of Islamic values in quality management context. Based on an extensive literature review by (Ishak & Osman, 2016), this article reports on the validity of 60 items vested under eight dimensions as a measurement scale to assess Islamic values application in quality management context. The IQMS is proposed as a new measurement since the existing tools have been marginally refer to Islamic substances. The existing empirical studies have narrowly analysed values based on the general framework of Hofstede s cultural dimensions (Baird, 2011), Competing Value Framework (CVF) of O Reilly (Prajogo & McDermott, 2005; 2011; Gambi et al., 2013), Organizational Culture Profile (OCP) of Quinn & Rohrbaugh (Denison & Spritzer, 1991), and Detert s framework (Detert et al., 2000; 2003). Since the past two decades, there are consistent works bridging quality management and Islamic values by contemporary scholars, such as (Ahmad, 1996; Sadeq & Ahmad, 1996; Al- Buraey; 2005; Basir et al., 2006; Mohd Mokhtar et al., 2011; Basir et al., 2011; Ishak & Rahimi, 2016). They similarly put forward the conceptual elaboration of a set of values, grounded in Islamic substance, and embedded in the practice of quality management. However, empirical data on that matter is lacking. Yet, an appropriate instrument could not be located. Thus, the IQMS targets to fill in the gap. The IQMS contains 60 items which has been prudently develop and accordingly tested (Ishak et al., 2015; 2016a; 2016b; 2017). The IQMS is the output of a well-planned scale development process, beginning with a systematic literature review of extensive online and offline sources. Remarkably, the dimensions are grounded from major Islamic substance; the Quran and the prophetic traditions. This criteria marked the uniqueness of the IQMS as compared to other existing measurement scales within the scope of quality management. The IQMS was endorsed by a carefully selected expert panel consists of 11 individuals. Once the review completed the items undergone a Fuzzy Delphi analysis (Ishak et al., 2015), which is a process to obtain validity of the items. Fuzzy Delphi was conducted with a different panel of experts containing 17 members. Later, in order to report on the unidimensionality of the psychometric properties, the set of items was administered among participants of ISO9001 training course. They consists of individuals related to quality management initiative in their respective organisations. The subsequent section reports on the unidimensionality of the IQMS via Rasch Model. 3. RESULTS AND DISCUSSION Unidimensionality clarifies whether items are measuring a single latent variable. In Rasch, such analysis is shown by Principal Component Analysis (PCA). It is assumed that the measurement dimensions account for more variance in the data rather than other dimensions. If there are signs of other dimensions, Rasch indicates items which are potentially contributing to another dimension, thus the results direct the researcher to further investigation either to drop or retain a particular item. However, Linacre (1998) posits that extracting another dimension based on PCA results is provisional. In this study, PCA is performed to check whether the items are measuring value application in quality management practice. According to (Bond & Fox, 2007; Aziz et al., 2013), unidimensionality may be detected using PCA, via two indicators; the Raw Variance Explained by Measures (RVEM) and Unexplained Variance in First Contrast (UVFC). The rule of thumb according to (Aziz et al., 2013), a unidimensional instrument requires a high editor@iaeme.com

4 Amal Hayati Ishak, Mohd Hapiz Mahaiyadin and Mohd Amzari Tumiran percentage of RVEM, minimum 40%. As for the UVFC (Aziz et al., 2013). A value below 15% explains that there was no substansial and meaningful secondary dimension in the data. The data in this study shows that the RVEM was substantial at 49.5% (above the minimum value of 40%), which indicates a strong principal measurement dimension. The UVFC of 5.2% was below the ceiling value of 15%, indicating non presence of secondary dimension. Table 1 summarises the findings. Indeed, a value of 5%-10% is considered as good by (Aziz et al., 2013). Thus, both results supported the undimensionality of IQMS, i.e the pshychometric properties of IQMS measures what they intended to measure. Table 1 Principal Component Analysis (PCA) Measures Value Raw Variance Explained by Measures (RVEM) 49.5% Unexplained Variance in First Contrast (UVFC). 5.28% 4. CONCLUSIONS All the items are in unidimension and intended to measure a similar latent trait. This has been approved from the Principal Component Analysis of Rasch as the measurements of RVEM and UVFC satisfies the rule of thumb uphold by the Rasch proponents (Bond & Fox, 2007; Aziz et al., 2013). The finding supports the reliability of the IQMS. Therefore, methodologically, Rasch Model is an appropriate and beneficial tool in developing a robust, precise and concise measurement tool. ACKNOWLEDGEMENTS The authors felt thankful to the support from the Institute of Research Management of Innovation (IRMI) of Universiti Teknologi MARA (UiTM). REFERENCES [1] Ahmad, Khaliq. "Quality management foundation: An agenda for Islamization of management knowledge." Malaysian Management Review 31, no. 1 (1996): [2] Abulhasan, M. Sadeq, and A. Khaliq Ahmad. "Quality management: Islamic perspective." Kuala Lumpur: Leeds Publication (1996). [3] Albano Jr, Joseph F. Developing a measure and an understanding of the individual experience of happiness at work. Saybrook University, [4] Al-Buraey, M. "Management principles derived from the sources of Islam." Management Principles Derived from the Sources of Islam in Quality Standard from The Islamic Perspective, Institute Kefahaman Islam Malaysia, Kuala Lumpur (2005): [5] Apple, Matthew T. "Using Rasch analysis to create and evaluate a measurement instrument for foreign language classroom speaking anxiety." JALT journal 35, no. 1 (2013): [6] Aziz, Azrilah Abdul, Mohd Saidfudin Masodi, and Azami Zaharim. Asas model pengukuran Rasch: pembentukan skala dan struktur pengukuran. Penerbit Universiti Kebangsaan Malaysia, editor@iaeme.com

5 Examining Unidimensionality of Psychometric Properties Via Rasch Model [7] Baird, Kevin, Kristal Jia Hu, and Robert Reeve. "The relationships between organizational culture, total quality management practices and operational performance." International Journal of Operations & Production Management31, no. 7 (2011): [8] Basir, Siti Arni, and Ilhaamie Abd Ghani Azmi. "Malaysian Islamic Quality Management System MS1900 from an Islamic Perspective: An Implementation Model." Jurnal Syariah 19, no. 2 (2011). [9] Basir, Siti Arni, Bharudin Che Pa, and Raja Hisyamudin Raja Sulong. "Suntikan nilainilai Islam ke atas perlaksanaan penambahbaikan berterusan dalam konteks sistem pengurusan kualiti (SPK) ISO 9000." Jurnal Syariah 18, no. 1 (2010): [10] Bechtel, Gordon G. "Generalizing the Rasch model for consumer rating scales." Marketing Science 4, no. 1 (1985): [11] Bond, Trevor G., and Christine M. Fox. Applying the Rasch model: Fundamental measurement in the human sciences. Psychology Press, [12] Chou, Yeh-Tai, and Wen-Chung Wang. "Checking dimensionality in item response models with principal component analysis on standardized residuals." Educational and Psychological Measurement 70, no. 5 (2010): [13] Denison, Daniel R., and Gretchen M. Spreitzer. "Organizational culture and organizational development: A competing values approach." Research in organizational change and development 5, no. 1 (1991): [14] Detert, James R., Roger G. Schroeder, and Robert Cudeck. "The measurement of quality management culture in schools: development and validation of the SQMCS." Journal of Operations Management 21, no. 3 (2003): [15] Detert, James R., Roger G. Schroeder, and John J. Mauriel. "A framework for linking culture and improvement initiatives in organizations." Academy of management Review 25, no. 4 (2000): [16] DeVellis, Robert F. Scale development: Theory and applications. Vol. 3. Sage publications, [17] Embretson, Susan E., and Steven Paul Reise. "Item response theory for psychologists. Maheah." (2000). [18] do Nascimento Gambi, Lillian, Mateus Cecílio Gerolamo, and Luiz Cesar Ribeiro Carpinetti. "A theoretical model of the relationship between organizational culture and quality management techniques." Procedia-Social and Behavioral Sciences 81 (2013): [19] Ishak, Amal Hayati, Muhamad Rahimi Osman, and Ghafarullahhuddin Din. "Expert consensus on dimensions of islamic values in quality management practice: Analysis of fuzzy delphi method." Turkish Online Journal of Educational Technology 2015 (2015): [20] Ishak, Amal Hayati, Muhamad Rahimi Osman, Siti Khadijah Ab Manan, and Rafeah Saidon. "Rasch Model Scale Calibration Analysis for Islamic Value." International Review of Management and Marketing 6, no. 7S (2016b): editor@iaeme.com

6 Amal Hayati Ishak, Mohd Hapiz Mahaiyadin and Mohd Amzari Tumiran [21] Ishak, Amal Hayati, Muhamad Rahimi Osmanaa, Siti Khadijah Ab Manana, Rafeah Saidona, and Ghafarullahhuddin Dina. "Investigating Rating Scale Design via Rasch Measurement Model: Analysis on a Scale to Assess Islamic Values Application in Quality management Context." (2016a). [22] Ishak, Amal Hayati, and Muhamad Rahimi Osman. "A systematic literature review on Islamic values applied in quality management context." Journal of business ethics 138, no. 1 (2016): [23] Ishak, A.H. et al. Investigating Item Redundancy via RASCH Model: Analysis on Islamic Values Scale. International Journal of Advanced Biotechnology and Research, 8, no.3 (2017): [24] Linacre, John Michael. "Detecting multidimensionality: which residual data-type works best?." Journal of outcome measurement 2 (1998): [25] Mohd Mokhtar, Sany Sanuri, Rushami Zien Yusoff, Zakaria Abas, Hartini Ahmad, Md Hussain, and Muhammad Nasri. "Aplikasi sistem pengurusan kualiti dari perspektif Islam." (2011). [26] Prajogo, Daniel I., and Christopher M. McDermott. "The relationship between total quality management practices and organizational culture." International Journal of Operations & Production Management 25, no. 11 (2005): [27] Prajogo, Daniel I., and Christopher M. McDermott. "The relationship between multidimensional organizational culture and performance." International Journal of Operations & Production Management 31, no. 7 (2011): [28] Salzberger, Thomas, and Monika Koller. "Towards a new paradigm of measurement in marketing." Journal of business research 66, no. 9 (2013): [29] Singh, Jagdip. "Tackling measurement problems with Item Response Theory: Principles, characteristics, and assessment, with an illustrative example." Journal of Business Research 57, no. 2 (2004): [30] Spector, Paul E. "Summated Rating Scale Construction: An Introduction Sage." Newbury Park, CA (1992). [31] Tennant, Alan, and Philip G. Conaghan. "The Rasch measurement model in rheumatology: what is it and why use it? When should it be applied, and what should one look for in a Rasch paper?." Arthritis Care & Research 57, no. 8 (2007): [32] Wright, B. D. "Local dependency, correlations and principal components." Rasch Measurement Transactions 10, no. 3 (1996): editor@iaeme.com