Meso-level institutional and journal related indices for Malaysian engineering research

Transcription

1 DOI /s Meso-level institutional and journal related indices for Malaysian engineering research Muzammil Tahira 1 Rose Alinda Alias 2 Aryati Bakri 2 A. Abrizah 1 Received: 13 July 2015 Akadémiai Kiadó, Budapest, Hungary 2016 Abstract The literature considers a journal s h-index as a compared measure, however the relation between institutional level h-index (IHI) and journal related indices (JRI) has not been explored in previous studies at meso-level research assessment. This study applies the scientometric approach to meso-level data to examine the association, functional relationship and correlation analysis to evaluate the reliability of IHI with respect to JRI. For this purpose, data from the Web of Science, journal citation report and time cited features were used. The unit of analysis was Malaysian engineering research with a wider time span of 10 year s data ( ) and a larger set of journals (1381). We explored the inter-correlation of IHI with a set of eight JRI and applied principle component analysis, regression analysis, and correlation. At the institutional level, the component analysis and functional relationship of the cumulative impact factor and 5 years IF yielded a more strong association with IHI. Cumulative impact factor is a strong predictor for IHI followed by cumulative 5 years impact factor. Correlation matrix results show that average impact factor (AIF) is correlated with immediacy index and Eigenfactor only. AIF and median impact factor (MIF) have no correlation with each other and IHI is correlated with all indices except AIF and MIF. This study puts forward a better understanding in considering new impact indices at meso level for performance evaluation purpose. & A. Abrizah abrizah@um.edu.my Muzammil Tahira mufals@yahoo.com Rose Alinda Alias alinda@utm.com Aryati Bakri aryatib@utm.com 1 2 Department of Library and Information Science, Faculty of Computer Science and Information Technology, University of Malaya, Kuala Lumpur, Malaysia Department of Information System, Faculty of Computing, Universiti Teknologi Malaysia (UTM), Skudai, Johor Bahru, Malaysia

2 Keywords Journal related indices Institutional h-index Institutional ranking Research performance evaluation Engineering research Malaysia Introduction Journal impact factor (JIF) introduced by the Institute of Scientific Information (ISI) by means of the journal citations report (JCR) has a long tradition as an impact factor (IF) indicator for scholarly research output. IF and its variants have been introduced and displayed on JCR site ( as a measure of research quality or impact of journals. In general research performance evaluation (RPE) practices, IF has become a chief quantitative measure of the quality of a journal, its research papers, the researchers who wrote those papers and even the institution they work in (Amin and Mabe 2000 p. 2) however, it cannot be used as a direct measure of quality (Bornmann et al. 2011). Although, IF remains the primary criteria when it comes to assessing the quality of journals and authors (Raj and Zainab 2012), it should not be used as a sole measure of a journal rank (Bornmann et al. 2011). To overcome the limitations of IF, researchers suggested that it could be used with new alternative tools (Braun et al. 2006; Borrnmann et al. 2011; Prathap 2011; Yin 2011) or as a measure of research quality or impact of journals (Braun et al. 2006). The notion of journal s h-index was introduced by Braun et al. (2005), who later found that h-index could be used as a measure of research quality or impact of a journal (Braun et al. 2006). After the introduction of h-index, a number of studies made a comparative analysis of both impact indices (h-index and IF) and their variants. These indices are easily comprehensible (Leydesdorff 2009) and have received worldwide recognition. However, the previous studies, as reviewed in the subsequent paragraphs were concerned with the evaluation of journal s h-index to journal related indices (JRI). Mingers et al. (2012) examined journal level h-index against impact factor (IF), 5-year impact factor IF(5y) and peer judgment for management journals. They preferred journal h-index to IF because of the former s selective time frame and the formulaic problem. Another journal study in the field of marketing was carried out by Moussa and Touzani (2010) using Google-Scholar (GS) as the source data. They used a variant of h-index, i.e. the hg-index along with 2 and 5 years IF. There was a substantial agreement found ([0.85) between IF5y and the hg-index ranking. They suggested hg-index as an alternative to the GS-based journals. Soutar and Murphy (2009) studied 40 marketing journals and ranked them according to IF and h-index, and compared their list with the Australian journal ranking. They suggested these indices as the basis for moving some journals up and other journals down in the list. Their study supported the use of GS as an alternative way to measure citations in the field of marketing. Harzing and Van der Wal (2009) compared the h-index generated from the citation in GS with the impact factors computed from the Web of Science (WoS TM ) of over 800 business and management journals. They concluded that the GS h-index provided a more accurate and comprehensive measure of journal impact, as other studies indicated that WoS TM did not have a good coverage across all subject areas (Abrizah et al. 2013; Tahira et al. 2012; Gavel and Iselid 2008; Hicks and Wang 2011; Mikki 2010). A comparative analysis of IF and h-index was carried out by Bador and Lafouge (2010) on pharmacology and psychiatry journals from JCR with 2-year publications. The journals

3 correlation coefficient between IF and h-index was high. They inferred that IF and h-index could be totally corresponding when analyzing journals of the similar scientific subject. Bornmann et al. (2009) studied the journal s h-index of twenty organic chemistry journals from WoS TM database for 2 year-time span. They analyzed a number of impact indicators including IF and journal s h-index and its variants g index, h 2 index, A, and R index, and found a high degree of correlation between the various measures. Combining the h-index with another measure in JCR i.e. the Eigenfactor, Yin et al. (2010) analyzed top journals in the field of science and engineering using data from WoS TM. They provided graphical plots of the relative positions of these journals and concluded that the two indices measured slightly different characteristics. In another study, Yin (2011) analyzed 20 top journals in chemical engineering also using data from WoS TM comparing journal h-index and Eigenfactor score. The researcher hypothesized that the combination of complementary journal indicators could provide a simple, flexible and practical alternative approach for evaluating scientific journals (p. 2). There is a positive correlation, although not strong among these indices and this indicates that the two indices can also be combined to complement each other. Yin suggested authors to get their research work published in high Eigenfactor scores journals. The objective of past studies was to evaluate a journal s h-index validity and reliability with respect to other JRI. Most of these studies considered journal s h-index as a contrasted measure with JCR indices such as IF, IF(5Y), and Eigenfactor score. These studies are meaningful to understand the properties of newly introduced indices (Tahira et al. 2014) and potential use of journal s h-index as a complement aid with IF and its variants (Bador and Lafouge 2010; Bornmann et al. 2012; Yin 2011) or, as a supplement (Braun et al. 2006). These all studies are limited in various ways, especially concerning the relation of JRI and institutional level h-index (IHI). To have a further evidence of the reliability of h-index at the institutional level communicated in Tahira et al. (2015) and a set of impact indices (including total citation and JRI), we have hypothesized that IHI is a potential index for RPE that can be used as a complement or as a supplement for RPE at the institution level. Objectives and method The objective of the research is to evaluate the reliability of Institutional level h-index (IHI) with respect to other journal related indices (JRI). We examined the intercorrelation by exploratory factor analysis (EFA). In addition, we have explored the functional relationship by regression analysis and checked the correlation among IHI and a set of impact indices to evaluate its reliability. The empirical part of this study focuses on one non- Western country, Malaysia, which has a developed and well-defined scholarly publishing industry based in its universities. Research productivity, citations record, and institutional journal data of twelve Malaysian universities were retrieved from WoS TM and JCR The universities having at least 50 publications (journal articles) during the past 10 years were selected. A total of eleven indices were used for the present study at the meso level. The indices are total publications (TP), total citations (TC) citation per publications (CPP), Institutional h-index (IHI), IF, cumulative impact factor (CIF), 5 years journal impact factor IF(5y), cumulative 5 years journal impact factor CIF(5y), average impact factor (AIF), median impact factor (MIF), immediacy-index (Imm-index) and Eigenfactor score (EF).Their definitions and the acronyms used are described in Table 1.

4 Table 1 The journal related indices used at meso level Journal related indicators (JRI) Definition 1 Total publications (TP) Total publications of a university over the set criteria 2 Total citations (TC) Total citations of a university over the set criteria 3 Institutional H-index (IHI) An institution has index h if h of institutional publication has at least h citation each and other publication have fewer than or equal to h citations each 4 Impact factor (IF) The average number of times articles from the journal published in the past 2 years has been cited in the JCR year (Thomson-Reuters 2015) 5 Cumulative journal impact This is the cumulative value of journal impact factor of each university factor (CIF) 6 Impact factor 5 years IF(5y) The average number of times articles from the journal published in the past 5 years has been cited in the JCR year (Thomson-Reuters 2015) 7 Cumulative l impact factor 5 years CIF(5y) This is the cumulative value of 5 years journal impact factor of each university. 8 Average impact factor (AIF) This is the average of the impact factor of each university 9 Median impact factor (MIF) This is the median of the impact factor of each university 10 Immediacy-index (Immindex) This is calculated by dividing the number of citations to articles published in a given year by the number of articles published in that year (Thomson-Reuters 2015) 11 Eigenfactor score(ef) Eigen factor score is calculated by the ratio of the total number of citations for the JCR year to the total number of articles published in the last 5 years. (Thomson-Reuters 2015) To get a meaningful evaluation, we used a set of WoS TM engineering journals (1381 journals with 68 unique titles) published by 12 Malaysian universities with a wider horizon of 10 years ( ) under specified nine categories. Malaysia s scholarly publication strength lies in the field of engineering and energy. The search term used was Malaysia in address, limited to document type research article and reviews only, and refined by nine engineering research categories. These nine engineering categories are: electrical and electronic, manufacturing, biomedical, industrial, civil, chemical, mechanical, environmental and multidisciplinary. An overview of data Data were suffered from affiliation problem, change of journal title and abbreviation of a journal name. The data were checked manually for publications, citations, institutional affiliation, and journal name change or emergence cases. Almost all journals in our data set had impact-factor, except for 22 articles published in six non-if journals. These data were included in the journal list for analysis purpose. All records were retrieved in a spreadsheet file, and IBM statistical product and service solutions was used for statistical analysis. Table 2 presents the university-wise journal contribution in the journals. The publication share of research-intensive universities (RU) was 66 % (908) while the non-ru status universities shared 34 % (473) of the total journals. The RU universities are more bound to published in IF journals to get more research funding. These five universities receive a big amount of budget as research grants for research and development (R&D) purposes, and

5 Table 2 Distribution of journals by universities (N = 1381) No University Total journals University status Contribution 1 University of Malaya (UM) 191 Research Universities 66 2 Universiti Sains Malaysia (USM) journals 3 Universiti Putra Malaysia (UPM) Universiti Teknologi Malaysia (UTM) Universiti Kebangsaan Malaysia (UKM) Universiti Teknologi Mara (UiTM) 87 Non-Research 34 % 7 University of Multimedia (MMU) 81 Universities 473 Journals 8 Universiti Teknologi PETRONAS (UTP) 78 9 International Islamic Universiti Malaysia (IIUM) University of Nottingham Malaysia Campus 61 (UNMC) 11 MONASH Universiti Sunway Campus 51 (MONASH) 12 Universiti Tenaga Nasional (UNITEN) 38 Total have to face the publication pressure to build an international reputation in scholarly publication, and this is especially prevalent among the Asian countries (Leydesdorff 2009). Since their research university status in 2007, the country s research activities and outputs have shown a tremendous increase, which proves that higher learning institutions in Malaysia can be at par with reputable universities in the world. The first five public universities (RU) have published in journals. Comparatively, the non-rus had fewer publications and published in less than 100 journals. The average number of journals publishing articles affiliated to RU and non-ru universities is 182 and 68 respectively. The statistical methodology of exploratory factor analysis (EFA) was used to examine for latent associations present in a set of observed variables, and reduce the dimensionality of the data to a few representative factors (Schreiber et al p. 349). It is mainly used to identify a smaller set of salient variables from a larger set and to explore the underlying dimensions or factors that explain the correlations among a set of variables (Conway and Huffcutt 2003). Analysis and findings In a tie with the problem, this section proceeds accordingly with descriptive statistics, data normality and EFA for the set of indices used as presented in Appendix. Descriptive statistics along with Skewness and Kurtosis are presented in Table 3. The Skewness and Kurtosis are valid tests to find the normality of data. Their values show a normal distribution of data. The results of the normality test based on raw data (excluding outliers) are reported in Table 4. Keeping in view the requirement of EFA statistical application, we have used two other options as well. We also examined the relation between the raw data (X), logarithmically transformed shifted (ln(x? 1) and square root transformation data (Hx). Table 5 shows a better Kaiser Meyer Olkin (KMO) results and a slightly better explained variance for log data. For this reason, we found that the logarithmic transformed

6 Table 3 Descriptive statistics, Skewness and Kurtosis tests Indices Descriptive statistics Skewness Kurtosis Mean St.dev Median Min Max TP TC IHI IF CIF MIF AIF IF(5Y) CIF(5Y) Imm-index EF Table 4 Test for normality of data * At a 5 % significance level Indices Kolmogorov Smirnov Shapiro Wilk Statistic Df Sig. Statistic Df Sig. TP TC IHI * IF CIF AIF * MIF * IF(5Y) CIF(5Y) Imm-index EF * Table 5 Kaiser Meyer Olkin (KMO) measure of sampling adequacy X Hx ln(x? 1) KMO Sig data is more adequate for EFA. Bornmann et al. (2008, 2009) used a cut-off threshold[0.6 for extraction loading factors while Schreiber et al. (2012) fixed it at [0.685 for Varimax rotation. Schreiber et al. (2012) argued that small sample size for EFA could produce reliable results. Quite a few factors and high communalities are in favour of small sample sizes (Preacher and MacCallum, 2002). Furthermore, to measure a sampling adequacy, a specific

7 Table 6 Communalities for 3 EFA models Indices X Hx ln(x? 1) Initial Extraction Initial Extraction Initial Extraction TC IHI IF CIF IF(avg) MIF IF(5Y) CIF(5Y) Imm-index EF test Kaiser Meyer Olkin (KMO) of value[0.5 is acceptable (Kaiser 1974). KMO value of the present data sample is[0.5 (Table 5) with high communalities ([0.85) (Table 6). Based on KMO values and variance explained (Tables 5, 7), we finally utilized logarithmically transformed data. We identified two unknown factors through Eigenvalues ([1) via variance explained. This is evident that EFA can be used and is appropriate for our formulated problem and data set. Initially, we considered eleven indices, TP, TC, IHI and 8 of JRI (JIF, CIF, AIF, MIF, IF (5y), CIF(5Y), Imm-Index, and EF). This set of indices produced inadequate results for EFA. After omitting the TP, we applied EFA to TC, IHI, and 8 JRI. Table 5 reports the results of KMO values of the transformed data for the appropriateness of factor analysis. Table 6 reveals the results of communalities for 3 EFA models that are the variance in observed variables accounted for by a common factor (Hatcher 1994). Table 7 provides initial Eigenvalues [1 and indicates that the total variance explained by first two factors is 75 and 17 % while cumulative variance explained by both factors is 91 %. Scree plot (Fig. 1) and component matrix (Table 8) illustrate that the set of indices clearly loads on two extracted factors. Rotated Component Matrix (Table 9) for EFA model shows that the indices have substantial loading on two established factors. It indicates the loading of two institutional impact of the productive core indices (TC and IHI) and six others JRI have high loading ([0.90) and a slightly less for EF ([0.891). AIF and MIF both have substantially high loading on the second factor [0.9. MIF is more accurate measure than the average value, due to the impact factor s skewed distribution (Costas and Bordons 2007). IF and CIF and IF(5y) and CIF(5y) require 2 and 5 years time span with different strengths of productivity. EF is another index based on 5-year data excluding journal self-citation to rate the total importance of the journal. Journals generating a higher impact on the field have larger Eigenfactor scores (Bergstrom 2007). Yin et al. (2010) has found that EF improves upon JIF and somewhat robust indicators of quality and prestige of the journal due the inclusion of 5 year s data, exclusion of journal self-citations (p. 3). Rather a high journal EF depicts producing of high-impact scientific findings in a specific area (Yin et al. 2010). IF(5y) indicates the speed with which citations to a specific journal appears in the published literature. Immediacy index that is based on 1-year data shows the same value as CIF on the first factor. They both require a different strength of data. Surprisingly they all loaded on the same factor along with IHI.

8 Table 7 Total variance explained for 3 EFA models Data type Component Initial Eigenvalues Extraction sums of squared loadings Rotation sums of squared loadings Total % of variance Cumulative % Total % of variance Cumulative % Total % of variance Cumulative % Raw indices Hx ln(x? 1) Extraction method: principal component analysis Eigenvalues [1 are reported only

9 Fig. 1 Scree plot of the EFA model Table 8 Component matrix Indices Components 1 2 Extraction method: principal component analysis TC IHI IF CIF IF(avg) MIF IF(5Y) CIF(5Y) Imm-index EF Functional relationships and predictive values of indices We tested the regression relation of JRI indices uploaded with IHI on the first component. To examine the functional relationship, if any, and their predictive value, we seek IHI model fit and power law relationship (Fig. 2a h). The results reveal that IHI has a good model fit with IF (R 2 = 0.779), IF(5y) (R 2 = 0.792) and with cumulative indices, and this fit is improved for CIF (R 2 = 0.867) and CIF(5y) (R 2 = 0.831). This relation is also found good for both immediacy index and EF (R 2 = and respectively) while not good for the cases of AIF and MIF (R 2 = 0.16 and.0017). In this case CIF is a strong predictor for IHI followed by CIF(5Y). The power regression trends between IHI and IF, CIF, IF(5Y), CIF(5Y) are IHI = 0.32IF0.75, 0.56CIF0.53, 0.28IF (5y) 0.76 and 0.53CIF (5y) 0.54 respectively. The

10 Table 9 Rotated component matrix Indices Components 1 2 Extraction method: principal component analysis Method: Varimax with Kaiser normalization Values [0.5 are highlighted in bold TC IHI IF CIF AIF MIF IF(5Y) CIF(5Y) Imm-index EF Eigenvalues Variance explained 73 % 18 % relation between IHI and new JRI are found to be 1.39 imm-index and 11.54EF0.67. The above values express that the square of IHI is approximately the multiple of CIF and CIF(5Y). At the institutional level, the component analysis and functional relationship of cumulative indices resulted in a much stronger association with IHI (Table 10). Correlation analysis between IHI and JRI To seek the correlation among our set of indices between IHI, TC, and JRI indices, we checked spearman two tail correlation. TC, IF, CIF, IF(5y) CIF(5y), are correlated with all impact indices except AIF and MIF. AIF is correlated with imm-index and EF only. AIF and MIF have no correlation with each other. MIF has no relationship with any JRI, TC, Imm-index and EF. EF is also correlated with all indices except MIF. Discussions and conclusions The caveats of h-index, JIF, and traditional metrics have been discussed in the abundant literature. Previous studies are meaningful to understand the properties of newly introduced indices and potential use of journal h-index as a complement aid with IF and its variants (Bador and Lafouge 2010; Bornmann et al. 2012; Yin 2011) or, as a supplement (Braun et al. 2006). The present study describes the case of Malaysian engineering research by applying the scientometric approach, method and techniques for research performance evaluation (RPE). Based on the 10 years data analysis from WoS TM, we applied a set of comparatively new indices. To achieve the research objectives, empirical analyses were carried out, and the hypothesis that IHI is a potential index for RPE that can be used to complement or as a supplement along with JRI for RPE at the institution level was examined statistically. The major findings of the study demonstrate that there seems to be an increasing trend to get published in IF journals. A steady increase of IF publications is observed from 2001 in the Malaysians scientific productivity of engineering research. The desire to publish in

11 Fig. 2 a h Functional relationships between IHI and JRI

12 Table 10 Correlation matrix TC JIF CIF AIF MIF JIF5y JCIF5y Imm EF IHI TC 1.836**.977** **.970**.841**.795**.958** Sig JIF 1.893** **.867**.999**.985**.883** Sig CIF **.997**.900**.869**.931** Sig AIF *.596*.436 Sig JIF(5y) 1.894**.994**.992**.890** Sig CIF(5y) 1.876**.851**.912** Sig Imm-index 1.984**.882** Sig EF 1.844** Sig..001 IHI 1 Sig. * Correlation is significant at the 0.05 level (2-tailed) ** Correlation is significant at the 0.01 level (2-tailed) IF WoS TM recognized publications is reinforced by the Malaysian Research Assessment (MyRA) exercise, which requires institutions to publish articles that are indexed in global citation databases such as WoS TM and Scopus. This is due to Malaysian Ministry of Education s policies towards research and publications during two five years plans ( ; ). RU status universities shared 68 and 74 % of total publications and citations respectively. These universities have published in 66 % of the total journals involved in this study. Overall, the RU universities lead in positioning order with the application of indices. USM is an exceptional case and remained in position one with respect to almost all indicators while others show a noteworthy change in their positioning order except for UNITEN. Often used metric C (as total impact indicator), imm-index and the EF (as prestige indicator) have a high association with IHI. This establishes the property of h-index as prestige impact measure of scientific productivity. This index appears as a useful yardstick, because of good functional relationship with C and P and has shown some discriminatory power for the ranking purpose. The EFA suggests the same distinguishing behaviour of IHI and total citation. IHI has a stronger functional relation with cumulative IF for 2 and 5 years. Comparatively cumulative IF 2 and 5 years have better model fit than the IF, IF(5y), imm-index and EF. CIF is a strong predictor for IHI followed by CIF(5y). The results of correlation matrix indicate that total citations, institutional level h-index and journal related indices except AIF and MIF have a significant correlation. These two indices are not correlated with each other. MIF shows no correlation with any indices while AIF is correlated with imm-index and EF. EF is correlated with all except MIF.

13 The findings put forward a better understanding of considering the new impact metric for RPE at the meso level. The Malaysian engineering institutional case indicates that h-index and others metric have not only strong functional association for institutional cumulative journal indices but also with total institutional citation data and shows correlation as well. However, rotation loading of variance explained for two components yield about 73 % for its first component and 18 % for the second component. Therefore, findings are based within the limitations of the statistical analysis. Publishing in high-quality IF journals is important if a country is to realize its ambition to have its universities amongst the top rated universities in the world. This is not peculiar to Malaysia. The Ministry of Education Malaysia is targeting one research university in the country to be in Asia s top 25, and two in the world top 100 best universities by 2025 (Malaysia, 2015). Other countries also place a high emphasis on publishing in IF journals and would want to be ranked as top world universities, even if they are not always explicit in saying so. Given the significant number of papers that have now been published by Malaysian institutions (56,571 in Web of Science, Essential Science Indicators, Web of Science (2015), there is an opportunity to carry out further analysis. It would be interesting, for example, to provide analysis at a discipline level to get a feeling for the strengths of the institution at a lower level. It would also be informative to consider other normalization measures to ascertain if they provide a better correlation with the MyRA ranking. Acknowledgments The work of Muzammil Tahira and A. Abrizah was supported by the Ministry of Higher Education Malaysia (HIR-MOHE) UM.C/HIR/MOHE/FCSIT/11. Appendix See Table 11. Table 11 Analysis of complete dataset for institutional level indices applied University TP TC IHI JIF CIF AIF MIF JIF(5Y) JCIF(5Y) Immindex EF USM UPM UM UTM UKM UiTM IIUM MMU UNMCC UTP MONASH UNITEN

14 References Abrizah, A., Zainab, A. N., Kiran, K., & Raj, R. G. (2013). LIS journals scientific impact and subject categorization: A comparison between Web of Science and scopus. Scientometrics, 94(1), Amin, M., & Mabe, M. (2000). Impact factor: Use and abuse. Perspectives in Publishing, 1, 1 6. Bador, P., & Lafouge, T. (2010). Comparative analysis between impact factor and h-index for pharmacology and psychiatry journals. Scientometrics, 84(1), Bergstrom, C. (2007). Eigenfactor: Measuring the value and prestige of scholarly journals. College & Research Libraries News, 68(5), Bornmann, L., Marx, W., Gasparyan, A. Y., & Kitas, G. D. (2012). Diversity value and limitations of the journal impact factor and alternative metrics. Rheumatology International, 32, Bornmann, L., Mutz, R., & Daniel, H. D. (2008). Are there better indices for evaluation purposes than the h-index? A comparison of nine different variants of the h- index using data from biomedicine. Journal of the American Society for Information Science and Technology, 59(5), Bornmann, L., Mutz, R., & Daniel, H. D. (2009). Do we need the h-index and its variants in addition to standard bibliometric measures? Journal of the American Society for Information Science and Technology, 60(6), Bornmann, L., Mutz, R., Hug, S. E., & Daniel, H. D. (2011). A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants. Journal of Informetrics, 5(3), Braun, T., Glänzel, W., & Schubert, A. (2005). A Hirsch-type index for journals. The Scientist, 19(22), 8. Braun, T., Glänzel, W., & Schubert, A. (2006). A Hirsch-type index for journals. Scientometrics, 69(1), Conway, J. M., & Huffcutt, A. (2003). A review and evaluation of exploratory factor analysis practices in organizational research. Organizational Research Methods, 6(2), Costas, R., & Bordons, M. (2007). The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro-level. Journal of Informetrics, 1(3), Gavel, Y., & Iselid, L. (2008). Web of Science and scopus: A journal title overlap study. Online Information Review, 32(1), Harzing, A.-W., & Van der Wal, R. (2009). A google scholar h-index for journals: An alternative metric to measure journal impact in economics and business? Journal of the American Society for Information Science and Technology, 60(1), Hatcher, L. (1994). A step-by-step approach to using the SAS system for factor analysis and structural equation modeling. Cary, NC: SAS Institute Inc. Hicks, D., & Wang, J. (2011). Coverage and overlap of the new social sciences and humanities journal lists. Journal of the American Society for Information Science and Technology, 62(2), Kaiser, H. F. (1974). An index of factorial simplicity. Psychometrika, 39, Leydesdorff, L. (2009). How are new citation-based journal indicators adding to the bibliometric toolbox? Journal of the American Society for Information Science and Technology, 59(2), Malaysia, Ministry of Education. (2015). Malaysian education blueprint (higher education): Executive summary. Putrajaya, Malaysia: Ministry of Education. Mikki, S. (2010). Comparing google scholar and ISI Web of Science for earth sciences. Scientometrics, 82(20), Mingers, J., Macri, F., & Petrovici, D. (2012). Using the h-index to measure the quality of journals in the field of business and management. Information Processing and Management, 48, Moussa, S., & Touzani, M. (2010). Ranking marketing journals using the google scholar-based hg-index. Journal of Informetrics, 4, Prathap, G. (2011). Correlation between h-index, Eigenfactor TM and article influence TM of chemical engineering journals (letter). Current Science, 100(9), Preacher, K. J., & MacCallum, R. C. (2002). Exploratory factor analysis in behavior genetics research: Factor recovery with small sample sizes. Behavior Genetics, 32, Raj, R. G., & Zainab, A. N. (2012). Relative measure index: A metric to measure quality. Scientometrics, 93(2), Schreiber, Malesios, C. C., & Psarakis, S. (2012). Exploratory factor analysis for the hirsch index, 17 h-type variants, and some traditional bibliometric indicators. Journal of Informetrics, 6, Soutar, G. N. & Murphy, J. (2009). Journal quality: A Google analysis. Australasian Marketing Journal, 17, Tahira, M., Abrizah, A., Alias, R. A., Bakri, A. (2015). New research performance evaluation development and journal level indices at meso level. Paper presented in the 15th international conference on

15 scientometrics & informatics (ISSI 2015) held in Istanbul, Turkey on 29th June to 4th July. pp Tahira, M., Alias, R. A., Bakri, A., & Shabri, A. (2014). h-index, h-type Indices, and the role of corrected quality ratio. Journal of Information Science Theory and Practice, 2(4), Tahira, M., Alinda, R. A., & Bakri, A., (2012). Bibliometrics, reference enhanced databases and research evaluation in knowledge management international conference (KMICe) 2012, Johor Bahru, Malaysia, 4 6 July Thomson-Reuters. (2015). Web of Science TM. impfact.htm). Retrieved 20 Mar Yin, C. Y. (2011). Do impact factor, h-index and Eigenfactor TM of chemical engineering journals correlate well with each other and indicate the journals influence and prestige? Current Science, 100(5), Yin, C.-Y., Aris, M., & Chen, X. (2010). Combination of Eigenfactor and h-index to evaluate scientific journals. Scientometrics, 84(3),