Research Note. Community/Agency Trust: A Measurement Instrument

Transcription

1 Society and Natural Resources, 0:1 6 Copyright # 2013 Taylor & Francis Group, LLC ISSN: print= online DOI: / Research Note Community/Agency Trust: A Measurement Instrument JORDAN W. SMITH Department of Forestry and Natural Resources, Purdue University, West Lafayette, Indiana, USA JESSICA E. LEAHY School of Forest Resources, University of Maine, Orono, Maine, USA DOROTHY H. ANDERSON Department of Parks, Recreation and Tourism Management, North Carolina State University, Raleigh, North Carolina, USA MAE A. DAVENPORT Department of Forest Resources, University of Minnesota, St. Paul, Minnesota, USA Many natural resource management agencies invest considerable time and financial resources into building relationships with their constituents. Theoretically, the building of trust produces a relationship that leads to socially acceptable planning and positive management outcomes. Despite the central role trust plays in natural resource management, empirical attempts to measure the construct have been limited. This research note presents the development and validation of a psychometric instrument intended to measure the trust held by local community members living adjacent to managed natural resource areas. The instrument is based in current theory, and exhibits reliable and valid psychometric properties when applied to different study populations. Our intention is to provide an accepted instrument through which knowledge regarding the unique dimensions of community=agency trust, and the entire trust construct as a whole, can be furthered. Keywords community agency relationships, scale development, social capital Despite the central role trust plays in natural resource management, empirical attempts to measure the construct have been limited. We present the development and validation of a psychometric instrument intended to measure the trust held by Received 3 May 2011; accepted 7 August Address correspondence to Jordan W. Smith, FORS201A, 195 Marsteller St., West Lafayette, IN 47907, USA. smit1547@purdue.edu 1

2 2 J. W. Smith et al. local community members living adjacent to managed resource areas. The instrument identifies and discriminates among multiple dimensions of trust in community=agency relationships. Methods We developed the community=agency trust instrument through a four-step process involving theoretical definition of the construct, identification of potential sub dimensions, the creation of measurement items, and the empirical evaluation of a measurement model (Bollen 1989). Theoretical definitions of trust dimensions were developed from the literature and 31 interviews with individuals living within the Kaskaskia River Watershed (KRW) (Leahy and Anderson 2008). Following analysis of the interview data, 22 statement items intended to measure 5 distinct dimensions of trust were developed. A subsequent mail survey was administered to three samples of residents living within the KRW (Smith et al. 2011) and two samples of residents living near Voyageurs National Park (Anderson et al. 2006). Analysis of the instrument s construct validity is conducted using data collected from these five samples. Construct Validity Construct validity is the extent to which a set of variables effectively measure the constructs they are supposed to measure; it is primarily comprised of reliability, convergent validity, and discriminant validity (Kline 2011). Reliability Vetting of all 22 measurement items for acceptable distributions (skewness and kurtosis between þ2 and 2) using pooled data from all five samples led to an elimination of six items. Cronbach s alpha values and reliability coefficients for each dimension of trust (Table 1) were above the recommended.70 level (Nunnally and Bernstein 1994). Reliability coefficients measure the proportion of an item s variance explained through its intended latent factor; the higher the coefficient, the more reliable is the measure (Kline 2011). All reliability coefficients were above.43. Given that the existing literature does not suggest a cutoff value for reliability coefficients, we deemed these values acceptable. These results suggest each dimension of trust in the instrument is being measured through reliable item sets. Convergent Validity Convergent validity was assessed through three tests: adequate factor loadings, acceptable model fit, and measurement invariance. All factor loadings for the measurement items were acceptable (i.e., >0.60; Hair et al., 2010). Model fit was assessed through a confirmatory factor analysis (using maximum likelihood estimation and replacing missing values with variable means 1 ) with the following fit statistics: relative v 2, root mean square error of approximation (RMSEA), comparative fit indices (CFI), and Tucker Lewis Indices (TLI). Values of 3.0 or less indicate an acceptable relative v 2 (Carmines and McIver 1981). RMSEA values less than 0.06 indicate an acceptable fit (Hu and Bentler 1999). Values of 0.90 or greater for both the CFI and TLI indexes also provide support that the data fit the theoretical model (Hu and Bentler 1999). The relative v 2 value estimated with the pooled sample (2.84) indicates acceptable model fit (Table 2). Also, the RMSEA, CFI, and TLI values

3 Community/Agency Trust Instrument 3 Table 1. Descriptive statistics Trust dimensions and item measurements M SD Cronbach s a Squared multiple correlations (mc 2 ) Unstandardized factor loadings (k) Dispositional trust:.76 You can t be too careful dealing with people People are almost always interested only in their own welfare One has to be alert or someone is likely to take advantage of you Trust in federal government:.85 I feel connected to the U.S government The federal government efficiently spends money The U.S. government is effective in solving problems I can trust the federal government to do what is right most of the time Shared values:.95 The agency supports my views The agency has similar goals to mine The agency thinks like me Moral competency:.82 Employees are not self-serving in decision making Managers really care what happens to me Employees are sensitive to the local impacts of tourism and recreation Technical competency:.86 Employees are well trained Employees are knowledgeable about technical matters I have confidence in agency employees to manage this area well Note. Statements were measured with a 5-point Likert-scale ranging from 1 (strongly disagree) to5(strongly agree).

4 4 J. W. Smith et al. Table 2. Fit statistics Confirmatory factor analysis v 2 df v 2 =df RMSEA [90% CI] CFI TLI Study area (s): Pooled Sample [ ] Voyageurs National Park Study International Falls (n ¼ 313) [ ] Other communities (n ¼ 297) [ ] Kaskaskia River Watershed Study Lake Shelbyville (n ¼ 213) [ ] Carlyle Lake (n ¼ 233) [ ] Navigation Project (n ¼ 201) [ ] Multigroup invariance tests: Multigroup configural model [ ] Multigroup test of measurement invariance (constrained measurement weights, intercepts, and residuals) [ ] Constrained correlation tests Dv 2 CFI DCFI Dispositional Trust Technical Competency Dispositional Trust Moral Competency Dispositional Trust Shared Values Dispositional Trust Trust in Fed. Govt Technical Competency Moral Competency Technical Competency Shared Values Technical Competency Trust in Fed. Govt Moral Competency Shared Values Moral Competency Trust in Fed. Govt Shared Values Trust in Fed. Govt Note. RMSEA ¼ root mean square error of approximation; CFI ¼ comparative fit index; TLI ¼ Tucker Lewis Index. Constrained correlation tests involved specifying factor correlations to be 1. Significance indicated by p.001; p.01; p.05. provide further support that the hypothesized model fits the data well. The final test for convergent validity required reestimation of the model with the additional specification of distinct samples. Multigroup estimation produces a configural model with summed v 2 statistics and reestimated fit indices (Table 2). The configural model provides a baseline against which models with specific equality constraints are compared. Constraining measurement weights, intercepts, and residuals to be equal across groups tests for measurement invariance, whether or not the instrument is measuring the same latent constructs across groups. Measurement invariance is supported if the DCFI between models is less than 0.01 (Cheung and Rensvold 2002). Test statistics (Table 2) reveal a DCFI less than 0.01, suggesting the same constructs

5 Community/Agency Trust Instrument 5 Table 3. Factor correlations for pooled data Factor Factor Dispositional Trust 2. Trust in Federal Government Shared Values Moral Competency Technical Competency are being measured across samples. Collectively, these tests demonstrate support for the convergent validity of the instrument. Discriminant Validity Discriminant validity was assessed through two tests: analysis of the factor correlations, and tests for significantly poorer model fit when individual correlations are constrained. Low factor correlations (<0.80) between latent constructs suggest discriminant validity (John and Benet-Martinez 2000). Analysis of factor correlations supported the instrument s discriminant validity (Table 3). The test for decrement in model fit with constrained factor correlations (v 2 difference test) compares the baseline model to an alternative where correlation estimates are constrained to A significantly higher v 2 value and notable decrements in the CFI for the constrained model support discriminant validity (Kline 2011). Each correlation parameter is tested. All v 2 difference tests (Table 2), conducted on the pooled data set, were significant, providing further support for discriminant validity. Conclusion The community=agency trust instrument is based in current theory and has exhibited reliable and valid psychometric properties when applied to different samples. The instrument enables scientists and managers to target specific factors, such as feelings toward moral and technical competencies, which may be hindering planning and management processes. It would be a valuable addition to a variety of agency planning processes such as social assessments and social impact analyses. Note 1. Variable mean replacement is acceptable when the proportion of missing data is low (<10%) or when correlations between variables are low (Kline 2011; Roth 1994). The missing data rate for all 5 data sets used in this study was 1.4%. Also, imputation and subsequent analysis were conducted with the AMOS 17 software package. References Anderson, D. H., J. L. Thompson, and J. M. Schertz Voyageurs National Park 2005 community trust study. St. Paul: University of Minnesota. prod/groups/cfans/@pub/@cfans/@cesu/documents/asset/cfans_asset_ pdf Bollen, K. A Structural equations with latent variables. New York, NY: Wiley.

6 6 J. W. Smith et al. Carmines, E. G., and J. P. McIver Analyzing models with unobserved variables. In Social measurement: Current issues, ed. G. W. Bohrnstedt and E. F. Borgatta, Beverly Hills, CA: Sage. Cheung, G. W., and R. B. Rensvold Evaluating goodness-of-fit indexes for testing measurement invariance. Struct. Equation Model. 9:233. Hair, J. F., W. C. Black, B. J. Babin, and R. E. Anderson Multivariate data analysis, 7th ed. Upper Saddle River, NJ: Prentice Hall. Hu, L., and P. M. Bentler Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct. Equation Model. 6:1 55. John, O. P., and V. Benet-Martinez Measurement: Reliability, construct validation, and scale construction. In Handbook of research methods in social and personality psychology, ed. H. T. Reis and C. M. Judd, London, UK: Cambridge. Kline, R. B Principles and practice of structural equation modeling, 3rd ed. New York, NY: Guilford. Leahy, J. E., and D. H. Anderson Trust factors in community water resource management agency relationships. Landscape Urban Plan. 87: Nunnally, J. C., and I. H. Bernstein Psychometric theory, 3rd ed. New York, NY: McGraw-Hill. Roth, P. L Missing data: A conceptual review for applied psychologists. Personnel Psychol. 47: Smith, J. W., M. A. Davenport, D. H. Anderson, and J. E. Leahy Place meanings and desired management outcomes. Landscape Urban Plan. 101: