HSIEN-YAO SWEE ALL RIGHTS RESERVED

Transcription

1 2009 HSIEN-YAO SWEE ALL RIGHTS RESERVED

2 A COGNITIVE PERSPECTIVE OF SELF-OTHER AGREEMENT: A LOOK AT OUTCOMES AND PREDICTORS OF SHARED IMPLICIT PERFORMANCE THEORIES A Dissertation Presented to The Graduate Faculty of The University of Akron In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy Hsien-Yao Swee August, 2009

3 A COGNITIVE PERSPECTIVE OF SELF-OTHER AGREEMENT: A LOOK AT OUTCOMES AND PREDICTORS OF SHARED IMPLICIT PERFORMANCE THEORIES Hsien-Yao Swee Dissertation Approved: Advisor Dr. Rosalie J. Hall Accepted: Department Chair Dr. Paul E. Levy Committee Member Dr. Steven R. Ash Dean of the College Dr. Chand Midha Committee Member Dr. James M. Diefendorff Dean of the Graduate School Dr. George R. Newkome Committee Member Dr. Paul E. Levy Date Committee Member Dr. Robert G. Lord ii

4 ABSTRACT Research examining self-other agreement (SOA) in performance ratings has been increasing over the years. Previous research on SOA has shown how demographic, individual difference, and cultural variables can influence the extent of agreement between self and other. However, this study extends the existing literature on SOA by examining agreement from a cognitive perspective. Specifically, the current study investigates how similarities in a specific type of cognitive structure, known as implicit performance theory (IPT), may have an effect on agreement. Implicit performance theories represent an individual s mental representation and organization of work related behaviors that are needed to perform effectively at one s job. The study hypothesizes that when individuals and their supervisors mentally organize performance related behaviors in a similar manner, SOA is more likely to occur. In addition, the study also investigates how demographic similarities between self and other, perceived system knowledge, and role clarity may influence the development of shared IPTs. Data were gathered from a group of managers and their supervisors. The results strongly suggest that managers and supervisors who exhibit higher levels of shared IPTs are also more likely to experience greater levels of agreement in performance ratings. In addition, the development of shared IPTs was found to be influenced by similarity in age, perceived system knowledge, and role clarity. Implications, future suggestions and limitations are discussed. iii

5 ACKNOWLEDGEMENTS This dissertation would not be possible without the support and help of many special people. First, I am extremely fortunate and deeply grateful to have Dr. Rosalie Hall as my advisor. In spite of her busy schedule, she was always there for me throughout my graduate career, be it to provide feedback and assistance, or to listen and offer words of encouragement. In addition, I would like to thank all my committee members, Paul Levy, Bob Lord, Jim Diefendorff, and Steve Ash, for their invaluable feedback and assistance. I am additionally thankful to Dr. Levy for welcoming me into his home and family, and for making my graduate experience an unforgettable one. I would like to extend my thanks to the numerous friends who have made my experience at Akron so much more enjoyable. First, I am very fortunate to have come to know Ben Ball and Mindi Thompson. I enjoyed spending my time with them and I am glad they have become such special friends. I am especially grateful to Samantha Chau for being an extraordinary person and friend. She was not only a fun research partner to work with, but was also always wiling to share her encyclopedic knowledge on shopping and fashion. I would also like to thank Boin Chang for giving me a life outside school. Finally, I am indebted to Russ Johnson, Holly Heckman, and Andrew Riggins for helping with my data collection and entry. Last but not least, my graduate experience would not have been possible without the love and support of my wife and parents. I am immensely grateful to both my parents iv

6 for supporting me throughout my educational journey. I am eternally and immeasurably thankful to Chih Ing for all her sacrifices. This journey would not have been possible without her words of encouragement and everlasting love. Thank you for believing in me. v

7 TABLE OF CONTENTS Page LIST OF TABLES... viii LIST OF FIGURES... ix CHAPTER I. STATEMENT OF THE PROBLEM...1 Brief Overview and Importance of Self-Other Agreement...1 Brief Overview of Implicit Performance Theory...5 Summary...9 II. LITERATURE REVIEW...11 Self-Assessment and Self-Other Agreement...12 IPT as Precursors to SOA...24 Role clarity and PSK: Inputs to Construction of IPT...35 III. METHOD AND RESULTS FOR PILOT STUDY...39 Pathfinder...39 Method...40 Results...50 Summary of Pilot Study...64 IV. METHOD AND RESULTS FOR FOCAL STUDY...66 Method...66 vi

8 Results...71 V. DISCUSSION...85 Overview of Findings...86 Theoretical and Practical Implications...93 Limitations and Future Research...95 VI. SUMMARY...98 REFERENCES APPENDICES APPENDIX A. MANAGER PILOT STUDY MEASURES APPENDIX B. SUPERVISOR PILOT STUDY MEASURES APPENDIX C. MANAGER MAIN STUDY MEASURES APPENDIX D. SUPERVISOR MAIN STUDY MEASURES APPENDIX E. IRB APPROVAL (UNIVERSITY OF AKRON) APPENDIX F. IRB APPROVAL (UNIVERSITY OF SOUTH FLORIDA) vii

9 LIST OF TABLES Table Page 1 Summary of Research Hypotheses Pilot study: Descriptive Statistics for Manager and Supervisor Essentialness Ratings of Behavior Items Pilot Study: Sample Descriptive Statistics for Demographics, Pathfinder Indices, and Performance Measures, with Significance Tests for Manager vs. Supervisor Differences Pilot study: Correlations among Demographic Variables, Pathfinder Indices, and Performance Measures for 20- and 14-Item Versions of Measures Main study: Means, Standard Deviations, and Cronbach s Alpha for Study Variables Main study: Correlations among Variables used for Hypothesis 1 (n=150) Main study: Hierarchical Regression of Self-Supervisor Agreement in Performance Rating (Absolute Difference) on Control Variables and Pathfinder Similarity (n=147) Main study: Hierarchical Regression of Self-Supervisor Agreement in Performance Rating (Algebraic Difference) on Control Variables and Pathfinder Similarity (n=147) Main study: Follow-up to Multivariate Test of Hypothesis 1: Univariate Regression of Manager and Supervisor Performance Rating Component Scores on Pathfinder Similarity Index (C) and Control Variables (n=147) Main study: Correlations among Variables used for Hypotheses Main study: Regression of Pathfinder Similarity on Hypothesized Predictor and Control Variables (n=123)...83 viii

10 LIST OF FIGURES Figure Page 1 Conceptual model of Self-Other Agreement and Implicit Performance Theory Complete Hypothesized Model High C Index Pair: Network of Manager High C Index Pair: Network of Manager s Supervisor Low C Index Pair: Network of Manager Low C Index Pair: Network of Manager s Supervisor...59 ix

11 CHAPTER I STATEMENT OF THE PROBLEM In this and following chapters I advance a proposal to examine the issue of selfother agreement in performance ratings from a cognitive perspective. In particular, the focus is on agreement in ratings between a subordinate, and his or her supervisor. I develop an argument that a key proximal factor influencing the level of agreement is the extent to which the subordinate and supervisor define good performance similarly, as indicated by the extent to which they share similar cognitive structures. First, the issue of why self-other agreement is important to consider is addressed. Brief Overview and Importance of Self-Other Agreement Interest by performance appraisal experts in examining the impact of self-other agreement (SOA) in ratings of work performance has been increasing over the past few years (Atwater, Ostroff, Yammarino, & Fleenor, 1998; Ostroff, Atwater, & Feinberg, 2004). This surge in interest is partly due to the fact that more organizations are beginning to implement feedback systems that collect performance information from a variety of sources, including self, peers, subordinates, and supervisors. In fact, over onethird of U.S. companies provide managers with some type of multi-source feedback (Bracken, Timmreck, and Church, 2001). 1

12 SOA is generally believed to serve a number of important functions for both the individual and organization. From an individual s perspective, if self-perceptions in abilities or performance are too discrepant from the perceptions of other sources, then it may be difficult for that individual to manage his working relationships, as well as to monitor his own performance (Fletcher, 1997). Studies by Atwater and colleagues (Atwater & Yammarino, 1992; Atwater, Ostroff, Yammarino, & Fleenor, 1998) demonstrated that agreement between self and others is associated with increased performance. In addition, to the extent that individuals accept the ratings of others, lack of SOA may serve as an indicator that the individual may need to identify areas for developmental. From an organization s perspective, SOA in performance is important if it supports the perceived favorableness of the appraisal process, thereby minimizing the possibility that the individual may challenge the legality of the evaluations. In general, research on SOA suggests that agreement between self and other raters tends to be somewhat low. For example, some meta-analyses have shown that the correlation between self and other ratings range between.29 and.36 (e.g., Harris & Schaubroeck, 1998; Mabe & West, 1982). However, some researchers (e.g., LeBreton, Burgess, Kaiser, Atchley, & James, 2003) have found SOA to be higher, possibly reflecting different analytic strategies for assessing level of agreement. In addition, results from both of the reviews mentioned earlier in this paragraph indicate SOA is likely moderated by a number of individual difference and organizational variables. Thus, regardless of the question of how much SOA is there?, the existence of moderators suggests that researchers need to pay more attention to the boundary conditions of SOA and their implications for understanding the mechanisms that influence agreement. 2

13 It is important to note that the issue of self-other agreement differs from questions about rating accuracy in that the former typically does not compare subjective ratings with objective measures and thus makes no assumptions about whether ratings are accurate. However, even if ratings from the various sources are not accurate, these evaluations are still important for the target individual because ratings from other sources not only allow the individual to obtain a different perspective on how he or she is perceived, but also enable him or her to better navigate the political realities in the organization (London & Smither, 1995). It should also be noted that while researchers generally argue that agreement between self and other ratings is by and large beneficial to the rating target, it may also sometimes signal a problem. For instance, there is some evidence indicating that supervisors may be guilty of inflating their ratings of subordinates, much like self-ratings are frequently inflated (Duarte, Goodson, Klich, 1994). Thus, to the extent that there is agreement due to inflation of both self and supervisor ratings, a number of negative outcomes may occur. For example, the target of the ratings may not be motivated to improve himself, or he may believe that he is entitled to a promotion or raise that he does not deserve. Nevertheless, regardless of whether high levels of SOA lead to positive or negative outcomes, SOA is still an important phenomenon for both scientists and practitioners because it plays an important role for the individual as well as the organization. Empirical evidence suggests a wide variety of factors can influence self-other agreement. Ostroff, Atwater, and Feinberg (2004) demonstrated that both demographic variables, such as ethnicity, gender, and age, and organizational variables, such as 3

14 experience and education, influence the tendency for an individual to over- or underestimate his performance. Similarly, Harris and Schaubroek s (1988) and Heidemeier and Moser s (2009) meta-analysis found higher agreement between self and other for bluecollar jobs compared to professional jobs. They suggest that performance in nonprofessional jobs may be less ambiguous and more objective, thereby leading to higher self-other agreement. Even though there is a substantial body of research on SOA, there are important gaps in our understanding of it. Although many studies focus on personal or contextual antecedents of self-other agreement, these studies have yet to directly examine the psychological mechanisms by which these antecedents influence SOA. More often than not, researchers propose that demographic and organizational variables influence SOA through mechanisms like stereotyping, expectations, and similarity effects, without explicitly testing for these mediating mechanisms. For example, in their review of SOA research, Atwater and Yammarino (1997) proposed that cognitive structures such as schemas, beliefs, and expectations play an important role in the SOA process. Therefore, the purpose of the current study is to investigate SOA from a cognitive process perspective. Specifically, I will attempt to examine how dyadic similarity (between self and supervisor) in a specific type of cognitive schema called an implicit performance theory may act as a mechanism through which SOA in performance ratings occurs. Furthermore, I will also investigate how other organizationally relevant constructs like role clarity and perceived system knowledge (PSK) may act as proximal antecedents of IPT. The next section gives a brief overview of implicit performance theories. 4

15 Brief Overview of Implicit Performance Theory Foti and Lord (1987) argue that implicit theories are cognitive structures that form the foundation upon which individuals interpret the actions of others. For example, implicit leadership theories represent personal cognitive structures and schemas that specify traits and behaviors that subordinates expect from their leader. It has been suggested that these schemas are formed through socialization and past experiences with leaders. These structures are important in helping subordinates form a cognitive basis for understanding and interacting with managers and leaders (Epitropaki & Martin, 2004). In a similar vein, implicit performance theories (IPTs) are cognitive representations of behaviors that indicate effective performance. For instance, Lord and Maher (1991) suggest that leaders rely on implicit performance theories to form impressions of their subordinates, by developing schemas and prototypes of effective subordinates and then comparing their perceptions of the subordinates actual behaviors to these prototypes. These schemas thus may be characterized as a collection of specific behaviors that, when exhibited, represent effective performance. Relevance of IPT to SOA To the extent that subordinates are familiar with and demonstrate the behaviors that are consistent with a leader s IPT, they are more likely to receive favorable evaluations by their leader. This implies that a subordinate s awareness of the leader s IPT may result in SOA because congruence in implicit theories provides the basis for a mutual understanding of what constitutes good performance. Thus, the subordinate s behaviors and both parties interpretations of these behaviors are likely to be in alignment because the schemas used to guide behaviors and interpretations are shared between 5

16 actors. That is, to the extent that both self and supervisor share IPTs, and thus agree on what constitutes effective performance, SOA is likely to occur. This is because the target employee is not only more likely to engage in behaviors that are important for effective performance, but also because both self and other are inclined to interpret such behaviors as effective. It is assumed here that similarities in IPTs result in SOA because individuals are better able to evaluate and direct their own behaviors. However, it is also plausible that SOA was achieved because individuals knew how others were going to evaluate them and were rating themselves as they thought others would, rather than in accord with their own personal evaluations. The latter possibility is interesting in that even if agreement in overt ratings is achieved, the individual may privately not agree with the ratings and may experience feelings of unfairness. However, the main purpose of this study will be to focus on how individuals truly view and evaluate their own behavior. Empirical evidence. Indeed, there is some empirical evidence demonstrating that raters of the same target may hold similar implicit performance theories, and when they do, SOA is higher. Yammarino and Waldman (1993) found that incumbents ratings of job skill importance correlated with supervisors ratings on the corresponding skill dimensions. This suggests that both incumbents and supervisors agreed on the skills that were necessary for the job. Similarly, although Tsui and Ohlott (1988) hypothesized that ratings from different sources at different organizational levels (superior, peers, and subordinates) would have different criteria for managerial effectiveness, their hypothesis was not supported. This suggests that these raters from different organizational levels may have had similar criteria for managerial effectiveness. Schrader and Steiner (1996) 6

17 found SOA increased when both self and supervisor raters were provided with common comparison standards and explicitly instructed to use these standards to evaluate the target. Their results strongly suggest that when raters have a common understanding of what constitutes effective performance, SOA increases. Issues in the measurement of IPT. The current study extends previous research on implicit performance theories by adopting Pathfinder as a tool to index similarities in implicit performance theories between self and supervisor. Previous research has tended to use correlations as indicators of similarities in implicit performance (e.g., Schrader and Steiner, 1996). Such techniques typically involve comparing rank ordering of scale scores between different raters. But correlations only indicate the extent to which summary scores from rating instruments are similar and do not measure actual similarities or dissimilarities in the centrality and inter-relationships of the concepts which make up performance schemas. However, Pathfinder can help address this deficiency. It is a relatively new technique designed to produce appropriate psychological scaling with respect to the underlying structure of relationships between concepts (Schvaneveldt, 1990). Respondents are typically presented with a pair of concept terms and are asked to rate the extent to which the terms are similar or share certain characteristics. The Pathfinder algorithm then transforms raw paired comparison ratings into a network of nodes. These nodes represent the psychological concepts. The relationships between concepts are represented as linkages among nodes. As a result, Pathfinder allows for a deeper and richer representation of how an individual conceptualizes effective performance. Pathfinder also provides an index for evaluating the similarity of two different networks. 7

18 Therefore, Pathfinder can allow us to directly assess the similarity in implicit performance theories, thereby permitting us to more accurately examine its effects as a mediating mechanism between antecedent variables and self-other agreement in performance ratings. Predictors of Implicit Performance Theories Another aim of this study is to investigate how similarities in IPTs may develop. Specifically, I examine whether shared IPTs may be increased by the extent to which the target has an increased sense of role clarity and perceived system knowledge, thus creating greater SOA (Figure 1). Subordinate Role Clarity Subordinate Perceived System Knowledge Similarity in Implicit Performance Theory Self-Supervisor Agreement in Performance Ratings Figure 1. Conceptual model of Self-Other Agreement and Implicit Performance Theory Role clarity refers to the extent to which an employee understands his or her job, role expectations, and responsibilities (Bauer, Morrison, & Callister, 1998). The effects of role clarity have been examined extensively in the newcomer socialization domain. In general, studies have shown that newcomer s proactive behaviors tends to be positively associated with role clarity and other organizational outcomes such as performance and job satisfaction (Bauer & Green, 1998; Wanberg & Kammeyer-Mueller, 2000). A key construct influencing role clarity is information seeking. As newcomers seek and gather more information about their jobs, they are able to better understand what is expected of 8

19 them (Wanberg & Kammeyer-Mueller, 2000). Therefore, role clarity should be positively associated with IPT. It has been suggested that leniency in self-assessments is partially a result of employees lacking the requisite information to make an informed assessment (Farh & Dobbins, 1989). In a similar vein, Williams and Levy (1992) argued that individuals who know what is expected of them, understand the appraisal system, and agree with the performance standards are less likely to inflate their self ratings. The authors found that perceived system knowledge (PSK), defined as one s knowledge of the appraisal system, moderated the relationship between self-assessments and supervisor ratings, such that the relationship between self and supervisor ratings was higher for individuals with higher levels of PSK. Because PSK deals directly with knowledge of performance standards and expectations, it is plausible that PSK will be positively related to similarity in IPTs. Summary To summarize, the main focus of this study is to examine SOA in performance ratings from a cognitive perspective, in order to test whether shared cognitive structures can help explain the extent of agreement between a supervisor and subordinate. In contrast, the majority of previous SOA research has focused on demographic and organizational variables as antecedents of SOA, without directly testing how cognitive mechanisms may exert a more proximal role in predicting agreement. It is proposed here that similarity between self and supervisor in IPTs operates as a proximal predictor of SOA because it reflects the extent to which individuals have similar definitions of effective performance. In addition, this study also examines how variables like role clarity and PSK may have an influence on similarities in implicit performance theories 9

20 between self and other. Figure 1 summarizes these proposed relationships. The model in Figure 1 implies that both role clarity and PSK are expected to be positively related to similarities between subordinate and supervisor in IPT. Furthermore, similarities in IPT are hypothesized to be related to SOA in performance ratings. In addition, because previous studies on SOA in performance ratings have already shown variables such as gender, age, and tenure with supervisor relate to SOA, measures of these variables will also be collected and incorporated in the model. Participants in the study will complete a paper and pencil survey. Students will be asked to help recruit a manager who is willing to complete the questionnaire. The manager will also hand his or her supervisor a questionnaire to complete. Both manager and supervisor will return their completed surveys individually to the researcher using a self-addressed envelope. If the model of Figure 1 is supported, the results will contribute to the SOA literature in a number of ways. First, by identifying similarities in IPT as the most proximal predictor of agreement, we are able to gain a cognitive explanation for how variables such as role clarity and PSK influence SOA. Second, the use of Pathfinder as an analytic tool has yet to be adopted in SOA research. Therefore the study extends previous research by using Pathfinder to more accurately determine and measure similarities in cognitive structures between subordinate and supervisor. Finally, to the extent that agreement is shown to be influenced by role clarity and PSK, organizations may want to incorporate more formal and structured socialization and mentoring programs, or provide better job descriptions so as to increase SOA. 10

21 CHAPTER II LITERATURE REVIEW The primary aim of the current study is to develop and test a shared cognitive structure explanation of self-other agreement (SOA) in performance ratings between managers and their supervisors. Specifically, the study will examine how similarities in implicit performance theories between managers and supervisors may lead to increased agreement in performance ratings. In addition, the current study also attempts to investigate how role clarity and perceived system knowledge may have a direct impact on similarities in implicit performance theories between self and supervisor. In the following sections, I will present a brief review of the self-other agreement research thus far, as well as discuss some of the theoretical and practical importance behind the current study. Following this discussion, the proposed model of self-other agreement is elaborated (see Figure 2). The model presents role clarity and perceived system knowledge as antecedents to implicit performance theories. Finally, it is proposed that similarities in implicit performance theories between self and supervisor should result in higher levels of self-other agreement in performance ratings. The relationships between the constructs are discussed in detail and relevant theoretical and empirical research is presented to support the proposed pathways. 11

22 Self-Assessment and Self-Other Agreement The use of self assessment of work performance is becoming a common practice in many organizations. Self assessments typically involve employees evaluating themselves on a variety of performance dimensions, skills, or attributes. These assessments are then used as part of the formal performance appraisal process, which feeds into pay and promotion decisions, or is used to identify developmental needs (Atwater, 1998). Often, self-ratings are considered in conjunction with additional rating information to develop a more complete assessment of performance. Indeed, Bracken, Timmreck and Church (2001) noted that over one-third of U.S. companies provide managers with some type of multi-source feedback. For example, the 360-degree feedback process which is popular currently in many organizations extends the collection of self-appraisals to include ratings from other constituents with which the target ratee has contact. These may be evaluations from superiors, subordinates, peers, and clients or customers. Potential benefits of self-assessment Using self-ratings as part of the performance appraisal process is popular in many organizations for a number of reasons. First, the very process of engaging in selfappraisal may increase communication between rater and ratees regarding a variety of important issues such as job content, performance criteria, and mutual expectations. The result may be reduced ambiguity in the appraisal process, potentially decreasing a source of rater-ratee disagreement. Second, self appraisals give ratees a greater sense of control over their own performance evaluations. This, in turn, may lead to increased employee satisfaction and 12

23 acceptance of appraisal results. Increasing the employee s favorable perceptions of the appraisal process is especially important because such perceptions have been shown to influence one s fairness and satisfaction reactions to the appraisal (Farh et al. 1988; Greenberg, 1986). Third, self-ratings often contain less halo error than supervisory ratings. Interestingly, supervisors may be more susceptible than employees rating themselves to overall, heuristic rating biases that lead them to assign uniformly high (or low) scores across all performance dimensions for a given ratee. Thus, compared to supervisory ratings, self-ratings may be more discriminating across performance dimensions, as evidenced by greater within-person variability across dimensions. This is important because discriminating specific dimensions of poorer performance may be helpful in identifying how to improve performance. Fourth, ratings from multiple sources allow extremely biased ratings from any one source to be more easily identified, because the biased ratings will stand out as discrepant in a set of ratings that otherwise show at least moderate agreement (Farh et al., 1988). Agreement of Self-Ratings of Performance with Ratings from Different Sources In spite of the previously described benefits, self-ratings and appraisals also can be problematic in that there is substantial evidence that they are frequently biased, inflated, less valid (when compared to organizational criteria), and more inaccurate when compared to ratings of others such as supervisors, subordinates, and peers (Harris & Schaubroek, 1988). Often a main objective of using multiple rating sources (e.g., in a 360-degree feedback process) is to help identify areas where discrepant self-perceptions exist, especially when the self-perception is more positive than the perceptions of others. 13

24 Actions may then be taken to rectify and address these discrepancies through the use of training and development programs. Extent and patterns of self-other agreement. Traditionally, studies on SOA have operationalized agreement in a variety of ways. A common way in which agreement is empirically determined is through the use of difference scores (Furnham & Stringfield, 1994). In such studies, investigators typically operationalize agreement in terms of mean differences in scores between raters. Another way in which studies have examined agreement is with correlations (Furnham & Stringfield, 1994, 1998; London & Wohlers, 1991). For example, investigators use the correlation between self and other ratings as an index of agreement. A third way that investigators have measured SOA is by categorizing individuals into groups (Atwater & Yammarino, 1997; Fleenor, McCauley, & Brutus; 1996). In these studies, self-raters are categorized into different types of agreement such as over or under-raters, or in-agreement (good or poor). Such approaches to determine agreement have been criticized as methodologically flawed. Edwards (1995) argued that by collapsing or combining self and other ratings into a single index, investigators are unable to determine and compare how each separate rating source may be influenced by predictors or may differentially impact outcomes. For example, if one determines that whites are more likely to be overraters than non-whites, this finding does not indicate whether the overrating is due to higher self ratings, lower other ratings, or a combination of both. Finding out the exact nature of the disagreement is important because it not only allows for a clearer understanding of the phenomenon, it also enables practitioners to better design interventions. For example, if the reason whites tend to be categorized as overraters is a 14

25 result of higher self-ratings, then any interventions should be targeted at the self-rating process. However, if over-rating occurs because non-whites are given lower ratings by others, then a different type of intervention is needed. To date, four meta-analyses of SOA have been conducted (Conway & Huffcutt, 1997; Harris & Schaubroek, 1988; Heidemeier & Moser, 2009; Mabe & West, 1982). Depending on the source of the other ratings, the correlations between different sources in these studies range between.22 and.80. It is interesting to note that the meta-analyses by Conway and Huffcutt (1997) and Harris and Schaubroek (1988) both showed the correlations between self and others to be lower than the correlations between different types of others. For example, Conway and Huffcutt (1997) demonstrated the mean selfsupervisor and self-peer correlations to be.29 and.28, respectively. However, the mean supervisor-peer correlation was.80. These results indicate that it is unlikely that low correlations between self and others found in previous studies are simply a generic result of different sources having different perspectives on performance, because if that were the case, self-supervisor and peer-supervisor ratings would be expected to have correlations of similar magnitudes. The typical finding from SOA research is that self-ratings are often inflated when compared to other ratings (e.g., Furnham & Stringfield, 1998; Yammarino & Waldman, 1993). A natural question, then, is whether this disagreement poses a problem. Some researchers argue that the psychological processes which can lead to inflated self-ratings may serve positive functions for the psychological well-being of the individual. For example, self-enhancement bias may help to minimize disabling negative thoughts or create higher expectancies of success (Taylor & Brown, 1988). However, ignorance of 15

26 poor performance as signaled by discrepancies in ratings also may mean the individual fails to take the necessary remedial actions to improve, because he or she does not perceive the need for increasing his or her level of performance. However, some studies find an opposite result, namely, that self-ratings are systematically lower than other ratings (see Sosik & Megerian, 1999; Yammarino & Atwater, 1997). Although less obviously so, this situation also may be problematic if an employee consistently under-estimates his or her ability to perform, and thus sets goals below his or her performance capability. Thus, over- or under-rating of the self can have implications for the employee s ability to take appropriate developmental actions, whether those involve correcting poor performance, or pushing oneself to attempt yet higher performance. Furthermore, as will be expanded in later sections, comparing selfratings with other ratings is also beneficial to the individual for developmental reasons. Research has demonstrated that agreement between self and others in performance ratings is associated with more positive organizational outcomes (Atwater, Waldman, Ostroff, Robie, & Johnson, 2005; Ostroff, Atwater, & Feinberg, 2004). For example, Ostroff et al. (2004) found agreement between self and others on leadership dimensions positively related to compensation and supervisor ratings of performance. Finally, it is important to mention that some researchers argue that statistical artifacts may exaggerate the extent to which it appears that there is disagreement between self- and other ratings. For example, LeBreton, Burgess, Kaiser, Atchley, and James (2003) suggest the lack of agreement in many SOA studies may be due to restriction of variance in performance measures. Lack of between-target variance may attenuate correlation-based estimates of the strength of between source ratings. Specifically, 16

27 LeBreton et al. suggested using r wg as an index of rating similarity would be more appropriate because it was less sensitive to lack of variance. Whether or not LeBreton et al. s approach is specifically adopted, literature of this nature suggests the importance of using appropriate statistical techniques when studying SOA, so as not to artificially inflate estimates of disagreement. Distinction between agreement and accuracy. In addition, it is critical that we do not assume that ratings from other sources represent some true score, against which a potentially erroneous self-rating is compared and found to be accurate or inaccurate. True scores are the actual performance levels of ratees, technically, they can only be inferred, and not directly observed. Accuracy refers to comparisons with a true score or an objective standard which is a proxy for a true score (and which, ironically, in practice may or may not be completely accurate). The comparison standards used to represent true scores may be test scores, performance outcomes, or evaluations provided by others. While these criterion measures are not assumed to be error free, the closer one s rating is to this true score, the more accurate one s rating is assumed to be (Noonan & Sulsky, 2001; Woehr, 1994). However, in contrast to the accuracy literature which assumes one specific score always represents an accurate rating, the SOA literature acknowledges that either, or both, rating sources (self and other) potentially may be inaccurate. Research on rater accuracy is often found in studies that examine the impact of training methods on reducing rating biases. Training methods such as frame of reference training, behavior observation training, and rater variability training are designed specifically to reduce and avoid rating errors such as halo errors and leniency bias (Hauenstein, 1998; Noonan & Sulsky, 2001). For example, frame of reference (FOR) 17

28 training instructs raters on how to correctly link specific behaviors with the appropriate performance dimensions, as well as how to evaluate the effectiveness of each behavior. Specifically, when raters are presented with a performance example, they are also given a quantitative estimate of how high or low this performance behavior should be rated in order to help the raters develop less idiosyncratic rating standards. Indeed, there are some parallels between the FOR literature, and the arguments that are made later in this paper for the importance of cognitive structure in determining SOA. However, most importantly for our purposes is the key point that agreement and accuracy are conceptually distinct constructs. For example, there can be agreement between two raters, yet both may be equally inaccurate. This could occur if two raters have similar misconceptions about rating standards, or if they share common rating biases, such as a tendency to over-rate. Similarly, raters might disagree, and again, neither of the ratings be considered to be accurate (for example, if one rater overestimates and the other under-estimates). The next section of this chapter gives a more indepth presentation of factors which may affect or be affected by SOA. Relationships of SOA with Antecedents and Outcomes Research on self-other agreement has been increasing over the past few years. These studies focused on factors that influence agreement, as well as on outcomes of agreement. Previous studies showing that a variety of individual and organizational level variables relate to agreement will be described in more detail shortly. However, before doing this, I briefly preview the arguments I will make about the need to extend our understanding of SOA in three key ways. 18

29 First, the study will investigate self-other agreement from a cognitive perspective. Previous studies of SOA do not directly investigate how internal cognitive structures consisting of performance-related knowledge can vary and may also directly influence self-other agreement in performance ratings. Specifically, I will attempt to determine how similarities and differences in the implicit performance theories held by the self and other may influence agreement. Examining cognitive processes that may influence SOA will allow us to gain a better understanding of the nature of the agreement process. Second, this study also examines how the content and structure of one s implicit performance theory may vary as a result of organizational factors such as role clarity and perceived system knowledge, which when present may indicate conditions that are likely to increase consistency in the IPTs of different organizational actors. Identifying predictors of implicit performance theory is important for a couple of reasons. Theoretically, doing so allows us to gain a better understanding of the construct in relation to other organizational variables. Practically, if these factors influence similarity in IPTs and SOA, organizations may be able to develop targeted interventions that have a significant influence on the development of one s implicit performance theory. Third, previous studies have frequently operationalized SOA in a manner that makes a clear understanding of factors that lead to disagreement difficult or impossible to demonstrate empirically. For example, it is important to be able to distinguish whether a given potential antecedent of SOA influences the self-rater, or one of the other raters (or, both). Yet single index operationalizations of SOA such as discrepancy scores often do not allow this. This issue, and relatedly, how analytic techniques suggested by Edwards 19

30 (1994, 1995) may be most appropriate in the further study of SOA, will also be presented in the upcoming sections of this chapter. Predictors of SOA In their review of SOA research, Atwater and Yammarino (1997) discussed a multitude of factors that they proposed could affect self-perceptions, self-ratings, and other ratings. They also addressed potential outcomes of SOA. For example, they suggest biographical characteristics such as the gender, age, tenure, and organizational position of the rater are likely to influence self-ratings. Cognitive processes (e.g., schemas), jobrelevant experiences, and contextual factors were three other broad sets of variables suggested to have an impact on self and other ratings. Research evidence on predictors of SOA which fall into these categories will be presented next. Contextual predictors of SOA. Atwater and Yammarino (1997) suggested that situational variables are likely to influence agreement not because they affect self perceptions per se, but because they may have a direct impact on the ratings given by the self and/or others. For example, whether the ratings are used for developmental purposes or administrative purposes may influence the value of the ratings, e., g., self-ratings are more likely to be inflated when they are used for evaluative purposes than developmental purposes (Cardy, Dobbins, & Truxillo, 1988; Farh & Werbel, 1986). However, because the purpose of the current study will be non-administrative in nature, it is unlikely that self-raters will be motivated to inflate their own ratings because there will not be any personal gains from doing so. Demographic predictors of SOA. Empirical evidence from self-other studies supports many of the propositions made in Atwater and Yammarino s (1997) model. First, 20

31 a number of biographic variables such as race, gender, and age have been shown to influence SOA. For example, females are more likely to rate themselves lower than males rate themselves (Atwater et al., 2005; Wohlers & London, 1989). One reason for this is because females may experience lower self-confidence in achievement settings (Vollmer, 1986). However, because female ratees are also more prone to receive lower ratings when compared to their male counterparts, they may demonstrate SOA since both rater and ratee are consistently underestimating performance. Similarly, minorities such as non-whites tend to receive lower ratings from white raters (Roberson & Block, 2001) due to dissimilarity effects and stereotyping. As such, non-whites have been categorized as overraters, even if they accurately rate themselves (Atwater et al., 2004; Wohlers, Hall, & London, 1993). Finally, older workers tend to overrate themselves because they view themselves more positively, may be less receptive to feedback, and/or are perceived more negatively by others due to age based stereotypes (Atwater et al., 2004; Brutus et al., 1999; Rosen & Jerdee, 1988; Ryan, Brutus, Gregurus, & Hakel, 2000). Because previous research has shown these demographic variables to be important predictors, they will be incorporated in the current model. Job type predictors of SOA. Second, with respect to organizational variables, individuals with more managerial experience are more likely to overrate themselves, for similar reasons to those given for older workers. In addition, some research has also shown that SOA can be influenced by the type of job that one holds. Workers in line positions or blue collar jobs tend to demonstrate higher levels of self-other agreement, perhaps because performance expectations for such jobs may be less ambiguous than for managerial positions (Harris & Schaubroek, 1988; London & Wohlers, 1991). Thus, the 21

32 current study will only sample employees in supervisory and managerial positions in order to hold constant potential differences in levels of SOA across job types. This will maximize the extent of variability observed in SOA attributable to the focal study variables by eliminating an extraneous source of variance. Personality and individual difference predictors of SOA. Third, individual difference variables can also influence SOA because they can influence the way people view themselves and the way in which they view and gather information from the environment. For example, research from the personality and social psychology literature suggests that individuals who are less competent are more prone to overestimate their own abilities. Kruger and Dunning (1999) argued that low-performing individuals are not only unable to perform adequately on tasks, their incompetence also prevents them from accurately recognizing effective performance in themselves and others. A series of studies conducted by Kruger and Dunning demonstrated that individuals who scored in the bottom quartile on tests of humor, grammar, and logic consistently estimated their own performance to be above the 60 th percentile. Other potential individual difference predictors of SOA include self-esteem and self-awareness. Individuals with low self-esteem tend to experience low SOA because they consistently underestimate their performance, while individuals with high selfesteem tend to overestimate their performance (Farh & Dobbins, 1989). Furthermore, individuals who are higher on self-awareness are proposed to exhibit greater SOA because they are better able to incorporate information from the environment, such as feedback or social comparisons (Sosik & Megerian, 1999). Although not a part of the 22

33 focal model, a short measure of self-esteem will be included in the current study, so that it can be included as a control variable if necessary. Cultural predictors of SOA. Finally, a number of studies have investigated the role of cultural variables on SOA. These cultural variables are argued to have an impact on agreement because they influence the extent to which one over or under-rates one s own performance. Results from such studies suggest that individuals who are more collectivistic or other-oriented are more likely agree with the perceptions of others because they are less self-enhancing (e.g., Chau & Levy, 2005; Korsgaard, Meglino, & Lester, 2004). Because the sample for the current study primarily consists of North Americans, individuals who are collectivistic and other-oriented will be minimal and also assumed to be randomly distributed in the sample, minimizing any systematic biasing effect of SOA. However, for exploratory purposes, a short measure of collectivism will be included. Methodological considerations. Research on SOA has frequently operationalized agreement between self and other raters either by using signed difference scores or by sorting raters into categories of agreement based on the direction of discrepancy of the target rater from other rating sources, e.g., resulting in categories such as in-agreement, over-rater, and under-rater. These approaches provide some useful information about whether ratings from different sources agree or not, and if not, about the extent and/or direction of disagreement. However, these techniques for operationalizing SOA may be methodologically flawed (see Edwards, 1995). When discrete information from both raters is collapsed into a single index or into a category of agreement, it is harder to probe the underlying nature 23

34 of the disagreement and its potential causes (Edwards, 1995). For example, as mentioned earlier, results that indicate whites are more likely to be over-raters compared to nonwhites do not tell us whether over-rating is a result of higher self ratings, lower ratings from others, or a combination of both. From an organizational perspective, the ability to identify the nature of the selfother disagreement is important because a more focused and targeted intervention can be developed. From a theoretical perspective, finding out how predictors influence the ratings from different sources is important because it gives us a better understanding of the unique influence that these variables exert on each source. Indeed, some researchers (Fleenor et al., 1996) argue that what is characterized as an SOA effect may actually be due simply to the level of other ratings, and that the extent of agreement of these ratings with self-ratings plays essentially no role. This competing explanation can only be tested with approaches that allow the separation of the alternative rating effects. Thus, the current study will not only use examine SOA using an index consisting of an absolute difference score, as has been done in some previous SOA research, but will also adopt multivariate regression procedures as suggested by Edwards (1995). An example of a previous SOA study which uses Edwards approach is Ostroff et al. (2004). IPT as Precursors to SOA The next section reviews the theoretical rationale and as well as empirical evidence supporting why cognitive structures, such as implicit performance theories, may be a critical construct in the investigation of self-other agreement. 24

35 Theoretical Works Suggesting Importance of Cognitive Structures to SOA Campbell and Lee s (1988) conceptual model provides a helpful organizing framework for how and when various cognitive processes may influence self-appraisals. They view self-appraisal as a four-step process. In the first step, the individual has certain beliefs about the requirements of the job and the actions needed in order to achieve those goals. Next, these cognitions regulate the individual s job-related behaviors as he or she attempts to meet perceived job requirements. Then, while appraising his or her own performance, the individual recollects the job behaviors exhibited as well as the outcomes associated with those actions. Finally, one s self-appraisal is compared with the evaluation given by the supervisor, who is argued to experience the same four-step process. Thus cognitive processes are important at multiple steps, as they influence the choice of work behaviors to be performed, the manner of that performance, the accuracy of recalled behaviors and outcomes, the standards against which behaviors and outcomes are evaluated, and beliefs about the nature of a specific other s evaluation. Directly relevant to the proposed study, Campbell and Lee (1988) suggest that differences in cognitive processes between self and supervisor may create discrepancies in performance ratings. For example, they suggest lack of agreement may be due to rating sources not sharing similar perceptions of which behaviors are important for effective job performance. Thus, Campbell and Lee s model, by emphasizing the potential role of informational and cognitive constraints in SOA, may more precisely isolate the psychological mechanisms involved than approaches described earlier which provide explanations only in terms of very broad personality and situational factors. These more proximal explanatory mechanisms may make it easier to identify specific organizational 25

36 actions that could be taken to improve SOA. For example, Campbell and Lee suggest that supervisors make concerted efforts to increase role clarification and improve job descriptions an action that addresses the first step of their model by potentially making more similar the employee s and supervisor s beliefs about job requirements. However, there is also room for refinement of the underlying logic of Campbell and Lee s (1988) model. The current study attempts to further define the idea of a cognitive structure consisting of representations of job expectations and the related behaviors, and to align the measurement of such a structure with practices prevalent in the cognitive literature. This builds on research from social psychology which has demonstrated that knowledge structures that we hold about events and people -- known as schemas or stereotypes -- vary from person to person (Feldman, 1981). Schemas and stereotypes not only organize information, but they influence recall and guide evaluations. For example, when a stimulus person matches a given prototype, schema-relevant information for the person is automatically recalled. Sometimes, schema- consistent but incorrect information is also recalled. Thus, depending on the extent to which the stimulus person matches a prototype, evaluations for the individual may be accurate, over-estimated, or under-estimated (Feldmand, 1981; Ilgen & Feldman, 1983). Definition and elaboration of the concept of implicit performance theory. The collection of cognitions, beliefs, expectations, and images of past performance referred to in Campbell and Lee s model of self-appraisal sounds very much like a cognitive schema. A schema is defined generally as a cognitive structure or framework that affects the way in which we notice, encode, retrieve, and interpret information (Fiske & Taylor, 1991). In this chapter, I develop an argument suggesting that a particular type of schema, called an 26

37 implicit performance theory, may be an important precursor of SOA. Building on the ideas of Campbell and Lee, I argue that similarity in supervisor and subordinate implicit performance theories is associated with SOA, such that greater similarity in the implicit performance theories held by two raters leads to greater agreement in ratings. Implicit theories are cognitive structures that form the foundation upon which individuals interpret the actions of others (Foti & Lord, 1987). These schemas not only represent how our knowledge in a certain domain is represented, it also influences how we interpret information and make decisions. For example, implicit leadership theories are representations of traits and behaviors expected from leaders. It has been suggested that these schemas are influenced by one s social interactions and are therefore formed through socialization and past encounters with leaders. These cognitive structures are important from a sense making perspective because they help subordinates form a cognitive basis for understanding and interacting with managers (Epitropaki & Martin, 2004). The principles behind implicit leadership theories may also be applied to job performance. Implicit performance theories (IPTs) refer to an individual s beliefs regarding the covariance of performance outcomes with relevant traits and behaviors (Wegner & Vallacher, 1977). In other words, they are cognitive representations of behaviors that indicate effective performance. They are useful for providing a cognitively efficient means of inferring behaviors that lead to a specific level of performance (Parker, 1999). These cognitive schemas not only guide a person s interpretation of events, they also guide and direct behaviors. Therefore, if an individual is aware of the necessary behaviors needed to perform effectively, he or she is likely to perform such behaviors to 27

38 the degree that he or she has sufficient motivation. In addition, because schemas also influence people s understanding of events, the individual is likely to organized these essential behaviors in a manner that will result in effective performance. As a result, an individual s implicit performance theory is likely to have a direct influence on his or her performance on the job. This idea is consistent with the first and second steps in Campbell and Lee s (1988) model. Similarly to subordinates, supervisors may also use and rely on implicit performance theories to evaluate and form impressions of effective subordinates. Supervisors form schemas and prototypes of effective subordinates and then compare subordinates to these cognitive representations. Thus, the IPTs of supervisors are also characterized by a number of specific behaviors that when exhibited represent effective performance for the subordinate (Lord & Maher, 1991). When an individual is asked to evaluate his own performance or the performance of another, those evaluations may be a mix of on-line judgments and memory-based judgments. On-line evaluations occur when the individual forms a judgment as relevant information is encountered (Hastie & Park, 1986). On-line judgments may also be more likely to occur when individuals believe that an evaluation is required at a later time (DeNisi, Robbins, & Summers, 1997). In contrast, memory-based judgments occur when the individual makes an evaluation by retrieving all the necessary evidence from longterm memory before rendering a judgment. Hastie and Park argued that the conditions under which memory-based judgments occur are rare and that on-line judgments are relatively more common. If a supervisor and subordinate differ in the extent to which an evaluation reflects on-line versus memory-based judgments, their ratings may well differ. 28

39 For example, on-line judgments may ignore or discount information about the ultimate effectiveness of a set of behaviors, while memory-based judgments may sometimes overweight such outcomes. The notion that self and others may share similar IPTs has been suggested in previous studies. For example, Tsui and Ohlott (1988) examined ratings from different sources at different organizational levels (superior, peers, and subordinates). Adopting Borman s (1974) contention that raters from different organizational levels should exhibit less agreement because they may have different conceptualizations of performance or different opportunities to observe that performance, the authors hypothesized that different sources would have different criteria for managerial effectiveness. However, their hypothesis was not supported. This suggests that these raters from different organizational levels may hold similar criteria for managerial effectiveness. Therefore, to the extent that raters from different organizational levels are able to share similar perceptions of what constitutes managerial effectiveness, is it likely that self and others are able to share IPTs. As mentioned previously, these cognitive structures are similar to those referenced in the theory behind FOR training in that both emphasize the ability to accurately identify effective performance behaviors. However, frame of reference focuses on not only creating a common standard with regards to raters perception of relevant performance dimensions and behaviors, but also emphasizes the effectiveness level associated with each behavior. On the other hand, IPTs are characterized in terms of how these behaviors interrelate with each other for effective performance. 29

40 It is important to note here that the current definition of IPT used in this study differs from Uggerslev and Sulsky s (2008) conceptualization of performance theory. The authors describe performance theory as an individual s knowledge of the performance dimensions and behaviors that are essential for effective performance. In their study, performance theory was operationalized by asking individuals to indicate which performance dimensions and behaviors they would include while evaluating a professor s teaching performance. More importantly, absent from Uggerslev and Sulsky s definition of performance theory is the mental organization of such work related dimensions and behaviors. The importance of performance schemas or knowledge structures has been emphasized in team-based research. In general, these studies argue that team members who share similar schemas are more likely to attend to and interpret events more similarly than members who do not share such schemas. For example, Rentsch and Hall (1994) described two types of team-related schemas that may be critical for effective team performance. One of these, the taskwork schema, is highly similar to an IPT. Salas and colleagues define taskwork as the understanding of the rules and requirements by which effective task performance is accomplished. These rules and requirements may include establishing patterns of interaction, exchanging task-related information, organizing, planning, and developing team solutions to problems (Glickman, Zimmer, Montero, Guerette, Campbell, Morgan, & Salas, 1987; Salas, Montero, Glickman, & Morgan, 1988). Therefore, the term taskwork schemas used in this literature is used similarly to our use of IPT to refer to knowledge structures that guide task performance. 30

41 However, IPTs and taskwork schemas are qualitatively different in one important aspect. Implicit performance theory describes the knowledge structure of behaviors for performing effectively as an individual, while taskwork schemas depicts the knowledge structure of behaviors that contribute to effective team performance. Nevertheless, the idea of shared cognitions, such as team mental models, is conceptually similar to the concept of similarity between self and supervisor in IPT because they both tap congruence in cognitive structures, albeit at different levels of analysis. The emphasis in the literature on team mental models is at the group level, while our proposed study of congruence in IPTs is at the dyadic level. Extending the findings from team-based research, it is plausible that subordinate awareness of the leader s IPT may result in higher levels of SOA in performance ratings because congruence in implicit theories provides the basis for mutual understanding. Thus, work behaviors, as well as the interpretations of these behaviors, are likely to be in alignment because the schemas used to guide behaviors and interpretations are shared between actors. That is, to the extent that both self and supervisor share IPTs, SOA is likely to occur. High SOA becomes more probable because the target employee is assumed to not only be more likely to engage in behaviors that are important for effective performance, but also because both self and other are inclined to organize and interpret such behaviors in a similar manner. Empirical evidence supporting the potential importance of IPTs. Although there have not been any studies that directly examine the effects of similarities in IPTs on SOA, there is some more general empirical evidence suggesting that raters of the same target may share similar perceptions of effective performance, and when they do, SOA is higher. 31

42 Yammarino and Waldman (1993) investigated whether incumbents and their supervisors had similar perceptions regarding job skills that were thought to be important for the incumbent s job. The authors found that incumbents ratings of job skill importance correlated with supervisors ratings on the corresponding skill dimensions. This means that if incumbents thought a particular skill, such as communication, was important for the job, supervisors also perceived this skill as important. This suggests that both incumbents and supervisors agreed on the importance of that skill for the job. Interestingly, Yammarino and Waldman (1993) found that correlations between incumbent and supervisor on importance ratings were higher than correlations between incumbents and supervisors ratings of performance on those skill dimensions. In an experimental study that more directly examined the effects of similarities in performance standards between self and supervisor, Schrader and Steiner (1996) provided both self and supervisor with various performance comparison benchmarks. Their results indicated that SOA was higher when both self and supervisor raters were provided with common comparison standards and explicitly instructed to use these standards to evaluate the target. Their results strongly suggest that when raters have a common understanding of what constitutes effective performance, SOA increases. Current study: Hypotheses relating similarity in IPTs to SOA. Therefore, if both self and supervisors share similar IPTs, it is likely that there will be greater agreement about what behaviors are necessary for effective performance. This means that when subordinates perform such behaviors, both self and supervisor are also likely to interpret and rate performance similarly, resulting in higher SOA. Thus, I propose 32

43 H1: Similarity between self and supervisor in IPTs will be positively related to SOA. Because previous studies have not only demonstrated the importance of performance expectations to SOA (Schrader & Steiner, 1996; Yammarino & Waldman, 1993) but have also measured performance theories using a different approach than the one that will be used in the current study (Uggerslev and Sulsky, 2008), this study will adopt and use two different methods of capturing one s IPT. The first approach will adopt a technique similar to the one used in Uggerslev and Sulsky s study. Specifically, participants will be asked to rate how each of the work-related behaviors used in this study are essential for effective performance. The second approach will adopt a knowledge capturing tool, known as PathFinder, as a means of measuring the mental organization aspect of IPTs (a detailed description of how Pathfinder works will be presented in Chapter 3). Using these two methods for measuring IPT will allow us to more accurately identify which aspect of one s IPT (essentialness vs. knowledge structure) is more critical for predicting agreement in performance ratings. The current study will also examine the possibility that similarities in IPT may be associated the previously observed effects of demographic variables on SOA. Roberson, Galvin, and Charles (2008) argued that one s group identity was an important factor in influencing bias in performance appraisals. The authors draw on research on stereotype fit and relational demography to explain how differences in demography between raters may play in important role in the performance evaluation rating that one receives. For example, the stereotype fit approach suggests that the performance ratings that individuals receive may be dependent on the match, or fit, between the individual s 33

44 group membership and the expectations of the attributes that are necessary for the job. Incongruity between job and group membership is suggested to influence ratings through expectations of success. These propositions by Roberson et al. (2008) have been empirically supported in the SOA literature. Ostroff et al. (2004) proposed and found demographic similarities between rater and rate relate to SOA. Specifically, they suggested raters who were similar demographically would have higher levels of agreement because they had similar biases when making performance judgments. They found overrating was less severe when the racial composition of peers of non-white managers were also largely non-white. In addition, overrating was more severe when older managers were coupled with younger supervisors, peers, or subordinates. This idea is also supported by Rentsch and Klimoski (2001), who found teams with greater demographic similarity in terms of educational background and positional level had higher team member schema similarity. Their rationale was in part based on Zegner and Lawrence s (1989) argument that demographic similarities serve as proxies for shared life experiences. Their results also indicated that increased demographic similarity was associated with more and higher quality communications, which resulted in greater team member schema similarity. Following a similar logic, I propose H2: Demographic similarity (age, race, gender) will be related to similarity in IPTs. Specifically, manager-supervisor pairs who are of similar age, race, and gender will be more likely to exhibit higher levels of shared IPTs. 34

45 Role clarity and PSK: Inputs to Construction of IPT The second aim of the current study is to examine how similar IPTs may develop. A review of the SOA literature does not identify any studies which have previously examined antecedents to similarities in IPTs specifically. However, some tentative speculations may be drawn from research on teams and team (shared) mental models. Increased interaction and communication, whether via formal routes such as training and formal socialization programs, or through more informal communication routes, seems to aid in the development of team shared mental models (Rentsch & Hall, 1994; Rentsch & Klimoski, 2001). Levesque, Wilson, and Wholey (2001) demonstrated that team members who communicate more with each other tend to develop shared mental models compared to teams whose members experience less communication with each other. Their findings are consistent with other investigators who have suggested that interactions between organizational members lead to the forming of a common frame of reference for interpreting organizational events (Klimoski & Mohammed, 1994; Schein, 1992). By analogy, factors that increase communications between supervisors and subordinates would be expected to result in more similar IPTs. In the current study, I propose that the extent of role clarity and perceived system knowledge (PSK) shown by employees is likely to predict whether their IPTs have greater similarity with their supervisor s IPT. Employees develop greater role clarity and perceived system knowledge through interactions with other organizational members, including both co-workers and supervisors. More specifically, role clarity is an important outcome of socialization (Wanberg & Kammeyer-Mueller, 2000). It refers to an employee s understanding of his or her job, expectations, and responsibilities (Bauer, 35

46 Morrison, & Callister, 1998). Measures of role clarity tend to be self-report, and thus reflect both an individual s comprehension of information about the job, and the extent to which sources of job information are consistent and easily accessible. In general, studies have shown that proactive information-seeking by newcomers tends to be positively associated with role clarity, as well as other organizational outcomes such as performance and job satisfaction (Bauer & Green, 1998; Wanberg & Kammeyer-Mueller, 2000). As newcomers seek and gather more information about their jobs, they are able to better understand what is expected of them (Wanberg & Kammeyer-Mueller, 2000). Thus, relevantly to the current study, when role clarity is high, employees are more likely to hold job expectations that turn out to be shared by relevant others in the organization, including the supervisor. Perceived system knowledge also develops through interaction and communication and has previously been shown to predict SOA (Williams & Levy, 1992). When PSK is high, it also is expected to indicate better employee understanding of job expectations and performance rating standards. For example, it has been suggested that one of the reasons individuals demonstrate leniency in self-assessments may be the lack of requisite information to make an informed assessment (Farh & Dobbins, 1989). In a similar vein, Williams and Levy (1992) argued that individuals who know what is expected of them, understand the appraisal system, and agree with the performance standards are less likely to inflate their self ratings. The authors found that perceived system knowledge (PSK), defined as one s knowledge of the appraisal system, moderated the relationship between self-assessments and supervisor ratings, such that the relationship between self and supervisor ratings was higher for individuals with higher 36

47 levels of PSK. In other words, self and supervisor ratings were more correlated for individuals who were familiar with the appraisal process. In sum, role clarity and perceived system knowledge may both potentially be important proximal inputs to the content and structure of an employee s IPT. In addition, as similarity in IPTs between manager and supervisor increases, SOA is also expected to increase. Although these three constructs that lead to agreement (i.e., similarity in IPTs, role clarity, and PSK) have some conceptual overlap, note that role clarity and PSK differ from IPTs in that they do not directly assess the respondent s cognitive representation of effective performance, but instead address the employee s perception that he or she understands what is expected in the work role and performance evaluation context. In contrast, IPTs are cognitive structures that represent how specific work-related behaviors relate to each other for effective performance. Thus, I propose H3: Role clarity will be positively related to similarities between self and supervisor in IPT. H4: Perceived system knowledge will be positively related to similarities between self and supervisor in IPT. 37

48 Table 1 Summary of Research Hypotheses Hypothesis 1 Hypothesis 2 Hypothesis 3 Hypothesis 4 Similarity between self and supervisor in implicit performance theory will positively related to self-other agreement. Demographic similarity (age, race, gender) will be related to similarity in IPTs. Specifically, manager-supervisor pairs who are of similar age, race, and gender will be more likely to exhibit higher levels of shared IPTs. Role clarity will be positively related to similarities between self and supervisor in implicit performance theory. Perceived system knowledge will be positively related to similarities between self and supervisor in implicit performance theory. Subordinate Role Clarity Subordinate Perceived System Knowledge H3 H4 Similarity in Implicit Performance Theory H1 Self-Supervisor Agreement in Performance Ratings Similarity in: Age Race Gender H2 Figure 2. Complete Hypothesized Model 38

49 CHAPTER III METHOD AND RESULTS FOR PILOT STUDY This chapter describes the rationale, procedure, results, and findings of the pilot study. The pilot study was undertaken primarily to fine-tune the procedures used to operationalize manager and supervisor implicit performance theories via Pathfinder, to determine agreement between those two IPTs, and to address the appropriateness of the proposed manner of collecting ratings of performance from both manager and supervisor. Pathfinder One major challenge of investigating SOA from a cognitive perspective is the difficulty in identifying measurement techniques that can adequately capture the characteristics of one s IPT. Mental models, such as implicit theories, are characterized as cognitive structures or network of associations between concepts in each individual s mind (Ward & Reingen, 1990). These concepts are viewed as organized and linked together in a manner that allows the individual to efficiently interpret a situation and behave appropriately. Identifying an appropriate measurement technique that can reflect both the content (i.e., the concepts) and structure of a cognitive representation is important because it increases our confidence that we are accurately measuring relevant aspects of IPTs. As noted in the previous chapter, previous research investigating similarities between the IPTs of self and others have relied on correlations as indicators of 39

50 similarities in implicit performance theories. Such correlations essentially compare the rank ordering of scale scores between different raters. They do not, however, measure similarities in the way in which the performance domain is structured. This study extends previous research on implicit performance theories by adopting Pathfinder as a tool to index similarities between self and supervisor in implicit performance theories. Pathfinder was developed primarily to capture the underlying knowledge structure of a particular domain for an individual. Specifically, it is a relatively new technique designed to produce appropriate psychological scaling with respect to the underlying structure of relationships between concepts (Schvaneveldt, 1990). Pathfinder has been used frequently in the educational and training research. In many of those studies, investigators were interested in the extent of match between the knowledge structure of novices and experts, or of trainees with instructors. Pathfinder has also recently been used in research examining team mental models. A team mental model represents team members' shared, organized understanding and mental representation of knowledge or beliefs about key elements of the team's relevant environment (Klimoski & Mohammed, 1994). Team mental models, as determined by Pathfinder, have been shown to be useful for predicting team performance. For example, Edwards, Day, Arthur Jr., and Bell (2006) found that similarities in knowledge structures between team members and experts were positively related to team performance. Method The following sections review the rationale, methodology, and results of the pilot study. 40

51 Purpose of Pilot Study A pilot study was conducted to determine whether the proposed data collection and measurement methods were viable. In particular, it was important to develop and validate the proposed measurement method for one of the key constructs in the model: the implicit performance theories (IPTs) of the subordinates (who are at the managerial level) and their supervisors. It was planned to create a network representation of the IPTs by asking these managers and their supervisors to rate a series of paired behaviors that the managers might be expected to engage in as part of their job performance. When using the Pathfinder program, individuals are presented with pairs of terms ( constructs ) that have been previously determined to be potentially part of the content of the mental model being investigated. If there are 10 terms, Pathfinder will present all possible pairings of those 10 terms. Respondents are then asked to think about how the paired terms are related and rate the extent to which they are similar or share certain characteristics. The Pathfinder program then uses an algorithm to transform raw paired comparison ratings into a network of nodes. The nodes represent the psychological concepts, their relationships with each other are represented as linkages. As a result, Pathfinder allows for a deeper and richer representation of how an individual conceptualizes effective performance. Pathfinder can also be used to evaluate various characteristics of networks, and to quantitatively compare different networks. In the current study, Pathfinder will be used to directly assess the similarity in implicit performance theories. As mentioned, I/O psychologists have previously used Pathfinder in research on team mental models (e.g., Edwards et al., 2006). In this research application, although 41

52 Pathfinder had traditionally been used to compare an expert s mental model to that of novices, researchers were able to successfully demonstrate the use of Pathfinder for comparisons of the mental models of non-experts such as team members (Mohammed et al., 2000). For example, Lim and Klein (2006) used Pathfinder to assess similarities in mental models between team members on a military task. Nevertheless, this approach has not previously been used to measure an individual s implicit performance theory for more conventional office based jobs. Therefore, the first objective of the pilot study was to determine whether the specific set of behavioral performance items chosen for this study could be used in conjunction with Pathfinder to create meaningful network representations of IPTs. It was also necessary to demonstrate the resulting mental models could then be compared across supervisorsubordinate pairs to determine how similar they are. In particular, several outcomes from the pilot study would support the use of these behavioral items and the use of Pathfinder. First, it was important to demonstrate that both the managers and their supervisors found the particular behavioral items rated to be relevant to the managers leadership performance. This was addressed by asking all pilot study participants to rate how essential each of the behaviors was to effective managerial performance for the manager. Second, it was important to show that the paired rating data for the set of behaviors could be analyzed using Pathfinder, with reasonable network solutions resulting from the analysis. Third, the pilot study results should give some assurance that there would be variability in the network representations, such that at least some manager-supervisor pairs had substantial agreement on the behaviors, while others did not. 42

53 Assuming that pathfinder proves to be a reasonable tool for representing IPTs, the data from the pilot study also could serve a second purpose. It was hoped that a pilot study would help in determining the minimum number of elements that needed to be paired and rated to create a shorter measure of IPT. Minimizing the number of elements is important in order to reduce the demand on survey participants, as they will not only have to rate the paired elements for the mental model, but also complete other measurement instruments as well. The pilot IPT measure had items for 20 different work related behaviors, thus participants were asked to rate a total of 190 possible pairings of the work behaviors. Other studies using Pathfinder (e.g., Edwards et al., 2006) have used as few as 14 items. The pilot study provides information that can be used to determine if a 14-behavior IPT measure (requiring the rating of only 91 pairs) could be just as predictive as the 20- behavior version of the measure. To address this question, networks from self-supervisor dyads with low and high similarity ratings, as well as the average networks for all managers and supervisors were examined before selecting seven behaviors each from the task- and relationship-based dimensions. Using these 14 behaviors, the C index was recalculated and all analyses involving C was repeated to determine if the initial results using the 20 behaviors replicated adequately with the shorter measurement instrument. Finally, the pilot study could yield a sense of whether the proposed method for soliciting participants by asking students to hand copies of survey packets to a manager was a reasonable one. The analysis of the pilot study data could also give a sense of whether or not the proposed relationship between similar IPTs and self-other agreement in performance ratings has empirical support. 43

54 Participants A convenience sample of managers and their respective supervisors was solicited by asking students in psychology courses to hand a survey packet to a relative or acquaintance who currently holds a managerial position. The manager completed his/her portion of the survey, and then gave another survey packet to his/her supervisor to complete. The manager and supervisor returned their own completed paper-and-pencil surveys by mail (in separate envelops) to the researcher. The survey packets for each manager-supervisor pair were given a unique number code which was used to match surveys. Three hundred paired surveys were distributed. Completed surveys were received from 99 managers (33%) and 76 supervisors (25.3%). Of these, it was possible to match data from 61 pairs. Measures The following sections review the measures used in the pilot study. All pilot study survey materials are included in Appendices A and B. Creation of Implicit Performance Theory measure. As mentioned earlier, in studies that use Pathfinder, investigators typically begin by creating a list of concepts or behaviors that are expected to relate to the mental construct being characterized. These behaviors are then paired with each other and the individual is asked to rate the extent to which members of each pair relate to each other. In the current pilot study, 20 managerial-related work behaviors were generated. The specific behaviors were adapted from measures of managerial work behavior used previously by Diefendorff and Silverman (2000) and by Yukl, Wall, and Lepsinger 44

55 (1990). Sample items included Make decisions independently and Reward individual performance. In the IPT measure, each of the 20 behaviors was paired once with all remaining behaviors, yielding a total of 190 possible pairings. Survey participants were asked to rate the paired behaviors with respect to the extent to which they are likely to co-occur in the manager s job. A 9-point Likert scale ranging from does not occur together at all (1) to occurs frequently together (9) was used. In order to generate representations of mental models within a knowledge domain, Pathfinder uses a network scaling procedure to summarize and graphically display relatedness ratings (Schvaneveldt, 1990). A pathfinder network is derived from proximities for pairs of concepts. Similarities, correlations, and probabilities could all be used to derive the proximity network. In the current study, proximities are obtained from research participants ratings of the co-occurrence of paired managerial behaviors. In a Pathfinder network, the nodes generated in the network correspond to the concepts while the links in the network are determined by the patterns of proximities among concepts. The current study uses several indices that either describe the characteristics of individual participants networks (i.e., number of links, coherence), or address the extent to which the network of a manager is similar to that of his or her supervisor (i.e., C index). Each of these is described in more detail in the following paragraphs. The number of links in a network represents a count of all the connections between concepts for a particular network. The similarity of two networks is determined by examining the correspondence in their links. More specifically, similarity is expressed as the ratio or proportion of the number of links held in common by the two networks to 45

56 the total number of links in the two networks. Each manager s IPT was measured and compared with his or her supervisor s IPT to derive the Pathfinder metric of closeness, C. The C index ranges from 0 to 1, with scores closer to 1 representing greater similarity. In other words, a higher C rating indicates higher similarities in IPTs between manager and supervisor. In the current investigation, this quantitative index of similarity is hypothesized to be a proximal predictor of SOA because it represents the degree of overlap and commonality between manager and supervisor in perceptions of behaviors that co-occur in order to contribute to effective job performance. In addition to the similarity score and number of links, the Pathfinder program also provides other types of metrics for each network. The current study also examined network coherence, which refers to the level of consistency in the data that generate a single network. The essence of coherence is based on the assumption that the relatedness between a given pair of concepts should be able to be predicted by the pattern of how all remaining concepts relate to each other. For each possible pair of concepts in the model, an indirect measure of relatedness is obtained by correlating the proximities of all items with each other. Then a coherence index is computed by going across all pairs, and correlating the direct measure of proximity between a pair of concepts with its indirect measure. Higher average correlations indicate that the original observed proximities tend to be consistent with the relatedness that has been inferred from the indirect relationships between the items. Although not part of the hypothesized model, the number of links and coherence were collected for the purpose of addressing the issue of whether the IPT networks 46

57 derived from the pilot data have desirable properties as well as to examine their possible relationships with other study variables. SOA of managerial-related behavior ratings. To determine agreement in the performance ratings, the ratings from managers and their supervisors were assessed using a set of items referencing the same 20 managerial-related behaviors as the IPT measure. Managers rated themselves on how well they perform each of the behaviors. In addition, supervisors rated their subordinates (i.e., the manager) using the same items. A 5-point rating scale with endpoints labeled marginal (1) and outstanding (5) was used. Performance of managerial-related behaviors. Several alternative methods of creating composite performance rating scores were used for exploratory purposes. First, an overall performance composite for each source that is, manager (self-rating) and supervisor (other rating) was calculated by averaging across all item responses for each respondent. These composites were used in analyses looking at the construct validity of the performance measure, and were also used in the multivariate regression analyses of the IPT-SOA relationship. Second, an item-by-item absolute difference composite was constructed by summing the absolute differences in ratings of the manager and his or her supervisor for each of the behavior items. The sum of the absolute differences was then used as an index of agreement. (Please note that higher values of this SOA absolute difference score indicate greater disagreement in ratings between the manager and supervisor.) Perceived essentialness of managerial-related behaviors. Both managers and supervisors were also asked to rate the extent to which each of the 20 managerial-related behaviors used to construct the IPT and performance rating measures was essential for 47

58 performing effectively on the job. For each behavior, raters used a 5-point rating scale with response anchors ranging from not essential at all (1) to absolutely essential (5) to indicate the degree to which the behavior was important for job performance. The absolute difference method used to calculate similarities in performance ratings was also adopted to capture and measure similarities in essentialness ratings between manager and supervisors. Specifically, the absolute difference in essentialness ratings between each manager and his or her supervisor for each item was calculated. The sum of the absolute differences was then used to indicate the extent to which managers and supervisors have similar perceptions of how these behaviors are essential for effective leadership performance. Once again, higher values on this essentialness absolute difference score indicate greater disagreement between the manager and supervisor regarding perceptions of whether these behaviors are essential for effective performance. Demographics. Managers were asked to report their age, gender, dyadic tenure with supervisor, and job tenure to assess whether these characteristics were associated with characteristics of the IPT, the essentialness ratings, performance ratings, or SOA. Analyses In the pilot study, the relationship between agreement in IPTs and self-other agreement on the performance ratings was tested using two different approaches. The first approach simply correlated the SOA absolute difference score with the Pathfinder similarity index, C, for the respective manager-supervisor pairs, thus directly examining whether similarity in IPTs relates to SOA. It is an approach similar to that used in previous research on SOA, which has operationalized agreement in performance ratings by either using difference scores or categories of agreement (i.e., in-agreement, overrater, 48

59 and underrater). However, such direct approaches have been criticized as methodologically flawed when certain assumptions are not met. Furthermore, collapsing self and other ratings into a single difference score does not allow one to examine the precise nature of the agreement. Thus, a second set of multivariate regression analyses was also conducted to determine the influence of agreement in IPT (as indicated by the C index from pathfinder) on SOA, using Edwards (1995) approach. Edwards approach separates the effects associated with each rating source by estimating separate regression models for each component of the dependent variable. Specifically, the effects of agreement in IPT (as represented by the value of the C index) on the managers and supervisors difference in overall performance composites may be tested using the following two equations. Y 1 = b 10 + b 11 X + b 12 W + b 13 WX + e 1 Y 2 = b 10 + b 21 X + b 22 W + b 23 WX + e 2 In the context of the current study, Y 1 and Y 2 are the manager and supervisor ratings of performance. X is the value of the C index for a given manager-supervisor pair. To capture the absolute value aspect of the proposed difference, a dummy variable W, is also introduced. W equals 0 when Y 1 > Y 2 and equals 1 when Y 1 < Y 2, thus W codes differentially for situations in which the manager s rating is higher than the supervisor s rating (e.g., over-rating ) versus when the supervisor s rating is higher than the manager s rating (e.g., under-rating ). This procedure for coding over and under estimators follows the recommendations of Edwards (1995). The estimated coefficients (b s) from these equations can be used to examine the effects of C on manager and supervisor ratings of performance when Y 1 > Y 2 and Y 1 < Y 2. 49

60 In this approach, multivariate regression analyses are used to estimate the coefficients for the effects of X, W, and WX on Y 1 and Y 2 jointly, and to determine their statistical significance. If the coefficient for X is significant, then follow up univariate regression analyses can be conducted separately to determine the exact nature of the relationship of the predictor (C) with manager and/or supervisor ratings. Furthermore, if the coefficient for WX is significant, then it suggests that the effect of C differs depending on whether Y 1 is greater than or less than Y 2. Again, follow up tests can be conducted to examine the effects of C when Y 1 is greater than or less than Y 2. All analyses for the pilot study were conducted using SPSS v.12. Results Preliminary analyses and data checks undertaken to determine the appropriateness of the pilot sample and the proposed use of Pathfinder to operationalize IPTs are first described. This is followed by a presentation of more detailed analytic results from the pilot sample. Appropriateness of Pilot Sample The main purpose of the pilot and current study is to investigate similarities in IPTs between managers and their supervisors. Therefore, before any other analyses were conducted on the pilot data, demographic and work-related responses were inspected determine whether respondents appeared to meet this criterion. Manager respondents in the pilot study averaged 41.4 years of age (range=20 to 69). About half of these respondents (49.2%) were male. Their average tenure with their current supervisor was 6.13 years. In general, this age, tenure, and gender composition seemed consistent with what would be expected of managerial respondents. 50

61 Even more important, however, was to ensure that only participants with managerial job titles were included in the analyses. On the basis of job title, it was decided that 15 of the 91 managerial participants would not be included in the analyses because their job titles did not appear to indicate a managerial position. Two participants were also excluded because they did not indicate their job title. Examples of job titles for participants that were excluded from the analyses were nanny and lab technician. Examples of job titles of participants who were retained included tax and accounting manager and portfolio control manager. Out of the remaining 74 participants with managerial titles, responses from 54 were matched with their respective supervisor ratings. Thus, it does seem that at least a large portion of the surveys were being handed out to persons who held appropriate types of managerial positions. It also highlights the importance of asking for these types of open-ended descriptive information in the focal study as well, so that data from inappropriate respondents may be dropped from further analysis. Relevance of Managerial-related behavior Items for Assessing Manager Performance Table 2 presents descriptive statistics for the essentialness ratings of the 20 managerial-related behaviors that constituted the IPT and SOA measures. Based on a seven-point rating scale, the average composite essentialness rating for the whole set of 20 behaviors is 5.11 (SD=.70) for managers and 5.08 (SD=.63) for supervisors. These two means were not significantly different from each other (t=.26, p=.79), suggesting that in general managers and their supervisors agreed on the set of behaviors as being essential to good managerial performance. In addition, out of the 40 means for the individual behavior items (20 for managers and 20 for supervisors), 35 had an average 51

62 rating of 4 or higher on the seven-point rating scale. This again suggests that both managers and supervisors tended to see these behaviors as essential for effective performance, and supports their appropriateness for use in the IPT and SOA performance measures. 52

63 53

64 Pathfinder Network Solutions Table 3 (see middle section) shows descriptive statistics for the Pathfinder network representations. There was sufficient variability in the number of links in the models and values of the similarity index (C), as well as evidence that coherence was above a desired level of.20 for most respondents. Table 3 Pilot Study: Sample Descriptive Statistics for Demographics, Pathfinder Indices, and Performance Measures, with Significance Tests for Manager vs. Supervisor Differences N Min Max M SD t Demographics Manager age Manager gender (% male) Manager tenure with supervisor Pathfinder Network Variables Similarity Similarity Coherence-20 (Mgr) a Coherence-20 (Sup) Coherence-14 (Mgr) Coherence-14 (Sup) Number of links-20 (Mgr) Number of links-20 (Sup) Number of links-14 (Mgr) Number of links-14 (Sup) Composite Performance Variables Perf-20 (Mgr) Perf-20 (Sup) Perf-14 (Mgr) Perf-14 (Sup) Perf-20 (Diff) Perf-14 (Diff) a Coherence generally refers to the level of consistency in the Pathfinder data used to generate each network. Note. Variables with 20 are calculated using 20 behaviors. Variables with 14 are calculated based on 14 behaviors. p <.10 54

65 Six managerial performance behaviors (specifically, those listed in Table 1 which are not marked with # ) were removed to determine whether it was possible to reduce the length of the IPT measure without sacrificing the quality of the measurement. These items were chosen after a careful inspection of the average manager and supervisor networks. Nodes that were not centrally located within the network were identified as potential behaviors to be trimmed. After a behavior was deleted from the original list of 20 items, the Pathfinder indices were re-computed and inspected to ensure that the indices for the shortened version was not too different from the original 20-item measure. This procedure was conducted until a suitable 14-item IPT measure was obtained. Thus, many of the pilot results reported here will include analyses performed twice, once with the 20-item version of the IPT or performance measure, and once with the 14-item version of the same measure, in order to demonstrate the comparability of the results based on the 20- versus 14-item measures. The two different versions of the IPT network representations were similar in many respects. Correlations between the 20-item and 14-item versions of the measures were.89 for C,.83 for manager coherence,.89 for supervisor coherence,.95 for number of links for manager, and.93 for number of links for supervisor. (Corresponding 20- and 14-item essentialness and performance ratings for managers and supervisors were also made, these too were very similar.) To give a flavor of the network representations generated with Pathfinder, several figures are presented. Figures 3 and 4 show the mental models for a manager-supervisor pair with high similarity while Figures 5 and 6 show the mental models for a managersupervisor pair with low similarity. In general, the locations of the nodes for Figures 3 55

66 and 4 share very similar positions. For example, consider the similarity of locations and interconnections for four nodes that are central in both of these networks: behaviors #11 ( help group members resolve conflict ), #12 ( let group members resolve conflict themselves ), #13 ( be professional when dealing with subordinates ), and #15 ( check on subordinates regularly ). That these particular concepts be central and substantially interlinked makes conceptual sense, as well as the similarity in the two network structures suggesting that both manager and supervisor have similar mental models of performance. 56

67 Figure 3. High C Index Pair: Network of Manager Figure 4. High C Index Pair: Network of Manager s Supervisor 57

68 On the other hand, the locations of the nodes for Figures 5 and 6 (see next page) have little or no overlap, indicating a lack of commonality in mental models between manager and supervisor. The right hand column of Table 3 (previously presented at the start of this section) presents results from paired-sample t-tests conducted to examine whether there were mean differences between managers and supervisors on network coherence, number of links in the network, and managerial performance for both the 20-item and 14-item measures. For both full and shortened versions, the t-tests revealed that the differences in mean scores between managers and supervisors on these three variables were not statistically significant, although the difference for number of links for the 20-item measure was marginally significant (t = 1.71, p <.10). Although not significantly different, the means of coherence scores for the managers are higher than the means of coherence scores for supervisors. This is not too surprising. It suggests that, in general, the networks of the managers show more consistency than do their supervisors networks. This result is to be expected given the managers greater day-to-day familiarity with their own jobs. In addition, the average ratings given by managers to their own performances are also slightly higher than are their supervisor s ratings. This is consistent with previous SOA research which often finds that self ratings are higher than ratings from other sources, including from the supervisor (e.g., Yammarino & Waldman, 1993). 58

69 Figure 5. Low C Index Pair: Network of Manager Figure 6. Low C Index Pair: Network of Manager s Supervisor 59

70 Relationships among Various IPT Indices The next sections of this chapter present selected analyses addressing the relationships: (a) among the IPT indices, (b) among the various performance variables, and (c) between the IPT similarity index and agreement in performance ratings. Table 4 (on page 62) shows the correlations among the pilot study variables. Relationships based on the 20-item versions of the IPT and performance measures are in the lower diagonal, while relationships based on the 14-item versions of these measures are in the upper diagonal of the matrix. Only those correlations in the matrix that have direct implications for the hypothesized model and the design of the focal study will be described in detail in the text. First, interrelationships between the various Pathfinder indices are addressed. In particular, it might be asked what other Pathfinder indices relate to the C index, which reflects similarity in IPTs between managers and supervisors. Both the 20- and the 14- item-based C index significantly related (p <.05 or better) to several other Pathfinder indices. The magnitudes of these correlations were similar for the 20-item and the 14- item versions of the indices, and are reported in that order. Specifically, C is positively related to coherence for supervisors (r =.32,.29), number of links for managers (r =.33,.35) and number of links for supervisors (r =.44,.37). Not too surprisingly, one conclusion from these results is that similarity was likely to be higher as the number of links in the model of either dyad member increased. This result will occur by definition that is, as there are more links, the likelihood that any one of them is also present in the other dyad member s model increases. Another important conclusion is that the 14-item 60

71 based Pathfinder solutions seem to be very similar to the 20-item based solutions, which supports using the shorter version in the focal study. 61

72 62

73 Relationships among managerial performance measures. As would be expected, there was a moderate level of positive association between the managers and supervisors composite performance ratings, with a significant r between the two sources of.40 for the 20-item version of the performance measure and an r of.35 for the 14-item version (p s <.01). This result is consistent with other SOA literature, which suggests moderate levels of agreement in performance ratings between self and supervisor. This result also again reinforces the belief that the behavior items used in this measure are relevant to the jobs of the managerial respondents. Next, correlations of managerial performance scores with the SOA absolute difference index were inspected. The SOA absolute difference index significantly correlated with the supervisor performance composite (r = -.66 for the 20-item measure and r = -.60 for the 14-item measure, p s <.01), but not with the manager performance composite. This suggests that the absolute difference index tended to be larger in situations where the supervisors rated performance systematically lower. The absolute difference index also correlated marginally with the supervisor essentialness ratings (r = -.27, p<.10) for the 20-item version, and significantly for the 14-item version (r = -.31, p <.05). This suggests that when the supervisor tended to view the behaviors as less essential to the manager s job, there was greater disagreement in ratings. Relationships between IPT indices and performance measures. Consistently with the theoretical model developed in earlier chapters of this dissertation, the empirical results from the pilot study found that similarity in IPTs (indicated by the C value) related to the SOA absolute difference index (r = -.43, p <.01). The negative sign of this relationship occurs because greater similarity is associated with smaller absolute 63

74 differences in ratings. Interestingly, of the three Pathfinder indices, only the C index significantly related to the SOA absolute difference index. This suggests that, as proposed, similarities in IPTs between manager and supervisor may be a meaningful construct to investigate as a proximal predictor of SOA. Therefore, this finding from the pilot study supports one of the main hypotheses in the study (H1). Finally, multivariate regression analysis was conducted to examine the relationship of C with manager and supervisor performance ratings. The Wilk s λ for this model was marginally significant (Wilk s λ =.91, F 2,50 = 2.49, p <.10). In addition, Wilk s λ for the product term of W with C was non-significant (Wilk s λ =.92, F 2,47 = 1.93, p =.16). Summary of Pilot Study The objectives of the pilot study were twofold. First, a set of 20 managerialrelated behavior items was compiled to determine whether they could be used with Pathfinder to measure similarities in IPT between manager and supervisor. In general, these 20 items were rated by both managers and their supervisors as essential for managerial performance, thus giving preliminary support for the construct validity of their use in constructing the IPT and performance rating measures. In addition, the pilot study results showed that the managerial performance items produced reasonable and conceptually interpretable network representations when entered into the Pathfinder analysis. Managers and their supervisors varied in similarity from pair to pair (as indicated by the C index). This was a necessary precondition for testing the key study hypothesis of a relationship between similarity in IPTs and SOA. And indeed, the pilot study results provided preliminary evidence that similarities in IPT 64

75 between managers negatively relate to absolute differences in performance ratings between manager and supervisor. This suggests that higher levels of similarity in IPT were associated with greater agreement in performance ratings between self and other, as proposed in the theoretical model. In contrast, the results from the test of this relationship using a multivariate regression approach were not statistically significant. However, this may at least in part be due to issues of statistical power, which is substantially lower for the multivariate technique than for the univariate technique at the pilot study sample size of 54 paired responses. The second aim of the pilot was to determine if a shorter, 14-item version of the IPT and performance measures would perform as well as the 20-item version. The pattern of results using the shortened version was very similar to the 20-item measure. Therefore, the shorter version will be used in the focal study so as to reduce the demands on survey participants. 65

76 CHAPTER IV METHOD AND RESULTS FOR FOCAL STUDY Chapter IV presents the procedures and empirical results from the focal study. The focal study allows a full test of the hypotheses developed in Chapter II. The current chapter is accordingly divided into two main parts, one describing method used and the other describing results. Method As was done in the pilot study, the main study also used a survey procedure to collect measurements of manager and supervisor IPTs (via Pathfinder) and their related performance ratings. Only the reduced set of 14 leader behavior items was used for this data collection. Measures of all variables presented in Figure 2 were also collected, including measures of all relevant control variables. Thus, the data collected in this focal study were used to test the hypotheses advanced in Chapter II. Sample The procedure used to recruit managers in the primary study was similar to the procedure described for the pilot study. However, for the focal study, participants were recruited from both The University of Akron (366 invitations to participate were distributed to managers) and The University of South Florida (243 invitations to participate were distributed to managers). This was done in order to get a large enough sample size within a reasonable time frame. The secondary de-identified data set from 66

77 The University of South Florida was then merged to supplement the data obtained from The University of Akron. In contrast to the pilot study where participants were required to complete a paper-and-pencil survey, all participants in the focal study completed the survey online via Survey Monkey. In total, 284 (46.4% response rate) managers and 225 supervisors (36.9% response rate) responded to the survey. These responses yielded 159 matched manager-supervisor pairs. Measures The managerial survey for the primary study included the shortened IPT and all measures in the hypothesized model (Figure 2), as well as some potential control variables (essentialness ratings, PANAS, leader-member exchange, performance appraisal training, self-esteem and collectivism). The supervisor survey for the primary study included only demographics, control variables (essentialness ratings, PANAS, performance appraisal training), the shortened IPT, and leadership performance measures. Copies of these surveys are included in Appendix C and D. Role Clarity. Sawyer s (1992) 10-item measure of role clarity was used. The measure has two dimensions. Goal clarity refers to the extent to which outcome goals and objectives are clearly stated and communicated. Process clarity refers to the extent to which the individual knows how to perform his or her job. The measure used a six-point rating scale ranging from very uncertain (1) to very certain (6). All items were aggregated to compute an individual s overall role clarity score. Perceived System Knowledge. Perceived system knowledge was measured with Williams and Levy s (1992) 11-item scale. The scale measures how much knowledge employees report having concerning the performance appraisal system. Each item was 67

78 measured with a 7-point Likert scale ranging from (1) strongly disagree to (7) strongly agree. Sample items include I understand the performance appraisal systems being used in my company and I would benefit from additional training in the process of the appraisal system. Implicit Performance Theory. The shortened version of the Pathfinder measure based on analysis of the results from the pilot study was administered. Similar to the pilot study, participants were presented with pairs of managerial-related behaviors and asked to rate, for each pair of behaviors, the extent to which they frequently occur together when performing effectively on the job. The 14 managerial-related behaviors identified from the pilot were paired with each other, yielding a total of 91 pairs. A 9-point Likert scale ranging from does not occur together at all (1) to occurs frequently together (9) was used. In addition to the Pathfinder indices collected in the pilot study (i.e., similarity (C), number of links, and coherence), the focal study will also include another Pathfinder metric, r-data. R-data refers to the aggregated correlation of the individual ratings (or proximity data) between two underlying networks. The correlation between proximity data is accomplished by simply correlating the corresponding individual rating entries in the two networks (i.e., manager and supervisor). Thus, this metric (r-data) indicates the extent to which managers and supervisors are providing similar ratings for each pairing of work-related behaviors. This is substantively different from similarities in the mental models since r-data does not incorporate the organization of these behaviors. However, because of r-data s potential influence in the focal study, it was included as a possible control variable. 68

79 Performance on managerial-related behaviors. Performance was assessed using the 14 work-related behaviors from the IPT measure. Managers rated themselves on how well they performed on each of the behaviors. Similarly supervisors rated their subordinates (i.e., the manager) using the same items. A 5-point rating scale ranging from marginal (1) to outstanding (5) was used. Overall performance for each source (i.e., self and supervisor) was calculated by aggregating all the items for each individual. In addition, the absolute difference score used in the pilot as an indicator of agreement was also calculated and used in the focal study. Essentialness of managerial-related behaviors. Similar to the pilot study, perceptions of whether each of the 14-items are essential for effective performance was collected for both managers and supervisors. The absolute difference index used in the pilot study was adopted for the focal study. Leader-Member Exchange. Linden and Maslyn s (1998) 12-item measure of leader-member exchange was used. Sample items include I like my supervisor very much as a person and My supervisor would come to my defense if I were attacked by others. A 5-point Likert-type response scale with anchors ranging from strongly disagree (1) to strongly agree (5) was used. Self Esteem. Rosenberg s (1965) 10-item measure of self esteem was used. Example items include On the whole, I am satisfied with myself and I feel that I have a number of good qualities. A 5-point Likert-type response scale with anchors ranging from strongly disagree (1) to strongly agree (5) was used. Collectivism. Dorfman and Howell s (1988) six item measure of collectivism was used. Example items include Group success is more important than individual 69

80 successes and Managers should encourage group loyalty even if individual goals suffer. A 5-point Likert-type response scale with anchors ranging from strongly disagree (1) to strongly agree (5) was used. PANAS. Watson and Tellegen s 20-item measure of positive and negative affectivity was used. Participants were asked to rate the extent to which they experience each of the 20 items in general using a 5-piont Liker-type rating scale with anchors from very slightly or not at all (1) to very much (5). Sample items include interested and irritable. In addition, similarities in PA and NA between manager and supervisor were calculated and included as possible control variables. The method used to calculate the absolute difference index for agreement in performance and similarities essentialness ratings was adopted for measuring similarities in PA and NA. Performance appraisal training. A three-item measure of performance appraisal training was developed for the study. Two items used a 5-point Likert-type response scale with anchors ranging from strongly disagree (1) to strongly agree (5). A sample item is I have been provided with a comprehensive overview of how my company's performance appraisal system works in general. Participants answered the third item I have attended a formal meeting or feedback session related to performance appraisal in my company within the past year by indicating a Yes or No response. Demographics. Age, gender, race, dyadic tenure with supervisor, and job tenure information were collected for both managers and supervisors. Age was measured in years. Male raters were coded as 0 and female raters were coded as 1. The ethnicity of managers and supervisors was coded 1 for whites, 2 for African Americans, 3 for Hispanics, etc. 70

81 A series of manager-supervisor agreement variables was also created, based on responses to the original age, gender and race variables. Specifically, a new variable indicating whether managers and supervisors are either of the same race (=0) or of different races (=1) was created. A similar gender agreement variable was also created. For similarity in age, an absolute difference in age between manager and his/her corresponding supervisor was calculated. For this variable, smaller values indicate greater similarity in age between manager and supervisor. These agreement variables were used as predictor variables in the regression models used for hypothesis testing. All analyses for the main study were conducted using SPSS v.12 and Mplus v Results The present section summarizes the results of the main study, including both descriptive statistics and the univariate and multivariate regression analyses conducted to test Hypotheses 1 to 4. Table 5 shows the means, standard deviations, and reliability coefficients for the study s control variables as well as for the focal study variables. As can be seen in the table, the internal consistency reliability of the measures is overall fairly high, ranging from.72 (Perceived System Knowledge) to.93 (Leader-Member Exchange). The managers had a slight tendency to be female (58%), and their supervisors a slight tendency to be male (46%). The managers and their supervisors were typically mid-career, with ages of 38.1 and 42.1 years respectively. Most of the managers had held their current job for over 5 years, M = 6.6 years, and been with their current supervisor for a relatively long period of time, M = 5.5 years. Similar to the pilot study, participants 71

82 who did not occupy a managerial level position were not included in all analyses. Subsequently, three participants were excluded because their job titles were listed as student, youth pastor, and inpatient nurse. Responses from 159 managers were matched to supervisor ratings. Descriptives from all participants are presented in Table 5. 72

83 Table 5 Main study: Means, Standard Deviations, and Cronbach s Alpha for Study Variables Manager Supervisor N M SD α N M SD α Demographic/Descriptive Variables Age Gender (0=M, 1=F) Race (0=W, 1=NW) Job Tenure Tenure with Sup Absolute Diff in Age Gender Similarity (0=same, 1=diff) Race Similarity (0=same, 1=diff) Individual Difference Variables Positive Affectivity (PA) Negative Affectivity (NA) Performance Appraisal Training Collectivism Self-Esteem Leader-Member Exchange (LMX) Perc d System Knowledge (PSK) Role Clarity Pathfinder Variables Similarity Coherence Number of Links R-data a Performance-Related Variables Essentialness Ratings Absolute Diff - Essentialness Performance Ratings Absolute Diff - Performance a R-data refers to the aggregated correlation of all Pathfinder ratings between a manager and his/her supervisor. 73

84 Tests of Hypothesis 1 Hypothesis 1 predicted a positive relationship between manager-supervisor similarity in implicit performance theories (as indicated by higher values of the C similarity index) and self-other agreement (SOA) in performance ratings. Specifically, it was proposed that manager-supervisor pairs which have more similar IPTs will exhibit higher levels of agreement in their performance ratings of the managerial-related behaviors. As was done for the pilot study, Hypothesis 1 was tested using two approaches: (a) analysis using a single dependent variable consisting of either an absolute value or an algebraic (i.e., signed) difference score, created by subtracting the supervisor s response from that of the manager; and (b) analysis using a multivariate model in which the managers and the supervisors performance rating scores are treated as multiple dependent variables. Test of Hypothesis 1 using an absolute difference score dependent variable. In a preliminary step designed to identify only that subset of the potential control variables which was necessary to include in the hypothesis testing model, a forward stepwise regression was conducted. In this analysis, the agreement index was regressed on all potential control variables. The potential control variables included: Pathfinder s r-data index, the number of links for manager and supervisor, coherences for manager and supervisor, positive and negative affectivity for manager and supervisor, performance appraisal training for manager and supervisor, manager and supervisor job tenure, manager s tenure with supervisor, collectivism, LMX, self-esteem, and similarities between manager and supervisor in essentialness ratings, PA, and NA. 74

85 Table 6 Main study: Correlations among Variables used for Hypothesis 1 (n=150) ) Similarity --- 2) R-data ) Manager Self Esteem ) Supervisor NA ) Essentialness (Absolute Diff) ) Performance (Algebraic Diff) ) Performance (Absolute Diff) Note. Values in bold are significant at p <. 05 or better. Variables with low magnitude and non-significant effects were eliminated from further consideration for Hypothesis 1. Specifically, control variables that were included in the forward stepwise regression model because they met the entry criterion of p <.05 were retained, but also retained were control variables that had been excluded from the analysis but had a p-value of less than.15. This decision rule resulted in the retention of the following control variables: r-data, manager self-esteem, similarities in essentialness ratings, and supervisor NA. Table 6 presents the bivariate correlations among the set of variables used in various tests of the relationship between C and SOA. Supporting the results established in the pilot study, C was found to be significantly related to the absolute difference measure of agreement in performance ratings. Table 7 shows the results from the full hierarchical regression analysis conducted to determine the effect of C on SOA. In Step 1, both r-data, B = -.17, p <.05, and the essentialness index, B =.35, p <.01, significantly related to the absolute difference SOA index. These results suggest that low manager-supervisor correlations in proximity ratings as well as greater manager-supervisor differences in perceptions of essentialness were associated with low SOA. 75

86 The Pathfinder similarity index (C) was then entered in Step 2 of the regression equation. C was negatively and significantly related to agreement, B = -.18, p <.05, indicating that managers and supervisors who have similar IPTs tend to also experience higher levels of SOA. Importantly, the significant unique effect of similarity was over and above the effects of the control variables, thus helping to rule out alternative explanations for the result. In sum, H1 was unambiguously supported by the results of analyses using the single difference score approach. Table 7 Main study: Hierarchical Regression of Self-Supervisor Agreement in Performance Rating (Absolute Difference) on Control Variables and Pathfinder Similarity (n=147) Step 1: Control Variables R-data Mgr Self Esteem Sup NA Essentialness Similarity R 2 Total Sample B SE -.18 * **.18 ** Step 2: Similarity ΔR 2 R 2 Note. Final model df = 5. p <.10 * ; p <.05; ** p < *.03 *.21 **.08 Follow-up analyses using an algebraic difference score dependent variable. In addition to examining the hypothesized effect of C on the magnitude of the SOA in performance ratings, further exploratory analyses were conducted to determine if these effects differed depending upon whether the managers were systematic over- or under- 76

87 raters of their own performance. (Over-raters are managers whose overall self-ratings on the managerial-related behaviors are higher than their supervisors ratings.) In order to do this, it was necessary to use a performance rating difference score which had a positive or negative sign, in order to distinguish not only the magnitude of disagreement between manager and supervisor, but also the direction of that disagreement. Conceptually, such potential differential effects of over- versus under-rating can be thought of as interactions between the sign of the performance rating discrepancy and the other predictor variables of interest. To test for differential effects for over- versus under-raters, a dummy variable (W) indicating the direction of the rating discrepancy was created. Following Edwards (1995) recommendation, it was coded W = 0 for over-raters and W = 1 for under-raters. This dummy code was also used to create multiplicative interaction terms consisting of products of W with each of the centered predictors. Therefore, to determine if the effects of C and control variables differed for over and under estimators, the algebraic (signed) difference in managers versus supervisors performance ratings was regressed on centered predictor variables in Step 1 of a hierarchical regression model. In Step 2, the interaction terms of W with each predictor variable were entered simultaneously. Table 8 (Total Sample columns) presents the results of these analyses. 77

88 Table 8 Main study: Hierarchical Regression of Self-Supervisor Agreement in Performance Rating (Algebraic Difference) on Control Variables and Pathfinder Similarity (n=147) Step 1: Control Variables R-data Mgr Self Esteem Sup NA Essentialness Similarity Similarity R 2 Total Sample B SE * Step 2 W R-data x W Mgr Self Esteem x W Sup NA x W Essentialness Similarity x W Similarity x W ΔR 2 R 2 p <.10; * p <.05; ** p < *.13* A marginal interaction between W and similarity, B =.23, p =.10 found. These results indicate differential effects of similarity on over- versus under-raters. Thus followup analyses were performed separately for the subsamples of over- and under-raters. However, the follow up analyses did not reveal any significant effect of similarity on performance ratings (algebraic difference) for over- and under-raters. Test of Hypothesis 1 using multivariate dependent variable approach. The second analytic approach, as recommended by Edwards (1995), involves determining the effects of the control variables and C on manager and supervisor ratings simultaneously in a multivariate model in which they are separate dependent variables in the analysis rather than a single discrepancy score. For this analysis, only the control variables that were 78

89 shown to significantly relate to the absolute difference SOA index in the preceding analyses were retained as predictors. Multivariate analysis revealed that self and supervisor ratings were jointly associated with C as hypothesized, Wilk s λ =.86, F 2,141 = 11.12, p <.01. In addition, there were significant multivariate relationships of the joint self and supervisor ratings with r-data, Wilk s λ =.93, F 2,141 = 5.31, p <.05; and similarity in essentialness ratings, Wilk s λ =.94, F 2,141 = 4.19, p <.05. The effects of manager self-esteem and supervisor NA on manager and supervisor performance ratings jointly were not significant. Follow-up univariate analyses were conducted by regressing manager and supervisor performance ratings separately on the control variables (r-data, supervisor NA, manager self esteem, and similarity in essentialness ratings) and on C. Results of these analyses are summarized in Table 9. Importantly, C was still significantly related to both manager and supervisor performance ratings, B s =.31 and.30, respectively, p <.01. In addition, R-data, B = -.27, p <.01 and manager self-esteem, B =.24, p >.01 were significantly related to manager performance ratings. Similarity in essentialness ratings was marginally related to manager self ratings, B = -.13, p <.10. Additionally, similarity in essentialness ratings, B = -.18, p <.05, were significantly related to supervisor performance ratings, while R-data, B = -.14, p <.10, was marginally related to supervisor performance ratings. These results suggests that agreement between managers and supervisors is due to the effects of r-data, manager self-esteem, similarities in essentialness, and C on managers ratings, and r-data, similarities in essentialness, and C on supervisor ratings. Thus, the multivariate test of Hypothesis 1 and its univariate 79

90 follow-up tests were consistent with earlier analyses in providing strong support for this hypothesis. Table 9 Main study: Follow-up to Multivariate Test of Hypothesis 1: Univariate Regression of Manager and Supervisor Performance Rating Component Scores on Pathfinder Similarity Index (C) and Control Variables (n=147) Constant R-data Mgr Self Esteem Sup NA Essentialness Similarity Similarity R 2 p <.10; * p <.05; ** p <.01 Manager Supervisor B SE B SE ** ** * ** ** **.15 ** Tests of Hypotheses 2-4 Hypotheses 2-4 examines the effects of demographic similarities (H2), role clarity (H3), and perceived system knowledge (H4) on manager-supervisor similarity in IPTs. Once again, the Pathfinder indices of r-data, number of links, and coherence were selected as control variables because it was important to determine if the hypothesized predictor variables were able to predict C over and above other Pathfinder indices. Other control variables, such as manager job tenure, supervisor job tenure, tenure with supervisor, PA, NA, and similarities between manager and supervisor in PA and NA were chosen because of their potential relationship with C. Similar to H1, a preliminary forward stepwise regression was first conducted to determine which control variables were significantly related to the dependent variable of manager-supervisor similarity in IPTs (i.e., C). R-data, number of links for manager and 80

91 supervisor, coherence for supervisor, manager and supervisor job tenure, and supervisor PA were retained as control variables as a result of these preliminary analyses. Table 10 (next page) shows the bivariate correlations between the control and focal variables used to test H2-H4. Table 11 shows the results from a hierarchical regression analysis conducted to test H2-H4. In Step 1 of the hierarchical regression, all previously selected control variables were entered as a group. R-data, B =.44, p <.01, as well as number of links for managers, B =.47, p <.05, and number of links for supervisors, B =.46, p <.01, were all positively related to C. Supervisor coherence was negatively related to similarity in IPTs, B = -.12, p <.05. These results indicate that as r- data and the number of links increase for managers and supervisors, so does the value of the C index, implying greater similarity in IPTs between manager and supervisor. Conversely, supervisors with low coherence scores tend to exhibit higher similarities in IPTs. 81

92 82

93 In Step 2 of the hierarchical regression, absolute difference in age, gender similarity, ethnicity similarity, role clarity, and perceived system knowledge were added as a group. Absolute age difference between manager and supervisor was significantly related to C, B = -.09, p <.05, but gender and ethnic similarities were not significantly related to C. This indicates that when the age difference between managers and supervisors is large, such pairings exhibit lower levels of similarity in IPTs. Thus H2 is partially supported. Table 11 Main study: Regression of Pathfinder Similarity on Hypothesized Predictor and Control Variables (n=123) Step 1: Control Variables R-data Mgr number of links Sup number of links Sup coherence Mgr job tenure Sup job tenure Sup PA R 2 Step 2: Hypothesized Predictors Age Diff Gender Similarity Ethnicity Similarity Role Clarity Perceived System Knowledge ΔR 2 R 2 p <.10; * p <.05; ** p <.01 B.44 **.47 **.46 ** -.12 * ** -.09 * *.04 *.73 * SE Role clarity was also marginally related to similarity in IPTs, B =.09, p <.10, providing partial support for H3. Finally, perceived system knowledge was significantly 83

94 related to C, B =.15, p <.05. Thus H4 is supported. These results suggest that managers with higher levels of PSK and role clarity tend to experience higher levels of shared IPTs. 84

95 CHAPTER V DISCUSSION The purpose of this study was to investigate a cognitive structure explanation of self-other agreement in performance ratings. Specifically, the main objective of the study was to examine the extent to which similarities in mental models with respect to work related behaviors may influence agreement in performance ratings between managers and their supervisors. Indeed, the importance of shared cognitive structures for the coordination of other workplace activities namely, teamwork -- has been previously demonstrated (e.g., Edwards et al. 2006, Lim & Klein, 2006). In studies of team activities, investigators often find that when team members interpret and cognitively organize work related behaviors in a similar way, positive outcomes such as increased performance occur. However, whether shared mental models have a role at the dyadic level for coordinating perceptions of individual level performance has yet to be extensively explored. Apart from Schrader and Steiner s (1996) study that demonstrated the importance of common comparison benchmarks in predicting agreement, little attention had been paid regarding the potentially important role of shared mental models in the SOA literature. Thus this study extends the current research on self-other agreement in performance ratings by proposing shared cognitive structures as a proximal predictor and explanation for SOA. 85

96 This chapter further describes and explores the implications of the major findings of the pilot and focal studies. In particular, a detailed discussion of how similarities in implicit performance theories between manager and supervisor may influence agreement is presented. Also, results pertaining to predictors of shared IPTs are also discussed. The chapter closes with a consideration of theoretical and practical implications, as well as some limitations of the collective findings. Overview of Findings The following section presents a review of the study s results and how they support previous research in self-other agreement. Implicit Performance Theory and SOA This study contributes to the current SOA literature by examining how similarities in cognitive structures may predict agreement in performance ratings over and above individual difference variables that have already been empirically related to agreement. Implicit performance theories represent an individual s schema regarding how workrelated behaviors are organized for the attainment of effective performance. Using an absolute difference score approach to index agreement, the results of this study indicate that when managers and supervisors organize such managerial-related behaviors in a similar manner, they are more likely to exhibit greater levels of agreement in performance ratings. The results support Schrader and Steiner s (1996) findings. However, as mentioned previously, the key difference between the two studies is that this study not only measures IPTs under a more naturalistic context, it also uses a tool that is able to accurately capture and quantify similarities in IPTs. 86

97 Some knowledge about antecedents of SOA existed before the current study, and those antecedents were included as control variables in tests of the effects of similarity in IPTs (i.e., C) on SOA. Importantly, the findings from the current study showed that C significantly predicted agreement over and above a variety of control variables. For example, past research shows self-esteem is an important antecedent of agreement because it impacts the way people perceive themselves (e.g., Farh & Dobbins, 1989), and indeed, in the current study self-esteem was found to have a positive relationship with the size of the discrepancy between manager and supervisor ratings of performance. Another important control variable in this study was the extent of similarity in managers and supervisors perceptions of the essentialness of each of the managerial-related behaviors for effective performance. Inclusion of this construct in the prediction model is critical because it was important to show that similarity in the organization of behaviors (i.e., shared IPTs) is conceptually different from perceptions of essentialness, and that shared IPTs have an effect on SOA beyond that of similar essentialness perceptions. Finally, r- data is a Pathfinder metric that is an aggregate of all the correlations between manager and supervisor for each proximity rating. That is, this metric indicates the extent to which managers and supervisors are providing similar ratings on the Pathfinder measure. However, much like the essentialness similarity index, it does not reflect similarities between manager and supervisor with respect to the cognitive organization of managerial-related behaviors. Thus, if C predicts over and above the r-data index, it indicates that prediction is based on more than simple similarity in the level of ratings. Manager self-esteem and supervisor NA were also shown to be significantly associated with agreement, supporting the findings from previous research. The fact that C was a 87

98 significant predictor of agreement over and above these control variables demonstrates its uniqueness as a construct as well as its relevance for influencing SOA. In addition to using an absolute difference score approach, the current study also adopts Edward s (1996) recommendation to decompose agreement into separate manager and supervisor performance ratings so as to better understand the effects of C on agreement. Edwards main argument for decomposing performance rating discrepancy into its individual components is because it allows for a clearer understanding of how predictors may have differential effects on each rating source. The results of the current main study indicate that similarity in IPTs between manager and supervisor has a significant effect on ratings from both sources separately. Also, the strength of the relationship of C with both sources was about the same. However, it is interesting to note that mangers and supervisors showed differences in how the control variables and C combined to influence performance ratings. Manager ratings were influenced by C, r-data, and manager self-esteem. However, supervisor ratings were influenced by C and similarity in perceptions of essential work behaviors. The findings of Hypothesis 1 support the theoretical model proposed by Campbell and Lee (1988). These authors suggest that differences in cognitive processes between self and supervisor may be responsible for discrepancies in performance ratings. Consequently, the authors propose that one prerequisite for manager-supervisors agreement on performance ratings is that both sources must conceptually organize the essential work related behaviors in a similar manner. Finally, the results of this study extend the SOA literature by empirically demonstrating that similarity in cognitive structures may have a significant impact on 88

99 agreement. Investigators in team-based research have been adopting a cognitive perspective to understand team outcomes for a number of years. For instance, the role of mental models has been studied in team research (e.g., Edwards et al. 2006, Lim & Klein, 2006). Empirical studies in the team literature have consistently shown how shared mental models have a positive impact on the performance of team members in accomplishing their team goals. Shared cognitions are critical for team-based activities because they help increase coordination and the anticipation of team member behaviors (Rentsch & Hall, 1994). However, while some researchers have proposed a cognitive model of SOA (e.g., Campbell & Lee, 1988), empirical studies that specifically examine how cognitions may influence agreement have been lacking. Therefore, this study demonstrates the importance of shared cognitive structures in predicting SOA. Specifically, similarity in IPTs between manager and supervisor appear to be a critical antecedent of agreement because it facilitates a common understanding and interpretation of important work behaviors, much like shared mental models in teams. Predictors of Similarity in Implicit Performance Theories The following paragraphs review the findings regarding constructs that were hypothesized to influence similarity in IPTs. Demographic similarities. In addition to investigating outcomes of shared IPTs, this study also explored constructs that were critical in predicting similarity in IPTs between manager and supervisor. This study proposed that when managers and supervisors were of the same demographic group in terms of age, gender, and ethnicity, such similarities should facilitate the development of shared IPTs. This proposition follows some of the earlier works by Ostroff et al. (2004) and Rentsch and Klimoski 89

100 (2001). These and other authors suggest that demographic variables may serve as proxies for shared life experiences. Furthermore, demographic similarity is also related to interaction frequency and quality, which are also potentially associated with the development of schema agreement. The results for Hypothesis 2 were partially supported. Specifically, managersupervisor similarity in age was significantly associated with shared IPTs, but similarity in gender and ethnicity were not. When the difference in age between managers and their respective supervisors is small, it may result in increased levels of common experiences, interpretation of events, and interaction. Such interactions may lead to increased discussions about job related activities, behaviors, and feedback. All these factors could serve to facilitate the development of schema agreement regarding the organization of essential behaviors for the job. Interestingly, while this study found empirical support for similarity in age and shared IPTs, Rentsch and Klimoski (2001) did not find age similarity to be a significant predictor of team member schema agreement. However, support was not found for effects of manager-supervisor similarity in gender and ethnicity on shared IPTs. Of the four bivariate correlations between demographic variables (i.e., manager gender and race, supervisor gender and race) and shared IPTs, only supervisor gender was significantly correlated with shared IPT (r=.18, p<.05). Specifically, female supervisors were more likely to exhibit higher levels of shared IPTs. The results for gender were similar to findings of Rentsch and Klimoski (2001). Their study also failed to find a significant gender similarity effect on team member schema agreement. Furthermore, while Ostroff et al. (2004) predicted that gender 90

101 similarity effects were expected to be found on agreement in performance ratings, their results were also non-significant. One possible explanation for the lack of empirical support for gender and race similarity effects might be the extent to which stereotyping could influence managers and supervisors perceptions of essential work behaviors. In fact, researchers in SOA frequently argue that stereotyping is often the main reason for the lack of agreement in performance ratings between self and other. For instance, Robbins and DeNisi (1993) argued that gender stereotyping may be situationally determined. They assert that situational factors may be responsible for the extent to which gender bias affects performance recall and evaluations. The role of gender schemas is important in SOA because gender-based categorization strategies not only have an influence on information processing, gender schemas also play a key role in how individuals organize, process, and evaluate information. The authors hypothesize that the profession of the individual s job may trigger gender-based stereotypes, especially when the ratee s gender is incongruent with the role implied by the job. The results of their study found that individuals in sex-incongruent professions tend to receive lower performance ratings. Stereotyping may also explain the lack of empirical support for ethnicity similarity effects. Specifically, non-white individuals are more likely to be perceived as occupying a lower status at the work place, especially so for an environment that is predominantly white (Berger, Fisek, & Norman, 1998). Our perceptions of individuals from different races may have an impact on our expectations of them and their behaviors. Some research has also suggested that non-whites are more likely to receive inconsistent 91

102 feedback (Crosby & Clayton, 1990). Therefore, it is possible that gender and race stereotypes may have an influence on the development of shared IPTs because they not only interfere in the way individuals perceive others, but also in the way that work related information is expected, organized, and recalled. Role clarity and perceived system knowledge. In addition to similarity in demography, the effects of two organizational variables on shared IPTs were also examined. Drawing on research in the newcomer socialization literature and other relevant SOA research, role clarity and perceived system knowledge were identified as possible predictors of shared IPTs. These two constructs were chosen primarily for their potential to influence an individual s ability to gather and organize information about work-related behaviors. The main study revealed that role clarity was marginally associated with shared IPTs, while perceived system knowledge was significantly associated with shared IPTs. These patterns of results indicate that for managers who exhibit higher levels of role clarity and PSK, these managers are more likely to organize their essential work behaviors in a manner that is similar with their supervisors organization of the manager s work behaviors. Once again, the positive results for role clarity support Campbell and Lee s cognitive approach towards performance appraisal. They proposed that organizational activities, such as role clarification and improved job descriptions, can minimize the informational and cognitive constraints that prevent self and other from achieving agreement. Therefore, it is quite clear that the authors perceive role clarity as a major determinant for facilitating the development of shared IPTs between self and other. 92

103 Similarly, perceived system knowledge refers to the extent to which a person is knowledgeable about the organization s performance appraisal system, the performance standards against which he/she will be evaluated, and the behaviors that are required to achieve the targeted performance level. As mentioned earlier, Williams and Levy (1992) found that the positive correlation between self and supervisor ratings was higher for individuals with higher levels of perceived system knowledge. The results from this study suggest that the moderating effect of PSK on the relationship between self and other ratings found in Williams and Levy s study may have been because individuals with high PSK were more likely to experience higher levels of similarities in IPTs with their managers. Theoretical and Practical Implications From a theoretical perspective, the focal study contributes to the literature on SOA in a number of ways. First, while there has been much research investigating the effects of predictor variables on agreement, a majority of them have not examined the psychological mechanisms by which agreement may be achieved. This study extends the findings of some of these earlier studies by demonstrating that similarities in implicit performance theories is one possible psychological mechanism for achieving agreement. Second, in Atwater and Yammarino s (1997) review of self-other agreement, the authors suggest a comprehensive list of the potential predictors of agreement. Prior and since this model was presented, many studies have been conducted that directly examined the effects of biographical, contextual, and individual and personality characteristics on SOA. However, there have been few that have specifically examined the effects of some of the cognitive processes as suggested by Atwater and Yammarino. The model in the 93

104 focal study draws heavily from the conceptual model put forth by Campbell and Lee (1988). Specifically, the study focuses on how a cognitive explanation may be used to better understand the process by which agreement is achieved. Third, this study incorporates a procedure commonly used in the training and team literature as a tool to capture and quantify similarities in implicit performance theories between managers and supervisors. Although Pathfinder is a tool designed specifically to map out an individual s organizational of information within a content domain, as far as is known, this is the first time it has been used to document the organization of work-related behaviors for an individual in the SOA literature. Thus this study is unique from other SOA studies that have examined implicit performance theories in that it uses Pathfinder in order to more accurately measure the study s focal construct that is one s implicit performance theory. From a practical perspective, the findings from the study may help organizations develop more targeted interventions to increase SOA. Achieving higher levels of agreement is important because it is associated with positive organizational outcomes (Atwater, Waldman, Ostroff, Robie, & Johnson, 2005; Ostroff, Atwater, & Feinberg, 2004). First, in order to overcome the cognitive and informational constraints that may impede an individual s role clarity, organizations should engage in activities such as role clarification and designing better job descriptions (Campbell & Lee, 1988). In addition, the newcomer socialization literature has demonstrated that behaviors like information seeking can have a positive influence on role clarity. Therefore, organizations should help develop a culture that encourages sharing performance-related information among individuals and their supervisors. 94

105 Second, perceived system knowledge may also be increased by developing a performance appraisal process that requires both the individual and his/her supervisor to meet and exchange information about what is expected of the ratee. In addition, supervisors should be trained and encouraged to provide feedback regularly so that the ratee is aware of how he/she is performing throughout the year. The increased interaction between individual and supervisor also has the added advantage of facilitating a more productive and less threatening performance appraisal session, resulting in increased acceptance of the appraisal outcome and minimizing negative reactions from negative discrepancies between self and supervisor appraisals (Williams & Levy, 1992). Third, the use of Pathfinder may also be adopted in organizations as a training tool to improve the performance appraisal process as well as increase agreement. For instance, Pathfinder could be used in performance appraisal training sessions to increase managers awareness of how the organization of work behaviors may influence the recall and evaluation of performance behaviors. Limitations and Future Research This study has provided some interesting and potentially useful information for future research. However, a number of limitations should be noted. First, as previously mentioned, the current study is the first known study that uses Pathfinder as a measurement tool to not only derive managers knowledge structures with respect to work-related behaviors, but also to determine similarities in IPTs between managers and supervisors. Future research should be conducted in order to better understand how other Pathfinder indices may or may not be related to agreement and other performance evaluations. For instance, the main Pathfinder metric for the current study was the 95

106 similarity index C, however, it is possible that other indices (such as r-data) might relate to other performance measures such as recall accuracy. A second limitation of the current study is the use of only managerial type behaviors to gauge performance. Managerial-related behaviors are just one of the many different KSAOs that are required by individuals at the workplace. It may be interesting to investigate if mental models can be derived for other competencies such as communication skills, situational judgment skills, and overall general performance. Investigating how mental models may be applied to other competencies will give researchers a greater understanding of the way in which different mental models may be related with each other. A third limitation is the use of a working population that consists of only managerial level positions. This level was chosen for the current study because the workrelated behaviors identified happen to be managerial-type behaviors and activities. For example behaviors that were used in the Pathfinder and performance measures involve dealing with subordinates, distribution of rewards between group members, and handling multiple projects and deadlines. Therefore, the results of the focal study may not be generalizable to lower level positions, such as individual contributors, and higher level positions, such as vice-presidents of departments. Consequently, future studies may wish to examine whether mental models of performance may be adequately captured for positions at various levels of the organizations. Finally, because of the cross-sectional nature of the study s design, one must be cautious when interpreting the directionality of the hypothesized relationships. For example, one may argue that shared IPTs may lead an individual to perceive himself as 96

107 having higher levels of perceived system knowledge and role clarity. As a result, it will be beneficial for future studies to adopt a longitudinal design and collect multiple measures of IPTs and other relevant constructs over time so as to more accurately ascertain the directionality of these relationships. 97

108 CHAPTER VI SUMMARY The main purpose of collecting performance ratings from a variety of sources is because agreement, as well as disagreement, can help provide valuable insights into how an individual is perceived at work. As a result, many organizations are beginning to develop and implement performance evaluations systems that incorporate ratings from not only the supervisor, but also from the individual himself, as well as other sources such as peers and subordinates. In addition to identifying areas where discrepancies in perceptions exist, involving the individual in the appraisal process may also result in other potentially beneficial outcomes. For instance, it not only gives the individual being rated a greater sense of control over his or her evaluation, but also increases the individual s perception of fairness in the eventual outcome and willingness to accept negative feedback (Farh et al. 1988; Greenberg, 1986). In addition to the increasing popularity of multi-source ratings in organizations, many investigators are also beginning to explore how different constructs that may either predict or are outcomes of self-other agreement (SOA). Meta-analyses conducted to determine the extent of agreement between sources generally find that the correlations between self and others to be lower than the correlations between others (Conway & Huffcutt, 1997; Harris & Schaubroek, 1988). Also, majority of the literature on SOA focus on a variety of variables that may predict agreement. Atwater and Yammarino 98

109 (1997) summarized the research on agreement and presented a comprehensive list of constructs that they believed to be critical predictors of SOA. The purpose of the focal study was to extend some of the findings in the SOA research by developing and testing a shared cognitive structure explanation of agreement between managers and their supervisors. While previous research have demonstrated how demographic, individual difference, cultural, and contextual variables may have an effect on agreement, few have adopted a cognitive approach in the explanation of SOA. A cognitive perspective of agreement is important because it not only adds to our knowledge in the SOA literature, it also identifies possible psychological mechanisms through with agreement is attained. This study adopted the framework of Campbell and Lee (1988) which emphasized the role of cognitions in predicting agreement between self and others. Specifically, the current study introduces a cognitive structure known as implicit performance theory (IPT). Implicit performance theories represent an individual s knowledge and organization of work related behaviors that are needed to perform effectively at one s job (Wegner & Vallacher, 1977). Therefore, it was hypothesized that when managers and supervisors have similar IPTs, they are more likely to experience SOA because both sources not only have a common understanding of the essential behaviors that are critical for effective performance, more importantly, their mental models for these behaviors are organized in a similar manner. Another purpose of this study was to examine how similarity in IPTs between managers and supervisors may develop. It was hypothesized that when managers and their respective supervisors are of similar age, race, and gender, the greater the likelihood 99

110 that both sources will exhibit shared IPTs. Furthermore, organizational constructs such as perceived system knowledge and role clarity were also hypothesized to be positively associated with shared IPTs. The sample for both the pilot and main study consisted of working individuals who currently hold a managerial level position. In addition, the supervisor of these managers was also asked to participate in the study. The pilot was conducted for two primary reasons. First, it was carried out to examine the feasibility of using a program known as Pathfinder as a tool to capture and measure similarity in IPTs. The second purpose of the pilot was to develop a shortened, yet equally valid, version of the IPT measure so as to minimize the time needed for participants to complete the study. Once it was established that the use of Pathfinder and the shortened IPT measure were viable, the main study was conducted to test all the hypotheses put forth in this study. The results of the main study were promising and generally supportive of most of the hypotheses. Similarity in IPTs between managers and supervisors was found to be associated with higher levels of SOA, over and above the effects of control variables. This suggests that when managers and supervisors have similar mental models regarding essential work behaviors, SOA is more likely to occur. In addition, similarity in age, perceived system knowledge, and role clarity were found to be significantly associated with shared IPTs. These results indicate that similarity in IPTs is more likely to be exhibited by managers whose age is similar with their supervisor s, or who experience higher levels of PSK and role clarity. Similarity in race and gender was not found to be significantly associated with shared IPTs. 100

111 In summary, this study contributes and extends the SOA literature by validating the theoretical propositions suggested by Atwater and Yammarino (1997) and Campbell and Lee (1988). Specifically, it empirically demonstrates the role of shared cognitive structures in predicting agreement. 101

112 REFERENCES Atwater, L. E. (1998). The advantages and pitfalls of self-assessment. In J. W. Smither (Ed.), Performance appraisal: State of the art in practice (pp ). San Francisco, CA: Jossey Bass. Atwater, L. E., Ostroff, C., Yammarino, F. J., & Fleenor, J. W. (1998). Self-other agreement: Does it really matter? Personnel Psychology, 51, Atwater, L., Waldman, D., Ostroff, C., Robie, C., & Johnson, K. M. (2005). Self-Other Agreement: Comparing its Relationship with Performance in the U.S. and Europe. International Journal of Selection and Assessment, 13(1), Atwater, L. E., & Yammarino, F. J. (1992). Does self-other agreement on leadership perceptions moderate the validity of leadership and performance predictions? Personnel Psychology, 45(1), Atwater, L. E., & Yammarino, F. J. (1997). Self-other rating agreement: A review and model. In G. R. Ferris (Ed.), Research in personnel and human resources management (Vol. 15, pp ). US: JAI Press. Baron, R., & Greenberg, J. (1990). Behavior in organizations (3 rd ed.). Boston: Allyn and Bacon. Bauer, T. N., & Green, S. G. (1998). Testing the combined effects of newcomer information seeking and manager behavior on socialization. Journal of Applied Psychology, 83, Bauer, T. N., Morrison, E. W., & Callister, R. R. (1998). Organizational socialization: A review and directions for future research. Research in personnel and human resources management, 16, Borman, W. C. (1974). The ratings of individuals in organizations: An alternative approach. Organizational Behavior and Human Decision Processes, 12, Bracken, D. W., Timmreck, C. W., & Church, A. H. (2001). The handbook of multisource feedback: The comprehensive resource for designing and implementing MSF processes. San Francisco, CA: Jossey-Bass. 102

113 Brutus, S., Fleenor, J. W., & McCauley, C. D. (1999). Demographic and personality predictors of congruence in multi-source ratings. Journal of Management Development, 18, Campbell, D. J. & Lee, C. (1988). Self-appraisals in performance evaluation: Development versus evaluation. Academy of Management Journal, 13, Dobbins, G. H., Cardy, R, L., & Truxillo, D. M. (1988). The effects of purpose of appraisal and individual differences in stereotypes of women on sex differences in performance ratings: A laboratory and field study. Journal of Applied Psychology, 73, Chau, S. L., & Levy, P.E. (2005, April). Disentangling the effect of race on supervisorsubordinate agreement. Poster session presented at the 20 th Annual Conference of the Society for Industrial and Organizational Psychology, Los Angeles, CA. Conway, J. M., & Huffcutt, A. I. (1997). Psychometric properties of multisource performance ratings: A meta-analysis of subordinate, supervisor, peer, and selfratings. Human Performance, 10, DeNisi, A. S., Robbins, T. L., & Summers, T. P. (1997). Organization, processing, and use of performance information: A cognitive role for appraisal instruments. Journal of Applied Social Psychology, 27, Diefendorff, J. M., & Silverman, S. B. (2000, April). Differences and similarities of 360 degree feedback across multiple countries. In S. B. Silverman s (Chair), Business as unusual? Are I-O psychology practices applicable across cultures? Paper presented at the 15 th Annual Society for Industrial and Organizational Psychology Conference, New Orleans, Louisiana. Dorfman, P.W., & Howell, J.P. (1988). Dimensions of national culture and effective leadership patterns: Hofstede revisited. In R.N. Farmer & E.G. McGound (Eds.), Advances in international comparative management (Vol. 3, pp ). Greenwich, CT: JAI Press. Duarte, N. T., Goodson, J. R., & Klich, N. R. (1994). Effects of dyadic quality and duration on performance appraisal. Academy of Management Journal, 37, Edwards, J. R. (1994). The study of congruence in organizational behavior research: Critique and a proposed alternative. Organizational Behavior and Human Decision Processes, 58, Edwards, J. R. (1995). Alternatives to difference scores as dependent variables in the study of congruence in organizational research. Organizational Behavior and Human Decision Processes, 64,

114 Edwards, B. D., Day, E. A., Arthur Jr. W., & Bell, S. T. (2006). Relationships among team ability composition, team mental models, and team performance. Journal of Applied Psychology, 91, Engle, E. M., & Lord, R. G. (1997). Implicit theories, self-schemas, and leader-member exchange. Academy of Management Journal, 40, Epitropaki, O., & Martin, R. (2004). Implicit Leadership Theories in Applied Settings: Factor Structure, Generalizability, and Stability Over Time. Journal of Applied Psychology, 89, Farh, J.-l., & Dobbins, G. H. (1989). Effects of comparative performance information on the accuracy of self-ratings and agreement between self- and supervisor ratings. Journal of Applied Psychology, 74, Farh, J.-l., Werbel, J. D. (1986). Effects of purpose of the appraisal and expectation of validation on self-appraisal leniency. Journal of Applied Psychology, 71, Farh, J.-l., Werbel, J. D., & Bedeian, A. G. (1988). An empirical investigation of selfappraisal-based performance evaluation. Personnel Psychology, 41(1), Feldman, J. M. (1981). Beyond attribution theory: Cognitive processes in performance appraisal. Journal of Applied Psychology, 66, Fiske, S, T., & Taylor, S. E. (Eds.) (1991). Social cognition (2nd ed.). New York: Mcgraw-Hill Book Company, Fleenor, J. W., McCauley, C. D., & Brutus, S. (1996). Self-other rating agreement and leader effectiveness. Leadership Quarterly, 7, Fletcher, C., & Baldry, C. (2000). A study of individual differences and self-awareness in the context of multi-source feedback. Journal of Occupational and Organizational Psychology, 73, Foti, R. J., & Lord, R. G. (1987). Prototypes and scripts: The effects of alternative methods of processing information on rating accuracy. Organizational Behavior and Human Decision Processes, 39, Furnham, A., & Stringfield, P. (1994). Congruence of self and subordinate ratings of managerial practices as a correlate of supervisor evaluation. Journal of Occupational and Organizational Psychology, 67, Gerstner, C. R., & Day, D. V. (1997). Meta-Analytic review of leader-member exchange theory: Correlates and construct issues. Journal of Applied Psychology, 82,

115 Glickman, A. S., Zimmer, S., Montero, R. C., Guerette, P. J., Campbell, W. J., Morgan, B., & Salas, E. (1987). The evolution of teamwork skills: An empirical assessment with implications for training. (Tech Report ). Orlando, FL: Office of Naval Research, Human Factors Division. Greenberg, J. (1986). Determinants of perceived fairness of performance evaluation. Journal of Applied Psychology, 71, Harris, M. M., & Schaubroeck, J. (1988). A meta-analysis of self-supervisor, self-peer, and peer-supervisor ratings. Personnel Psychology, 41(1), Hastie, R., & Park, B. (1986). The relationship between memory and judgment depends on whether the judgment task is memory-based or on-line. Psychological Review, 93, Hauenstein, N. M. A., (1998). Training raters to increase the accuracy of appraisals and the usefulness of feedback. In J. W. Smither (Ed.), Performance appraisal: State of the art in practice (pp ), San Francisco, CA: Jossey Bass. Heidemeier, H., & Moser, K. (2009). Self-other agreement in job performance ratings: A meta-analytic test of a process model. Journal of Applied Psychology, 94, Ilgen, D. R., & Feldman, J. M. (1983). Performance appraisal: A process focus. Research in Organizational Behavior, 5, Johnson, J. W., & Ferstl, K. L. (1999). The effects of interrater and self-other agreement on performance improvement following upward feedback. Personnel Psychology, 52, Korsgaard, M. A., Meglino, B. M., & Lester, S. W. (2004). The effect of other orientation on self-supervisor rating agreement. Journal of Organizational Behavior, 25, Klimoski, R., & Mohammed, S. (1994) Team mental model: Construct or metaphor. Journal of Management, 20, Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77, Landy, F., & Farr, J (1980). Performance rating. Psychological Bulletin, 87, LeBreton, J. M., Burgess, J. R. D., Kaiser, R. B., Atchley, E. K., & James, L. R. (2003). The restriction of variance hypothesis and interrater reliability and agreement: Are ratings from multiple sources really dissimilar? Organizational Research Methods, 6(1),

116 Levesque, L. L., Wilson, J. M., Wholey, D. R. (2001). Cognitive divergence and shared mental models in software development project teams. Journal of Organizational Behavior, 22, Lim, B. C., & Klein, K. J. (2006). Team mental models and team performance: A field study of the effects of team mental model similarity and accuracy. Journal of Organizational Behavior, 27, London, M., & Smither, J. W. (1995). Can multi-source feedback change perceptions of goal accomplishment, self-evaluations, and performance-related outcomes? Theory-based applications and directions for research. Personnel Psychology, 48, London, M., & Wohlers, A. J. (1991). Agreement between subordinate and self-ratings in upward feedback. Personnel Psychology, 44, Lord, R. G., & Maher, K. J. (1991). Leadership and information processing: Linking perceptions and performance. Cambridge, MA: Unwin Hyman. Mabe, P. A., & West, S. G. (1982). Validity of self-evaluation of ability: A review and meta-analysis. Journal of Applied Psychology, 67, Noonan & Sulsky (2001). Impact of frame-of-reference and behavioral observation training on alternative training effectiveness criteria in a Canadian military sample. Human Performance, 14, Ostroff, C., Atwater, L. E., & Feinberg, B. J. (2004). Understanding Self-Other Agreement: A Look at Rater and Ratee Characteristics, Context, and Outcomes. Personnel Psychology, 57, Parker, C. (1999). The impact of leaders implicit theories of employee participation on test of the Vroom-Yetton model. Journal of Social Behavior & Personality, 14, Rentsch, J. R. (1990). Climate and culture: Interaction and qualitative differences in organizational meanings. Journal of Applied Psychology, 75, Rentsch, J. R., & Hall, R. J. (1994). Members of great teams think alike: A model of team effectiveness and schema similarity among team members. Advances in Interdisciplinary Studies of Work Teams, 1, Rentsch, J. R., & Klimoski, R. J. (2001). Why do great minds think alike?: Antecedents of team member schema agreement. Journal of Organizational Behavior, 22,

117 Roberson, L., & Block, C. J. (2001). Racioethnicity and job performance: A review and critique of theoretical perspectives on the causes of group differences. Research in Organizational Behavior, 23, Roberson, L., Galvin, B. M., & Charles, A. C. (2008). When group identities matter: Bias in performance appraisal. The Academy of Management Annals, 1, Rosen, B., & Jerdee, T. H. (1988). Managing older workers careers. Research in Personnel and Human Resources Management, 3, Rosenberg, M. (1965). Society and the adolescent self-image. Princeton, NJ: Princeton University Press. Ryan, A. M., Brutus, S., Gregurus, G. J., & Hakel, M D. (2000). Receptivity to assessment-based feedback for management development. Journal of Management Development, 194, Salas, E., Montero, R. C., Glickman, A. S., & Morgan, B. B. (1988). Group development, teamwork skills and training. Paper presented at the American Psychological Association Meetings, Atlanta, Georgia. Sawyer, J. E. (1992). Goal and process clarity: Specification of multiple constructs of role ambiguity and a structural equation model of their antecedents and consequences. Journal of Applied Psychology, 77, Scandura, T. A., & Graen, G. B. (1984). Moderating effects of initial leader-member exchange status on the effects of a leadership intervention. Journal of Applied Psychology, 69, Schneider, B., & Rentsch, J. (1988). Managing climates and cultures: A futures perspective. In J Hage (Ed.), Futures of organizations (pp ). Lexington, MA: Lexington Books. Schrader, B. W., & Steiner, D. D. (1996). Common comparison standards: An approach to improving agreement between self and supervisory performance ratings. Journal of Applied Psychology, 81, Schvaneveldt, R. W. (1990). Proximities, networks, and schemata. In R. W. Schvaneveldt, (Ed), Pathfinder associative networks: Studies in knowledge organization. (pp ). Westport, CT: Ablex Publishing. Sosik, J. J. (2001). Self-other agreement on charismatic leadership: Relationships with work attitudes and managerial performance. Group & Organization Management, 26,

118 Sosik, J. J., & Megerian, L. E. (1999). Understanding leader emotional intelligence and performance: The role of self-other agreement on transformational leadership perceptions. Group & Organization Management, 24, Steelman, L. A., Levy, P. E., & Snell, A. F. (2004). The Feedback environment scale: Construct definition, measurement, and validation. Educational and Psychological Measurement, 64(1), Sulsky, L. M., & Day, D. V. (1992). Frame-of-reference training and cognitive categorization: An empirical investigation of rater memory issues. Journal of Applied Psychology, 77, Sundvik, L., & Lindeman, M. (1998). Performance rating accuracy: Convergence between supervisor assessment and sales productivity. International Journal of Selection and Assessment, 6(1), Taylor, S., & Brown, J. (1988). Illusion and well-being: A social psychological perspective on mental health. Psychological Bulletin, 103, Tsui, A. S., & Ohlott, P. (1988). Multiple assessment of managerial effectiveness: Interrater agreement and consensus in effectiveness models. Personnel Psychology, 41, Uggerslev, K. L., & Sulsky, L. M. (2008). Using frame-of-reference training to understand the implications of rater idiosyncrasy for rating accuracy. Journal of Applied Psychology, 93, Vollmer, F. (1986). Why do men have higher expectancy than women? Sex Roles, 14, Wanberg, C. R., & Kammeyer-Mueller, J. D. (2000). Predictors and outcomes of proactivity in the socialization process. Journal of Applied Psychology, 85, Ward, J. C., & Reingen, P. H. (1990). Sociocognitive analysis of group decision making among consumers. Journal of Consumer Research, 17, Watson, D., Hubbard, B., & Wiese, D. (2000). Self-other agreement in personality and affectivity: The role of acquaintanceship, trait visibility, and assumed similarity. Journal of Personality and Social Psychology, 78, Wayne, S. J., & Ferris, G. R. (1990) Influence tactics, affect, and exchange quality in supervisor-subordinate interactions: A laboratory experiment and field study. Journal of Applied Psychology, 75,

119 Wegner, D. M., & Vallacher, R. R. (1977) Implicit psychology: An introduction to social cognition. New York: Oxford University Press. Williams, J. R., & Levy, P. E. (1992). The effects of perceived system knowledge on the agreement between self-ratings and supervisor ratings. Personnel Psychology, 45, Woehr, D. J. (1994). Understanding frame-of-reference training: The impact of training on the recall of performance information. Journal of Applied Psychology, 79, Wohlers, A. J., Hall, M.-J., & London, M. (1993). Subordinates rating managers: Organizational and demographic correlates of self/subordinate agreement. Journal of Occupational and Organizational Psychology, 66, Wohlers, A. J., & London, M. (1989). Ratings of managerial characteristics: Evaluation difficulty, co-worker agreement, and self-awareness. Personnel Psychology, 42, Yammarino, F. J., & Atwater, L. E. (1997). Do managers see themselves as others see them? Implications of self-other rating agreement for human resources management. Organizational Dynamics, 25, Yammarino, F. J., & Waldman, D. A. (1993). Performance in relation to job skill importance: A consideration of rater source. Journal of Applied Psychology, 78, Yukl, G., & Lepsinger, R. (1990). Preliminary report on validation of the Managerial Practices Survey. In K. E. Clark & M. E. Clark (Eds.), Measures of leadership (pp ). West Orange, NJ: Leadership Library of America. Zegner, T. R., & Lawrence, B. S. (1989). Organizational demography: The differential effects of age and tenure distributions on technical communication. Academy of Management Journal, 32,

120 APPENDICES 110

121 APPENDIX A MANAGER PILOT STUDY MEASURES My job title is: I have been in my current position for yrs months I have been reporting to my current supervisor for yrs months Gender: Male / Female (Please circle) Age: yrs As you answer items throughout this survey, it is important that you put yourself in the mindset that you normally have while working at your job. To help you do that, we ask you to start by mentally picturing yourself at work. Tell us what kind of work you do and then please list and briefly describe three important tasks that you perform regularly for your job. Important tasks that I do are: 1) 2) 3) Next, read each of the behaviors listed on the next page carefully. The behaviors that are listed may or may not be relevant for your job. 111

122 Please rate the extent to which each behavior is important for you to perform effectively at your job. Circle the best response for each item, using the 7-point scale provided. A behavior that is not essential at all means that successfully doing the behavior will not influence your performance on your job. Conversely, a behavior that is absolutely essential means that successfully doing the behavior is absolutely required for effective job performance. Not Essential at All Helpful but not Essential Absolutely Essential Think outside the box when solving problems Work within existing organizational structure when solving problems Make decisions independently Consult others before making decision Pay attention to details Focus on broad picture and let others work on details Make a plan and stick to it Have a basic plan but be adaptive Ask for extension when you need more time Do the best you can within deadline and move on Help group members resolve conflict Let group members resolve conflict by themselves Be professional when dealing with subordinates Be friendly when dealing with subordinates Check on subordinates regularly Let subordinates approach you only when they face problems Delegate important work to subordinates Keep important work for yourself Reward individual performance Reward group performance

123 The next section looks long, but once you are used to the format, you should be able to understand and answer each option expediently! The section consists of items that include all possible pairings of the work behaviors that you just rated. Your task in this part of the survey involves judging the relatedness of pairs of work related behaviors. In making these types of judgments, there are several ways to think about the items being judged. For instance, two behaviors might be related because they share common features or because they frequently occur together. For the purpose of this section, relatedness is defined as the extent to which the pairs of work behaviors frequently occur together when performing effectively in your job. Use the 9-point scale provided at the top of the items (1=does not occur together at all to 9=occurs together frequently), and circle your answer. If you feel both of the paired behaviors do not frequently occur together when performing effectively in your job, rate the pair as a 1. Conversely, if you feel that the paired behaviors frequently occur together when performing effectively in your job, then rate them as does not occur occurs together together at all frequently Think outside the box when solving problems and Make decisions independently Think outside the box... and Pay attention to details Think outside the box... and Make a plan and stick to it Think outside the box... and Ask for extension when you need more time Think outside the box... and Work within existing organizational structure when solving problems Think outside the box... and Consult others before making decision Think outside the box... and Focus on broad picture and let others work on details Think outside the box... and Have a basic plan but be adaptive Think outside the box... and Do the best you can within deadline and move on Think outside the box... and Facilitate resolution among group members Think outside the box... and Be professional when dealing with subordinates Think outside the box... and Check on subordinates regularly Think outside the box... and Delegate important work to subordinates Think outside the box... and Reward individual performance Think outside the box... and Let group members resolve conflict by themselves Think outside the box... and Be friendly when dealing with subordinates Think outside the box... and Let subordinates approach you only when they face problems Think outside the box... and Keep important work for yourself Think outside the box... and Reward group performance

124 does not occur occurs together together at all frequently Let subordinates approach you only when they face problems and Keep important work for yourself Let subordinates approach only when face problems and Reward group performance Make decisions independently and Pay attention to details Make decisions independently and Make a plan and stick to it Make decisions independently and Ask for extension when you need more time Make decisions independently and Work within existing organizational structure when solving problems Make decisions independently and Consult others before making decision Make decisions independently and Focus on broad picture and let others work on details Make decisions independently and Have a basic plan but be adaptive Make decisions independently and Do the best you can within deadline and move on Make decisions independently and Facilitate resolution among group members Make decisions independently and Be professional when dealing with subordinates Make decisions independently and Check on subordinates regularly Make decisions independently and Delegate important work to subordinates Make decisions independently and Reward individual performance Make decisions independently and Let group members resolve conflict by themselves Make decisions independently and Be friendly when dealing with subordinates Make decisions independently and Let subordinates approach you only when they face problems Make decisions independently and Keep important work for yourself Make decisions independently and Reward group performance Delegate important work to subordinates and Reward individual performance Delegate important work and Let group members resolve conflict by themselves Delegate important work and Be friendly when dealing with subordinates Delegate important work and Let subordinates approach you only when they face problems Delegate important work and Keep important work for yourself Delegate important work and Reward group performance Attention to detail and Make a plan and stick to it Attention to detail and Ask for extension when you need more time Attention to detail and Work within existing organizational structure when solving problems 114

125 does not occur occurs together together at all frequently Attention to detail and Consult others before making decision Attention to detail and Focus on broad picture and let others work on details Attention to detail and Have a basic plan but be adaptive Attention to detail and Do the best you can within deadline and move on Attention to detail and Facilitate resolution among group members Attention to detail and Be professional when dealing with subordinates Attention to detail and Check on subordinates regularly Attention to detail and Delegate important work to subordinates Attention to detail and Reward individual performance Attention to detail and Let group members resolve conflict by themselves Attention to detail and Be friendly when dealing with subordinates Attention to detail and Let subordinates approach you only when they face problems Attention to detail and Keep important work for yourself Attention to detail and Reward group performance Be professional when dealing with subordinates and Check on subordinates regularly Be professional with subordinates and Delegate important work to subordinates Be professional with subordinates and Reward individual performance Be professional with subordinates and Let group members resolve conflict by themselves Be professional with subordinates and Be friendly when dealing with subordinates Be professional with subordinates and Let subordinates approach you only when they face problems Be professional with subordinates and Keep important work for yourself Be professional with subordinates and Reward group performance Make a plan and stick to it and Ask for extension when you need more time Plan/stick to it and Work within existing organizational structure when solving problems Plan/stick to it and Consult others before making decision Plan/stick to it and Focus on broad picture and let others work on details Plan/stick to it and Have a basic plan but be adaptive Plan/stick to it and Do the best you can within deadline and move on Plan/stick to it and Facilitate resolution among group members Plan/stick to it and Be professional when dealing with subordinates Plan/stick to it and Check on subordinates regularly

126 does not occur occurs together together at all frequently Plan/stick to it and Delegate important work to subordinates Plan/stick to it and Reward individual performance Plan/stick to it and Let group members resolve conflict by themselves Plan/stick to it and Be friendly when dealing with subordinates Plan/stick to it and Let subordinates approach you only when they face problems Plan/stick to it and Keep important work for yourself Plan/stick to it and Reward group performance Facilitate resolution among group members and Be professional when dealing with subordinates Facilitate resolution and Check on subordinates regularly Facilitate resolution and Delegate important work to subordinates Facilitate resolution and Reward individual performance Facilitate resolution and Let group members resolve conflict by themselves Facilitate resolution and Be friendly when dealing with subordinates Facilitate resolution and Let subordinates approach you only when they face problems Facilitate resolution and Keep important work for yourself Facilitate resolution and Reward group performance Ask for extension when you need more time and Work within existing organizational structure when solving problems Ask for extension on time and Consult others before making decision Ask for extension on time and Focus on broad picture and let others work on details Ask for extension on time and Have a basic plan but be adaptive Ask for extension on time and Do the best you can within deadline and move on Ask for extension on time and Facilitate resolution among group members Ask for extension on time and Be professional when dealing with subordinates Ask for extension on time and Check on subordinates regularly Ask for extension on time and Delegate important work to subordinates Ask for extension on time and Reward individual performance Ask for extension on time and Let group members resolve conflict by themselves Ask for extension on time and Be friendly when dealing with subordinates Ask for extension on time and Let subordinates approach you only when they face problems Ask for extension on time and Keep important work for yourself Ask for extension on time and Reward group performance Reward individual performance and Let group members resolve conflict by themselves

127 does not occur occurs together together at all frequently Reward individual performance and Be friendly when dealing with subordinates Reward individual performance and Let subordinates approach you only when they face problems Reward individual performance and Keep important work for yourself Reward individual performance and Reward group performance Let group members resolve conflict by themselves and Be friendly when dealing with subordinates Let group members resolve and Let subordinates approach you only when they face problems Let group members resolve and Keep important work for yourself Let group members resolve and Reward group performance Be friendly when dealing with subordinates and Let subordinates approach you only when they face problems Be friendly when dealing with subordinates and Keep important work for yourself Be friendly when dealing with subordinates and Reward group performance Work within existing organizational structure when solving problems and Consult others before making decision Work within existing structure and Focus on broad picture and let others work on details Work within existing structure and Have a basic plan but be adaptive Work within existing structure and Do the best you can within deadline and move on Work within existing structure and Facilitate resolution among group members Work within existing structure and Be professional when dealing with subordinates Work within existing structure and Check on subordinates regularly Work within existing structure and Delegate important work to subordinates Work within existing structure and Reward individual performance Work within existing structure and Let group members resolve conflict by themselves Work within existing structure and Be friendly when dealing with subordinates Work within existing structure and Let subordinates approach you only when they face problems Work within existing structure and Keep important work for yourself Work within existing structure and Reward group performance Consult others before making decision and Focus on broad picture and let others work on details Consult others and Have a basic plan but be adaptive Consult others and Do the best you can within deadline and move on Consult others and Facilitate resolution among group members Consult others and Be professional when dealing with subordinates Consult others and Check on subordinates regularly

128 does not occur occurs together together at all frequently Consult others and Delegate important work to subordinates Consult others and Reward individual performance Consult others and Let group members resolve conflict by themselves Consult others and Be friendly when dealing with subordinates Consult others and Let subordinates approach you only when they face problems Consult others and Keep important work for yourself Consult others and Reward group performance Keep important work for yourself and Reward group performance Check on subordinates regularly and Delegate important work to subordinates Check on subordinates regularly and Reward individual performance Check on subordinates regularly and Let group members resolve conflict by themselves Check on subordinates regularly and Be friendly when dealing with subordinates Check on subordinates regularly and Let subordinates approach you only when they face problems Check on subordinates regularly and Keep important work for yourself Check on subordinates regularly and Reward group performance Focus on broad picture and let others work on details and Have a basic plan but be adaptive Focus on broad picture and Do the best you can within deadline and move on Focus on broad picture and Facilitate resolution among group members Focus on broad picture and Be professional when dealing with subordinates Focus on broad picture and Check on subordinates regularly Focus on broad picture and Delegate important work to subordinates Focus on broad picture and Reward individual performance Focus on broad picture and Let group members resolve conflict by themselves Focus on broad picture and Be friendly when dealing with subordinates Focus on broad picture and Let subordinates approach you only when they face problems Focus on broad picture and Keep important work for yourself Focus on broad picture and Reward group performance Do the best you can within deadline and move on and Facilitate resolution among group members Do best and move on and Be professional when dealing with subordinates Do best and move on and Check on subordinates regularly Do best and move on and Delegate important work to subordinates

129 does not occur occurs together together at all frequently Do best and move on and Reward individual performance Do best and move on and Let group members resolve conflict by themselves Do best and move on and Be friendly when dealing with subordinates Do best and move on and Let subordinates approach you only when they face problems Do best and move on and Keep important work for yourself Do best and move on and Reward group performance Have a basic plan but be adaptive and Do the best you can within deadline and move on Plan but be adaptive and Facilitate resolution among group members Plan but be adaptive and Be professional when dealing with subordinates Plan but be adaptive and Check on subordinates regularly Plan but be adaptive and Delegate important work to subordinates Plan but be adaptive and Reward individual performance Plan but be adaptive and Let group members resolve conflict by themselves Plan but be adaptive and Be friendly when dealing with subordinates Plan but be adaptive and Let subordinates approach you only when they face problems Plan but be adaptive and Keep important work for yourself Plan but be adaptive and Reward group performance Please read the following work-related behaviors carefully and use the 5-point scale to rate how well you perform each of them. How well are you able to marginal satisfactory good very good Outstanding Think outside the box when solving problems Work within existing organizational structure when solving problems Make decisions independently Consult others before making decision Pay attention to details Focus on broad picture and let others work on details Make a plan and stick to it Have a basic plan but be adaptive Ask for extension when you need more time Do the best you can within deadline and move on

130 How well are you able to marginal satisfactory good very good Outstanding Help group members resolve conflict Let group members resolve conflict themselves Be professional when dealing with subordinates Be friendly when dealing with subordinates Check on subordinates regularly Let subordinates approach you only when they face problems Delegate important work to subordinates Keep important work for yourself Reward individual performance Reward group performance Adequately complete assigned duties Fulfill responsibilities specified in job description Perform task that are expected of me Meet formal performance requirements Engage in activities that will directly affect my performance evaluation Focus on aspects of the job that I am obligated to perform Perform essential duties Thank you for your participation We appreciate your efforts in providing information that will help us better understand people s views of their behaviors and performance at work. Your responses on this survey are appreciated! 120

131 APPENDIX B SUPERVISOR PILOT STUDY MEASURES As you answer items throughout this survey, it is important that you think about the primary tasks that your subordinate performs regularly. Your subordinate refers to the person that handed you this survey. Please list and briefly describe three important tasks that your subordinate performs regularly for his/her job. My job title is: Important tasks that my subordinate does are: 1) 2) 3) Next, read each of the behaviors listed on the next page carefully. The behaviors that are listed may or may not be relevant for your subordinate s job. 121

132 Please rate the extent to which each behavior is important for your subordinate to perform effectively at his/her job. Circle the best response for each item, using the 7-point scale provided. A behavior that is not essential at all means that successfully doing the behavior will not influence your subordinate s performance on his/her job. Conversely, a behavior that is absolutely essential means that successfully doing the behavior is absolutely required for effective job performance. Not Essential at All Helpful but not Essential Absolutely Essential Think outside the box when solving problems Work within existing organizational structure when solving problems Make decisions independently Consult others before making decision Pay attention to details Focus on broad picture and let others work on details Make a plan and stick to it Have a basic plan but be adaptive Ask for extension when he/she needs more time Do the best he/she can within deadline and move on Help group members resolve conflict Let group members resolve conflict by themselves Be professional when dealing with subordinates Be friendly when dealing with subordinates Check on subordinates regularly Let subordinates approach him/her only when they face problems Delegate important work to subordinates Keep important work for himself/herself Reward individual performance Reward group performance

133 The next section looks long, but once you are used to the format, you should be able to understand and answer each option expediently! The section consists of items that include all possible pairings of the work behaviors that you just rated. Your task in this part of the survey involves judging the relatedness of pairs of work related behaviors. In making these types of judgments, there are several ways to think about the items being judged. For instance, two concepts might be related because they share common features or because they frequently occur together. For the purpose of this section, relatedness is defined as the extent to which the pairs of work behaviors frequently occur together when your subordinate is performing effectively in his/her job. Use the 9-point scale provided at the top of the items (1=does not occur together at all to 9=occurs together frequently), and circle your answer. If you feel both of the paired behaviors do not frequently occur together when your subordinate is performing effectively in his/her job, rate the pair as a 1. Conversely, if you feel that the paired behaviors frequently occur together when your subordinate performs effectively in his/her job, then rate them as does not occur occurs together together at all frequently Think outside the box when solving problems and Make decisions independently Think outside the box... and Pay attention to details Think outside the box... and Make a plan and stick to it Think outside the box... and Ask for extension when he/she needs more time Think outside the box... and Work within existing organizational structure when solving problems Think outside the box... and Consult others before making decision Think outside the box... and Focus on broad picture and let others work on details Think outside the box... and Have a basic plan but be adaptive Think outside the box... and Do the best you can within deadline and move on Think outside the box... and Facilitate resolution among group members Think outside the box... and Be professional when dealing with subordinates Think outside the box... and Check on subordinates regularly Think outside the box... and Delegate important work to subordinates Think outside the box... and Reward individual performance Think outside the box... and Let group members resolve conflict by themselves Think outside the box... and Be friendly when dealing with subordinates Think outside the box... and Let subordinates approach you only when they face problems Think outside the box... and Keep important work for himself/herself Think outside the box... and Reward group performance Let subordinates approach him/her only when they face problems and Keep important work for himself/herself Let subordinates approach only when face problems and Reward group performance Make decisions independently and

134 does not occur occurs together together at all frequently Pay attention to details Make decisions independently and Make a plan and stick to it Make decisions independently and Ask for extension when he/she needs more time Make decisions independently and Work within existing organizational structure when solving problems Make decisions independently and Consult others before making decision Make decisions independently and Focus on broad picture and let others work on details Make decisions independently and Have a basic plan but be adaptive Make decisions independently and Do the best you can within deadline and move on Make decisions independently and Facilitate resolution among group members Make decisions independently and Be professional when dealing with subordinates Make decisions independently and Check on subordinates regularly Make decisions independently and Delegate important work to subordinates Make decisions independently and Reward individual performance Make decisions independently and Let group members resolve conflict by themselves Make decisions independently and Be friendly when dealing with subordinates Make decisions independently and Let subordinates approach him/her only when they face problems Make decisions independently and Keep important work for himself/herself Make decisions independently and Reward group performance Delegate important work to subordinates and Reward individual performance Delegate important work and Let group members resolve conflict by themselves Delegate important work and Be friendly when dealing with subordinates Delegate important work and Let subordinates approach him/her only when they face problems Delegate important work and Keep important work for himself/herself Delegate important work and Reward group performance Attention to detail and Make a plan and stick to it Attention to detail and Ask for extension when he/she needs more time Attention to detail and Work within existing organizational structure when solving problems Attention to detail and Consult others before making decision Attention to detail and Focus on broad picture and let others work on details Attention to detail and Have a basic plan but be adaptive Attention to detail and Do the best you can within deadline and move on

135 does not occur occurs together together at all frequently Attention to detail and Facilitate resolution among group members Attention to detail and Be professional when dealing with subordinates Attention to detail and Check on subordinates regularly Attention to detail and Delegate important work to subordinates Attention to detail and Reward individual performance Attention to detail and Let group members resolve conflict by themselves Attention to detail and Be friendly when dealing with subordinates Attention to detail and Let subordinates approach you only when they face problems Attention to detail and Keep important work for himself/herself Attention to detail and Reward group performance Be professional when dealing with subordinates and Check on subordinates regularly Be professional with subordinates and Delegate important work to subordinates Be professional with subordinates and Reward individual performance Be professional with subordinates and Let group members resolve conflict by themselves Be professional with subordinates and Be friendly when dealing with subordinates Be professional with subordinates and Let subordinates approach you only when they face problems Be professional with subordinates and Keep important work for himself/herself Be professional with subordinates and Reward group performance Make a plan and stick to it and Ask for extension when he/she needs more time Plan/stick to it and Work within existing organizational structure when solving problems Plan/stick to it and Consult others before making decision Plan/stick to it and Focus on broad picture and let others work on details Plan/stick to it and Have a basic plan but be adaptive Plan/stick to it and Do the best you can within deadline and move on Plan/stick to it and Facilitate resolution among group members Plan/stick to it and Be professional when dealing with subordinates Plan/stick to it and Check on subordinates regularly Plan/stick to it and Delegate important work to subordinates Plan/stick to it and Reward individual performance Plan/stick to it and Let group members resolve conflict by themselves Plan/stick to it and Be friendly when dealing with subordinates Plan/stick to it and Let subordinates approach you only when they face problems

136 does not occur occurs together together at all frequently Plan/stick to it and Keep important work for himself/herself Plan/stick to it and Reward group performance Facilitate resolution among group members and Be professional when dealing with subordinates Facilitate resolution and Check on subordinates regularly Facilitate resolution and Delegate important work to subordinates Facilitate resolution and Reward individual performance Facilitate resolution and Let group members resolve conflict by themselves Facilitate resolution and Be friendly when dealing with subordinates Facilitate resolution and Let subordinates approach you only when they face problems Facilitate resolution and Keep important work for himself/herself Facilitate resolution and Reward group performance Ask for extension when he/she needs more time and Work within existing organizational structure when solving problems Ask for extension on time and Consult others before making decision Ask for extension on time and Focus on broad picture and let others work on details Ask for extension on time and Have a basic plan but be adaptive Ask for extension on time and Do the best you can within deadline and move on Ask for extension on time and Facilitate resolution among group members Ask for extension on time and Be professional when dealing with subordinates Ask for extension on time and Check on subordinates regularly Ask for extension on time and Delegate important work to subordinates Ask for extension on time and Reward individual performance Ask for extension on time and Let group members resolve conflict by themselves Ask for extension on time and Be friendly when dealing with subordinates Ask for extension on time and Let subordinates approach him/her only when they face problems Ask for extension on time and Keep important work for himself/herself Ask for extension on time and Reward group performance Reward individual performance and Let group members resolve conflict by themselves Reward individual performance and Be friendly when dealing with subordinates Reward individual performance and Let subordinates approach you only when they face problems Reward individual performance and Keep important work for himself/herself Reward individual performance and Reward group performance

137 does not occur occurs together together at all frequently Let group members resolve conflict by themselves and Be friendly when dealing with subordinates Let group members resolve and Let subordinates approach you only when they face problems Let group members resolve and Keep important work for himself/herself Let group members resolve and Reward group performance Be friendly when dealing with subordinates and Let subordinates approach him/her only when they face problems Be friendly when dealing with subordinates and Keep important work for himself/herself Be friendly when dealing with subordinates and Reward group performance Work within existing organizational structure when solving problems and Consult others before making decision Work within existing structure and Focus on broad picture and let others work on details Work within existing structure and Have a basic plan but be adaptive Work within existing structure and Do the best you can within deadline and move on Work within existing structure and Facilitate resolution among group members Work within existing structure and Be professional when dealing with subordinates Work within existing structure and Check on subordinates regularly Work within existing structure and Delegate important work to subordinates Work within existing structure and Reward individual performance Work within existing structure and Let group members resolve conflict by themselves Work within existing structure and Be friendly when dealing with subordinates Work within existing structure and Let subordinates approach you only when they face problems Work within existing structure and Keep important work for himself/herself Work within existing structure and Reward group performance Consult others before making decision and Focus on broad picture and let others work on details Consult others and Have a basic plan but be adaptive Consult others and Do the best you can within deadline and move on Consult others and Facilitate resolution among group members Consult others and Be professional when dealing with subordinates Consult others and Check on subordinates regularly Consult others and Delegate important work to subordinates Consult others and Reward individual performance Consult others and Let group members resolve conflict by themselves Consult others and Be friendly when dealing with subordinates

138 does not occur occurs together together at all frequently Consult others and Let subordinates approach you only when they face problems Consult others and Keep important work for himself/herself Consult others and Reward group performance Keep important work for himself/herself and Reward group performance Check on subordinates regularly and Delegate important work to subordinates Check on subordinates regularly and Reward individual performance Check on subordinates regularly and Let group members resolve conflict by themselves Check on subordinates regularly and Be friendly when dealing with subordinates Check on subordinates regularly and Let subordinates approach you only when they face problems Check on subordinates regularly and Keep important work for himself/herself Check on subordinates regularly and Reward group performance Focus on broad picture and let others work on details and Have a basic plan but be adaptive Focus on broad picture and Do the best you can within deadline and move on Focus on broad picture and Facilitate resolution among group members Focus on broad picture and Be professional when dealing with subordinates Focus on broad picture and Check on subordinates regularly Focus on broad picture and Delegate important work to subordinates Focus on broad picture and Reward individual performance Focus on broad picture and Let group members resolve conflict by themselves Focus on broad picture and Be friendly when dealing with subordinates Focus on broad picture and Let subordinates approach him/her only when they face problems Focus on broad picture and Keep important work for himself/herself Focus on broad picture and Reward group performance Do the best you can within deadline and move on and Facilitate resolution among group members Do best and move on and Be professional when dealing with subordinates Do best and move on and Check on subordinates regularly Do best and move on and Delegate important work to subordinates Do best and move on and Reward individual performance Do best and move on and Let group members resolve conflict by themselves Do best and move on and Be friendly when dealing with subordinates Do best and move on and Let subordinates approach you only when they face problems

139 does not occur occurs together together at all frequently Do best and move on and Keep important work for himself/herself Do best and move on and Reward group performance Have a basic plan but be adaptive and Do the best you can within deadline and move on Plan but be adaptive and Facilitate resolution among group members Plan but be adaptive and Be professional when dealing with subordinates Plan but be adaptive and Check on subordinates regularly Plan but be adaptive and Delegate important work to subordinates Plan but be adaptive and Reward individual performance Plan but be adaptive and Let group members resolve conflict by themselves Plan but be adaptive and Be friendly when dealing with subordinates Plan but be adaptive and Let subordinates approach you only when they face problems Plan but be adaptive and Keep important work for himself/herself Plan but be adaptive and Reward group performance Please read the following work-related behaviors carefully and use the 5-point scale to rate how well your subordinate performs each of them. How well is your subordinate able to marginal satisfactory good very good Outstanding Think outside the box when solving problems Work within existing organizational structure when solving problems Make decisions independently Consult others before making decision Pay attention to details Focus on broad picture and let others work on details Make a plan and stick to it Have a basic plan but be adaptive Ask for extension when he/she needs more time Do the best he/she can within deadline and move on

140 How well is your subordinate able to Help group members resolve conflict Let group members resolve conflict by themselves Be professional when dealing with subordinates Be friendly when dealing with subordinates Check on subordinates regularly Let subordinates approach him/her only when they face problems Delegate important work to subordinates Keep important work for himself/herself Reward individual performance Reward group performance Adequately complete assigned duties Fulfill responsibilities specified in job description Perform task that are expected of him/her Meet formal performance requirements Engage in activities that will directly affect his/her performance evaluation Focus on aspects of the job that he/she is obligated to perform Perform essential duties marginal satisfactory good very good Outstanding Thank you for your participation We appreciate your efforts in providing information that will help us better understand people s views of their behaviors and performance at work. Your responses on this survey are appreciated! 130

141 APPENDIX C MANAGER MAIN STUDY MEASURES 1. Age 2. Sex 3. Race 4. Tenure with supervisor 5. Job tenure Demographics Collectivism (Dorfman & Howell, 1988) 5 pt Strongly Disagree to Strongly Agree 1. Group welfare is more important than individual rewards. 2. Group success is more important than individual successes. 3. Being accepted by the members of your work group is very important. 4. Employees should only pursue their goals after considering the welfare of the group. 5. Managers should encourage group loyalty even if individual goals suffer. 6. Individuals may be expected to give up their goals in order to benefit group successes. Self Esteem (Rosenberg, 1965) 5 pt Strongly Disagree to Strongly Agree 1. On the whole, I am satisfied with myself. 2. At times, I think I am no good at all. 3. I feel that I have a number of good qualities. 4. I am able to do things as well as most other people. 5. I feel I do not have much to be proud of. 6. I certainly feel useless at times. 7. I feel that I'm a person of worth, at least on an equal plane with others. 8. I wish I could have more respect for myself. 9. All in all, I am inclined to feel that I am a failure. 10. I take a positive attitude towards myself. Performance Appraisal Training (Swee, 2009) 5 pt Strongly Disagree to Strongly Agree 1. I have been provided with a comprehensive overview of how my company's performance appraisal system works in general. 2. My company has provided me with all of the necessary information for me to understand what my own personal performance appraisal rating is, how it is determined, and the implications for my performance. 3. I have attended a formal meeting or feedback session related to performance appraisal in my company within the past year. (Yes/No) 131

142 PANAS (Watson, D., Clark, L. A., & Tellegen, A., 1988) The next 20 items in this questionnaire describe the different types of feelings that may be experienced by a person. For each item, use the following 5-point rating scale to describe the extent to which you experience the feeling in general very slightly a little moderately quite a bit very much or not at all IN GENERAL, that is on the average, rate how you feel: interested distressed excited upset strong guilty scared hostile enthusiastic proud irritable alert ashamed inspired nervous determined attentive jittery active afraid LMX (Liden & Maslyn, 1998) 5 pt Strongly Disagree to Strongly Agree Affect 1. I like my supervisor very much as a person 2. My supervisor is the kind of person one would like to have as a friend 3. My supervisor is a lot of fun to work with Loyalty 4. My supervisor defends my work action to a superior, even without complete knowledge of the issue in question 5. My supervisor would come to my defense if I were attacked by others 6. My supervisor would defend me to others in the organization if I made an hones mistake Contribution 7. I do work for my supervisor that goes beyond what is specified in my job description 8. I am willing to apply extra efforts, beyond what is normally required, to meet my supervisor s work goals 9. I do not mind working my hardest for my supervisor Professional Respect 10. I am impressed with my supervisor s knowledge of his/her job 11. I respect my supervisor s knowledge of and competence on the job 12. I admire my supervisor s professional skills 132

143 Essential Ratings (Swee, 2009) 7 pt Not Necessary At All to Extremely Essential 1. Think outside the box when solving problems 2. Work within existing organizational structure when solving problems 3. Make decisions independently 4. Pay attention to detail 5. Focus on broad picture and let others work on details 6. Have a basic plan but be adaptive 7. Ask for extension when you need more time 8. Help group members resolve conflict 9. Let group members resolve conflict themselves 10. Be professional when dealing with subordinates 11. Be friendly when dealing with subordinates 12. Let subordinates approach you only when they face problems 13. Delegate important work to subordinates 14. Reward individual performance Role Clarity (Sawyer, 1992) 6 pt Very Uncertain to Very Certain Goal Clarity 1. My duties and responsibilities 2. The goals and objectives for my job 3. How my work related to the overall objectives of my work unit 4. The expected results of my work 5. What aspects of my work will lead to a positive evaluation Process Clarity 6. How to divide my time among the tasks required of my job 7. How to schedule my work day 8. How to determine the appropriate procedures for each work task 9. The procedures I use to do my job are correct and proper 10. Considering all your work tasks, how certain are you that you know the best ways to do these tasks PSK (Williams & Levy, 1992) 5 pt Strongly Disagree to Strongly Agree 1. I understand the performance appraisal system being used in my company. 2. I would benefit from additional training in the process of the appraisal system. 3. My supervisor and I concur on the criteria used in the performance appraisal system. 4. I understand the objectives of the present performance appraisal system. 5. I have a real understanding of how the performance appraisal rating was determined. 6. Procedures regarding the performance appraisal system are not generally understood by the employees. 7. An attempt should be made to increase employees understanding of the performance appraisal system. 8. I do not understand how my last performance appraisal rating was determined. 9. I know the criteria used by my employer to evaluate my performance. 133

144 10. I understand the standards of performance my employer expects. 11. My employer clearly communicates to me the objectives of the performance appraisal system. Performance (Swee, 2009) 5 pt Marginal to Outstanding 1. Think outside the box when solving problems 2. Work within existing organizational structure when solving problems 3. Make decisions independently 4. Pay attention to detail 5. Focus on broad picture and let others work on details 6. Have a basic plan but be adaptive 7. Ask for extension when you need more time 8. Help group members resolve conflict 9. Let group members resolve conflict themselves 10. Be professional when dealing with subordinates 11. Be friendly when dealing with subordinates 12. Let subordinates approach you only when they face problems 13. Delegate important work to subordinates 14. Reward individual performance Pathfinder (Swee, 2009) 9 pt Does not occur together at all to Occurs together frequently 1. Think outside the box when solving problems and Make decisions independently 2. Think outside the box... and Pay attention to details 3. Think outside the box... and Ask for extension when you need more time 4. Think outside the box... and Work within existing organizational structure when solving problems 5. Think outside the box... and Focus on broad picture and let others work on details 6. Think outside the box... and Have a basic plan but be adaptive 7. Think outside the box... and Facilitate resolution among group members 8. Think outside the box... and Be professional when dealing with subordinates 9. Think outside the box... and Delegate important work to subordinates 10. Think outside the box... and Reward individual performance 11. Think outside the box... and Let group members resolve conflict by themselves 12. Think outside the box... and Be friendly when dealing with subordinates 13. Think outside the box... and Let subordinates approach you only when they face problems 14. Make decisions independently and Pay attention to details 134

145 15. Make decisions independently and Ask for extension when you need more time 16. Make decisions independently and Work within existing organizational structure when solving problems 17. Make decisions independently and Focus on broad picture and let others work on details 18. Make decisions independently and Have a basic plan but be adaptive 19. Make decisions independently and Facilitate resolution among group members 20. Make decisions independently and Be professional when dealing with subordinates 21. Make decisions independently and Delegate important work to subordinates 22. Make decisions independently and Reward individual performance 23. Make decisions independently and Let group members resolve conflict by themselves 24. Make decisions independently and Be friendly when dealing with subordinates 25. Make decisions independently and Let subordinates approach you only when they face problems 26. Attention to detail and Ask for extension when you need more time 27. Attention to detail and Work within existing organizational structure when solving problems 28. Attention to detail and Focus on broad picture and let others work on details 29. Attention to detail and Have a basic plan but be adaptive 30. Attention to detail and Facilitate resolution among group members 31. Attention to detail and Be professional when dealing with subordinates 32. Attention to detail and Delegate important work to subordinates 33. Attention to detail and Reward individual performance 34. Attention to detail and Let group members resolve conflict by themselves 35. Attention to detail and Be friendly when dealing with subordinates 36. Attention to detail and Let subordinates approach you only when they face problems 37. Ask for extension when you need more time and Work within existing organizational structure when solving problems 38. Ask for extension on time and Focus on broad picture and let others work on details 39. Ask for extension on time and Have a basic plan but be adaptive 40. Ask for extension on time and Facilitate resolution among group members 41. Ask for extension on time and Be professional when dealing with subordinates 42. Ask for extension on time and Delegate important work to subordinates 43. Ask for extension on time and Reward individual performance 44. Ask for extension on time and Let group members resolve conflict by themselves 45. Ask for extension on time and Be friendly when dealing with subordinates 46. Ask for extension on time and Let subordinates approach you only when they face problems 135

146 47. Work within existing structure and Focus on broad picture and let others work on details 48. Work within existing structure and Have a basic plan but be adaptive 49. Work within existing structure and Facilitate resolution among group members 50. Work within existing structure and Be professional when dealing with subordinates 51. Work within existing structure and Delegate important work to subordinates 52. Work within existing structure and Reward individual performance 53. Work within existing structure and Let group members resolve conflict by themselves 54. Work within existing structure and Be friendly when dealing with subordinates 55. Work within existing structure and Let subordinates approach you only when they face problems 56. Focus on broad picture and let others work on details and Have a basic plan but be adaptive 57. Focus on broad picture and Facilitate resolution among group members 58. Focus on broad picture and Be professional when dealing with subordinates 59. Focus on broad picture and Delegate important work to subordinates 60. Focus on broad picture and Reward individual performance 61. Focus on broad picture and Let group members resolve conflict by themselves 62. Focus on broad picture and Be friendly when dealing with subordinates 63. Focus on broad picture and Let subordinates approach you only when they face problems 64. Plan but be adaptive and Facilitate resolution among group members 65. Plan but be adaptive and Be professional when dealing with subordinates 66. Plan but be adaptive and Delegate important work to subordinates 67. Plan but be adaptive and Reward individual performance 68. Plan but be adaptive and Let group members resolve conflict by themselves 69. Plan but be adaptive and Be friendly when dealing with subordinates 70. Plan but be adaptive and Let subordinates approach you only when they face problems 71. Facilitate resolution among group members and Be professional when dealing with subordinates 72. Facilitate resolution and Delegate important work to subordinates 73. Facilitate resolution and Reward individual performance 136

147 74. Facilitate resolution and Let group members resolve conflict by themselves 75. Facilitate resolution and Be friendly when dealing with subordinates 76. Facilitate resolution and Let subordinates approach you only when they face problems 77. Be professional with subordinates and Delegate important work to subordinates 78. Be professional with subordinates and Reward individual performance 79. Be professional with subordinates and Let group members resolve conflict by themselves 80. Be professional with subordinates and Be friendly when dealing with subordinates 81. Be professional with subordinates and Let subordinates approach you only when they face problems 82. Delegate important work to subordinates and Reward individual performance 83. Delegate important work and Let group members resolve conflict by themselves 84. Delegate important work and Be friendly when dealing with subordinates 85. Delegate important work and Let subordinates approach you only when they face problems 86. Reward individual performance and Let group members resolve conflict by themselves 87. Reward individual performance and Be friendly when dealing with subordinates 88. Reward individual performance and Let subordinates approach you only when they face problems 89. Let group members resolve conflict by themselves and Be friendly when dealing with subordinates 90. Let group members resolve and Let subordinates approach you only when they face problems 91. Be friendly when dealing with subordinates and Let subordinates approach you only when they face problems 137

148 APPENDIX D SUPERVISOR MAIN STUDY MEASURES 1. Age 2. Sex 3. Race 4. Job Tenure Demographics PANAS (Watson, D., Clark, L. A., & Tellegen, A., 1988) The next 20 items in this questionnaire describe the different types of feelings that may be experienced by a person. For each item, use the following 5-point rating scale to describe the extent to which you experience the feeling in general very slightly a little moderately quite a bit very much or not at all IN GENERAL, that is on the average, rate how you feel: interested distressed excited upset strong guilty scared hostile enthusiastic proud irritable alert ashamed inspired nervous determined attentive jittery active afraid 138

149 Essential Ratings (Swee, 2007) 5 pt Not Necessary At All to Extremely Essential 1. Think outside the box when solving problems 2. Work within existing organizational structure when solving problems 3. Make decisions independently 4. Pay attention to detail 5. Focus on broad picture and let others work on details 6. Have a basic plan but be adaptive 7. Ask for extension when you need more time 8. Help group members resolve conflict 9. Let group members resolve conflict themselves 10. Be professional when dealing with subordinates 11. Be friendly when dealing with subordinates 12. Let subordinates approach you only when they face problems 13. Delegate important work to subordinates 14. Reward individual performance Job Performance (Swee, 2007) 5 pt Marginal to Outstanding 1. Think outside the box when solving problems 2. Work within existing organizational structure when solving problems 3. Make decisions independently 4. Pay attention to detail 5. Focus on broad picture and let others work on details 6. Have a basic plan but be adaptive 7. Ask for extension when you need more time 8. Help group members resolve conflict 9. Let group members resolve conflict themselves 10. Be professional when dealing with subordinates 11. Be friendly when dealing with subordinates 12. Let subordinates approach you only when they face problems 13. Delegate important work to subordinates 14. Reward individual performance 139

150 Pathfinder (Swee, 2007) 9 pt Does not occur together at all to Occurs together frequently 1. Think outside the box when solving problems and Make decisions independently 2. Think outside the box... and Pay attention to details 3. Think outside the box... and Ask for extension when you need more time 4. Think outside the box... and Work within existing organizational structure when solving problems 5. Think outside the box... and Focus on broad picture and let others work on details 6. Think outside the box... and Have a basic plan but be adaptive 7. Think outside the box... and Facilitate resolution among group members 8. Think outside the box... and Be professional when dealing with subordinates 9. Think outside the box... and Delegate important work to subordinates 10. Think outside the box... and Reward individual performance 11. Think outside the box... and Let group members resolve conflict by themselves 12. Think outside the box... and Be friendly when dealing with subordinates 13. Think outside the box... and Let subordinates approach you only when they face problems 14. Make decisions independently and Pay attention to details 15. Make decisions independently and Ask for extension when you need more time 16. Make decisions independently and Work within existing organizational structure when solving problems 17. Make decisions independently and Focus on broad picture and let others work on details 18. Make decisions independently and Have a basic plan but be adaptive 19. Make decisions independently and Facilitate resolution among group members 20. Make decisions independently and Be professional when dealing with subordinates 21. Make decisions independently and Delegate important work to subordinates 22. Make decisions independently and Reward individual performance 23. Make decisions independently and Let group members resolve conflict by themselves 24. Make decisions independently and Be friendly when dealing with subordinates 25. Make decisions independently and Let subordinates approach you only when they face problems 26. Attention to detail and Ask for extension when you need more time 27. Attention to detail and Work within existing organizational structure when solving problems 28. Attention to detail and Focus on broad picture and let others work on details 140

151 29. Attention to detail and Have a basic plan but be adaptive 30. Attention to detail and Facilitate resolution among group members 31. Attention to detail and Be professional when dealing with subordinates 32. Attention to detail and Delegate important work to subordinates 33. Attention to detail and Reward individual performance 34. Attention to detail and Let group members resolve conflict by themselves 35. Attention to detail and Be friendly when dealing with subordinates 36. Attention to detail and Let subordinates approach you only when they face problems 37. Ask for extension when you need more time and Work within existing organizational structure when solving problems 38. Ask for extension on time and Focus on broad picture and let others work on details 39. Ask for extension on time and Have a basic plan but be adaptive 40. Ask for extension on time and Facilitate resolution among group members 41. Ask for extension on time and Be professional when dealing with subordinates 42. Ask for extension on time and Delegate important work to subordinates 43. Ask for extension on time and Reward individual performance 44. Ask for extension on time and Let group members resolve conflict by themselves 45. Ask for extension on time and Be friendly when dealing with subordinates 46. Ask for extension on time and Let subordinates approach you only when they face problems 47. Work within existing structure and Focus on broad picture and let others work on details 48. Work within existing structure and Have a basic plan but be adaptive 49. Work within existing structure and Facilitate resolution among group members 50. Work within existing structure and Be professional when dealing with subordinates 51. Work within existing structure and Delegate important work to subordinates 52. Work within existing structure and Reward individual performance 53. Work within existing structure and Let group members resolve conflict by themselves 54. Work within existing structure and Be friendly when dealing with subordinates 55. Work within existing structure and Let subordinates approach you only when they face problems 56. Focus on broad picture and let others work on details and Have a basic plan but be adaptive 57. Focus on broad picture and Facilitate resolution among group members 58. Focus on broad picture and Be professional when dealing with subordinates 141

152 59. Focus on broad picture and Delegate important work to subordinates 60. Focus on broad picture and Reward individual performance 61. Focus on broad picture and Let group members resolve conflict by themselves 62. Focus on broad picture and Be friendly when dealing with subordinates 63. Focus on broad picture and Let subordinates approach you only when they face problems 64. Plan but be adaptive and Facilitate resolution among group members 65. Plan but be adaptive and Be professional when dealing with subordinates 66. Plan but be adaptive and Delegate important work to subordinates 67. Plan but be adaptive and Reward individual performance 68. Plan but be adaptive and Let group members resolve conflict by themselves 69. Plan but be adaptive and Be friendly when dealing with subordinates 70. Plan but be adaptive and Let subordinates approach you only when they face problems 71. Facilitate resolution among group members and Be professional when dealing with subordinates 72. Facilitate resolution and Delegate important work to subordinates 73. Facilitate resolution and Reward individual performance 74. Facilitate resolution and Let group members resolve conflict by themselves 75. Facilitate resolution and Be friendly when dealing with subordinates 76. Facilitate resolution and Let subordinates approach you only when they face problems 77. Be professional with subordinates and Delegate important work to subordinates 78. Be professional with subordinates and Reward individual performance 79. Be professional with subordinates and Let group members resolve conflict by themselves 80. Be professional with subordinates and Be friendly when dealing with subordinates 81. Be professional with subordinates and Let subordinates approach you only when they face problems 82. Delegate important work to subordinates and Reward individual performance 83. Delegate important work and Let group members resolve conflict by themselves 84. Delegate important work and Be friendly when dealing with subordinates 85. Delegate important work and Let subordinates approach you only when they face problems 142

153 86. Reward individual performance and Let group members resolve conflict by themselves 87. Reward individual performance and Be friendly when dealing with subordinates 88. Reward individual performance and Let subordinates approach you only when they face problems 89. Let group members resolve conflict by themselves and Be friendly when dealing with subordinates 90. Let group members resolve and Let subordinates approach you only when they face problems 91. Be friendly when dealing with subordinates and Let subordinates approach you only when they face problems 143

154 APPENDIX E IRB APPROVAL (UNIVERSITY OF AKRON) 144