Political Ideology and the Implementation of Executive-Branch Reforms: The Contingent Impact of PART on Performance Management in Federal Agencies

Transcription

1 Political Ideology and the Implementation of Executive-Branch Reforms: The Contingent Impact of PART on Performance Management in Federal Agencies Stéphane Lavertu Assistant Professor John Glenn School of Public Affairs, The Ohio State University Donald Moynihan Professor of Public Affairs La Follette School of Public Affairs University of Wisconsin Madison May 23,

2 Abstract A central purpose of performance management reforms such as the Bush administration s Program Assessment Rating Tool (PART) is to promote the use of performance information in federal agencies. But as reforms become identified with a partisan agenda, the scope of their influence may decline. Using data from a survey of agency managers, we analyze the extent to which agencies ideological predispositions mediated the impact of PART reviews on managers use of performance information. The results indicate that the overall positive impact of managers involvement with PART reviews on information use may have been completely contingent on an agency being associated with a moderate or conservative ideology. Breaking down the analysis by management activity reveals that this ideological effect obtains for management activities that were more difficult for the Bush administration to observe, which is consistent with principal-agent theory. These and other results suggest that PART reviews were an ineffective mechanism for promoting performance management in liberal agencies when information asymmetries provided managers with discretion over information use. 2

3 Introduction Political executives such as presidents, governors, and mayors often come to office with the intent to reform the administrative agencies they oversee. Their reforms typically require some cooperation and effort from agency personnel. If executives have difficulty observing agency personnel s actions and capacities, reform outcomes, or the link between personnel s actions and reform outcomes, executives may face a principal-agent problem. When such information asymmetries exist, employees whose policy preferences differ from those of the executive may fail to devote necessary effort or may even sabotage the executive s policy initiatives. In this paper, we focus on the pursuit of a management reform in federal agencies in the face of such information asymmetries, and we examine whether divergence in the ideological preferences of a presidential administration and federal agencies has an impact on that reform s success. The 2001 President s Management Agenda represented the Bush administration s policy agenda for reforming management in federal agencies. It involved five management priorities, one of which was the integration of performance assessment and budgeting. The Office of Management and Budget (OMB) created and employed the Program Assessment Rating Tool (PART), a battery of diagnostic questions, to assist in pursuing that policy goal. The OMB used the tool to systematically evaluate the effectiveness of nearly all federal programs for the purpose of informing its budget formulation process and to improve and promote performance management in federal agencies. Indeed, OMB recommendations to agencies through PART reviews typically focused on promoting or altering performance measurement in federal agencies (GAO 2005). The administration s success in promoting performance management within 3

4 agencies depended on the extent to which agency managers used performance information in their decision-making. We propose that the extent to which managers involvement with PART reviews promoted performance information use depended on the match in ideological orientation between the Bush administration and federal agencies, as well as the information asymmetry between the administration and agency managers. Managers in agencies pursuing conservative policies should have had relatively little concern that PART reviews and recommendations undermined their programmatic priorities, perhaps by altering program goals or justifying cuts in their budgets. Managers working in agencies pursuing liberal missions, however, were relatively more likely to possess such concerns. Moreover, while the collection of performance information is observable by political principals to some extent, the use of performance information is an action that is more difficult to observe, and the ability to observe it varies across management activities. For example, oversight of performance information use for the purpose of modifying performance measures is relatively observable by political appointees and OMB PART reviewers, as opposed to the use of performance information for the purpose of solving problems. Thus, we propose that managers in relatively liberal agencies were less likely than those in conservative agencies to use performance information if they were involved in PART reviews, particularly for management activities that were more difficult for the administration to monitor. We explore these propositions empirically using respondent-level data from a 2007 Governmental Accountability Office (GAO) survey of mid- and upper-level agency managers. The GAO oversampled managers in some agencies, which enables us to examine the impact of managers involvement with PART reviews on their use of performance information across 29 4

5 agencies and a number of management activities. The survey included a variety of questions that asked managers if they used performance data for a range of purposes. Thus, we are able to examine whether managers performance management practices are related to the ideological leanings of the agencies in which they work. The results indicate that the overall positive impact of managers involvement with PART reviews on information use may have been completely contingent on an agency being associated with a moderate or conservative ideology. Breaking down the analysis by different managerial uses of performance data reveals that this ideological effect obtains for management activities that were more difficult for the administration to observe, which is consistent with principal-agent theory. Additionally, the analysis indicates that managers in liberal agencies who were involved with PART reviews agreed to a greater extent than those not involved that performance measurement problems impeded the collection and use of performance information, whereas there generally were no such differences in moderate and conservative agencies. These and other results suggest that PART reviews were an ineffective mechanism for promoting performance management in liberal agencies when information asymmetries provided managers with discretion over information use. There is growing agreement among both academics (Moynihan and Pandey 2010; Van de Walle and Van Dooren 2008) and practitioners (GAO 2008; OMB 2010 & 2011) that managerial performance information use is the key goal of performance management systems, but we know little about how political ideology influences use. This is an important issue because performance management initiatives often come from political executives and therefore may be, or may be viewed as, politically motivated. The OMB devoted a great deal of effort to creating what it presented as a nonpartisan, objective, and rigorous management tool (Dull 2006). Yet, there is some evidence that ideological or policy preferences affected PART scores and that these scores 5

6 were more likely to be used to set the budgets of relatively liberal agencies and programs (Gilmour and Lewis 2006a, 2006b, & 2006c). This paper is the first to provide systematic evidence that ideological factors also were associated with the Bush administration s success in promoting performance information use via PART reviews. The paper proceeds as follows. First, we provide background on the politics of presidential control and the politics of PART. Second, we describe the conditions under which agency managers involvement with PART reviews should lead to performance information use. Third, we describe the data and empirical methods. Fourth, we describe and discuss the results. Finally, we offer some concluding thoughts about the implications of this study s findings. The Politics of Presidential Control The politics of public management enjoys perennial scrutiny now that scholars have reevaluated the politics-administration dichotomy (e.g., Appleby 1949; Lynn, Heinrich and Hill 2000) and emphasize the role of politics in the implementation of policy (e.g., Pressman and Wildavsky 1973) and the design of the federal executive branch (e.g., Gormley 1989; McNollGast 1987; Moe 1989). Much of the empirical research focuses on how legislatures, particularly Congress (e.g., Epstein & O Halloran 1999; Lewis 2003), and political executives, particularly presidents (e.g., Lewis 2008; Rudalevige 2002), attempt to control executive-branch policymaking. A prominent narrative in the academic literature is that, as their political incentives have changed and the scope of the federal bureaucracy has expanded, presidents increasingly have sought to control bureaucratic behavior (Moe 1985). Modern accounts often begin with Nixon s politicization of federal agencies through appointments and his impoundment of agency budgets, or Reagan s imposition of various decision-making procedures 6

7 intended to limit agencies policymaking discretion and the promulgation of federal regulations altogether. David Lewis s (2008) research, for example, shows that presidents of both parties have employed their power of appointment to influence the policymaking of agencies whose programs are tied to ideological constituencies that differ from their own, and that this politicization had a negative impact on the performance of public programs. The need for and effectiveness of presidential efforts to control bureaucratic behavior is often assumed. That presidential control attempts take place suggests that there is indeed some value in them. Importantly, however, research increasingly focuses on what motivates bureaucrats, sometimes examining whether principal-agent models imported from economics appropriately characterize the determinants of bureaucratic behavior (e.g., Brehm and Gates 1997; Golden 2000). Some of this research indicates that bureaucrats infrequently shirk with low levels of effort and seldom seek to sabotage programs with which they disagree, but both popular and academic accounts frequently suggest otherwise. Rosemary O Leary (2006), for example, describes what she refers to as guerrilla government, in which career officials actively undermine the efforts of political appointees when they believe a president s policies are too extreme. Although not necessarily so explicit, bureaucratic resistance to presidential initiatives aimed at altering public management practices should be expected. Civil servants are acutely aware of the implications of such presidential actions for their programs. Reagan s management policies, for example, were perceived as an attempt to centralize authority and undermine liberal programs (Durant 1987; Tomkin 1998). Even reforms that are less overtly political are susceptible to bureaucratic indifference or even hostility if agencies are antagonistic to the president sponsoring them. A case in point is the negative reception for President Clinton s Reinventing Government reforms in the Department of Defense (Durant 2008). 7

8 The Politics of the Program Assessment Rating Tool Efforts to make public management more goal- and results-oriented, known as performance management reforms, have long enjoyed bipartisan support. Depending on how one defines performance management, one might identify a number of starting points: the recommendations of the Hoover Commission, which President Truman established; the Planning, Programming, and Budgeting System implemented in the Department of Defense under President Kennedy; the expansion of Management by Objectives under President Nixon; or the promotion of pay-for-performance by President Carter and the first President Bush. But the point of origin for the modern era of federal performance management is probably the Government Performance and Results Act (GPRA) of 1993 (Radin 2000). The act requires agencies to set long-term strategic goals and short-term annual goals, measure performance toward achieving those goals, and report on their progress via performance plans and reports to Congress and the Office of Management and Budget (OMB). GPRA enjoyed broad bipartisan support when it passed in 1993 and when it was amended in At various times since its enactment members of both parties have sought to use for their political purposes the performance information that GPRA generated. For example, Majority Leader Dick Armey employed GPRA results to publicize the poor performance of federal agencies (Dull 2006). But the Clinton administration also valued GPRA. For example, Vice President Gore launched an initiative that utilized GPRA goals in an attempt to illustrate the performance improvement that took place during the Clinton era (Moynihan 2003). GPRA is generally credited as having helped foster results-oriented management in federal agencies or, at least, as having helped set a foundation for the use of performance information in management activities (GAO 2004). 8

9 The Bush administration s President s Management Agenda built upon the bipartisan statutory framework for performance management that GPRA established (OMB 2001). By the time President Bush arrived in office, GPRA was not extensively used by either party and was seen by the Bush administration as a helpful but under-exploited tool for performance management (Dull 2006). Among other things, the President s Management Agenda called for the integration of performance assessment and budgeting. As a result, the OMB created and employed PART to systematically evaluate nearly all federal programs for the purpose of informing its budget formulation process and for promoting program performance. Specifically, it used the PART to grade federal programs on an ineffective-to-effective scale according to four different criteria (program purpose and design, strategic planning, program management, and program results/accountability) and weighted those scores to assign programs an overall score. Evaluating programs using the PART was a labor-intensive process conducted by OMB budget examiners in cooperation with program managers. PART reviews were conducted in waves from 2003 through 2008 until nearly all programs were evaluated. Some of the Bush administration s management practices were criticized as partisan and damaging to neutral competence (Pfiffner 2007); but the PART remained largely above such criticism, sometimes characterized as an inoffensive formal management agenda at odds with much of actual management practice (Moynihan and Roberts 2010). The PART was helped in this regard by the general perception that performance management is not an overtly political tool (Radin 2006). Perhaps more than any other scholar, Beryl Radin has sought to unmask the implicit normative assumptions of performance management, including the PART. While she argues that the PART clearly is not a value-free technical tool, and operates as part of a political process, she does not argue that it is a partisan tool (Radin 2000 & 2006). 9

10 The Bush administration took great care to establish the PART s credibility as a management tool (Dull 2006). It was pilot-tested and revised based on extensive feedback from a wide range of experts and stakeholders. A special team within OMB was created to make early versions of the PART more systematic. An outside advisory council of public management experts and a workshop from the National Academy of Public Administration were consulted. PART questions were dropped if they were perceived as lending themselves to an ideological interpretation. The OMB-trained budget examiners created a 92-page training manual and established a team to cross-check responses for consistency, all in the name of reducing subjectivity. The OMB also made all of PART assessments public and available on the internet in order to check against examiner biases a practice that demonstrates the confidence that the OMB had in the tool and the judgments it elicited. Mitch Daniels, the OMB director who created the PART, pushed staff to develop a non-partisan instrument (Moynihan 2008), and public presentations of the PART by OMB officials to stakeholders and agency personnel promoted it as a non-partisan tool. Why did the Bush administration devote such effort to developing a non-partisan tool and promoting it as such? It is because it wanted PART reviews to inform OMB, congressional, and agency decision-making (Dull 2006). PART reviews could serve the administration s policy priorities and enable it to enhance its control of the budget formulation process, an important aspect of federal policymaking. Poor PART reviews, for example, could be used as justification for cutting programs. But the tool s credibility had to be established if PART reviews were to affect the decision-making of legislators and agency managers. Dull (2006), for example, notes that management reform efforts seen as curbing neutral competence become tarnished and fail to gain the necessary credibility. He also states that while recent presidents have often bypassed 10

11 OMB, believing the organization to be unresponsive to the president s political needs, the Bush administration s PART seeks to discipline and employ OMB s policy competence to advance the administration s political agenda (Dull 2006, 207). Therefore, if seen as a credible tool for promoting performance management and enhancing the performance of federal programs, PART would also further the administration s goals in budgeting, policymaking, and implementation. Whatever the intent of the Bush administration, many actors outside of the White House were skeptical or questioned the usefulness of PART reviews. There is evidence that PART scores influenced executive branch budget formulation (Gilmour and Lewis 2006a) but that they did not significantly influence congressional budgetary decisions (Heinrich forthcoming; Frisco and Stalebrink 2008). Few congressional staff used PART information (GAO 2008, 19) and congressional Democrats considered PART a partisan tool. Efforts to institutionalize PART review via statute failed, reflecting partisan disagreement about its purpose and merit (Moynihan 2008). Indeed, the Obama administration characterized PART as an ideological tool and decided against continuing its implementation. PART and Performance Management: The Perspective of Agency Managers Agency managers had a number of reasons to view PART review as an important process and, therefore, to take steps to improve performance management of the programs with which they are involved. PART reviews were a presidential priority, and PART scores had an impact on OMB s budgetary decisions (Gilmour and Lewis 2006a & 2006b). The OMB also implemented mechanisms so that PART reviews would inform program management. For example, each PART assessment generated a series of recommendations for agency managers 11

12 that OMB officials could later review. The GAO concluded that agencies have clear incentives to take the PART seriously (GAO 2005b, 16). That many agency managers were directly involved in the PART review process also might have attenuated the type of suspicions that many members of Congress and congressional staff, who were likely less familiar with the PART review process, are thought to have held. 1 PART reviews were a product of a consultation between career counterparts at the OMB and the agency. When agency and OMB officials disagreed, it might have been chalked up to professional or interpersonal, rather than political, disagreement (Moynihan 2008). There were also reasons for agency officials to be wary of PART. Any government-wide reform will encounter claims that it lacks nuance and fails to appreciate the particular characteristics of a specific program (Radin 2006). PART, which was essentially a standardized questionnaire, was no exception. There is also the related issue of whether agency officials who enjoy an information advantage over OMB officials would accept PART evaluations as valid. The GAO (2008) asserted that agency managers lack of confidence in the credibility and usefulness of OMB s assessments, primarily due to a lack of programmatic expertise by PART reviewers, was a key impediment to OMB s leadership on performance management issues. There are at least three reasons why ideology might have had an impact on the extent to which involvement with PART reviews influenced managers use of performance information. First, ideological disagreement between the administration and agency managers could capture policy disagreement about appropriate management processes. For example, ideologically conservative agency managers may be more likely to agree with the notion of performance management and oversight via PART-like tools. (Although, the general bipartisan support for 1 Indeed, in an analysis of a GAO survey item that asks respondents to what extent PART needs to be changed, those involved with PART reviews expressed the need for less significant changes, regardless of the agency in which they worked. 12

13 performance management suggests that this may not be too significant a factor.) Second, managers who share a president s ideology may be more receptive to presidential initiatives. Third, ideology roughly captures substantive policy preferences. Agency managers with relatively liberal policy preferences, or those who manage programs traditionally supported by liberal political constituencies, may resist attempts by the administration to alter programs in substantively significant ways for example, through alterations in program goals and the use of performance measures that promote these altered goals. In addition to managers prior beliefs regarding the Bush administration s policy preferences, PART reviews themselves may have been interpreted as signaling the administration s policy priorities. Negative PART reviews, for example, might have signaled to managers that the administration was hostile to their programs. As we mention above, John B. Gilmour and David E. Lewis provide empirical evidence that political ideology played a role in the PART review process. Programs established under Democratic presidents received systematically lower PART scores than those created under Republican presidents (Gilmour and Lewis 2006c). Additionally, programs in traditionally Democratic agencies were the only ones that corresponded to OMB budgetary decisions, suggesting that programs more consistent with Republican ideology were insulated from PART scores during OMB s budget formulation (Gilmour and Lewis 2006b). Managers in liberal agencies had good reason to believe that PART reviews were not benefiting their programs, and perceived or real policy differences surely influenced managerial receptivity to changes in performance management promoted via the PART review process. There is substantial agreement that a primary goal of PART reviews was to make federal managers more results-oriented (GAO 2005; Dull 2006; Moynihan 2008). And managers had 13

14 significant incentives to take PART reviews seriously (Gilmour 2006). On the other hand, there is evidence that political considerations may have had an impact on PART scores and OMB s use of these scores in formulating the budget. Program managers in traditionally liberal agencies had reasons to perceive the review process as biased and invalid. The experience with PART gives rise to the following general proposition, which we elaborate in greater detail below. General Proposition: PART reviews are less likely to promote the use of performance information among managers whose policy preferences differ from those of the President. PART s Impact on Performance Information Use: A Principal-Agent Theory In the parlance of principal-agent theory, when agency managers possess an informational advantage over a presidential administration regarding the extent to which they engage in performance management practices, there exists a moral hazard if agency managers goals differ from those of the administration. (See Dixit [2002] for a survey of this theoretical literature, and Heinrich and Marschke [2010] for a review of applications to performance management.) In other words, from the perspective of administration officials, all managers conceivably could devote less-than-optimal effort to performance management, and managers whose policy preferences conflict with those of an administration have incentives to pursue policies they prefer. PART reviews were, to a significant extent, the Bush administration s costly attempt to reduce this information asymmetry and promote performance management by scrutinizing management practices. Thus, with regard to performance management activities on which PART reviews could shed light, such as modifying performance goals and measures, involvement with PART reviews might have promoted the use of performance information. 14

15 PART reviews may have created incentives for agency managers to focus on performance management if they did not already, but policy differences between the Bush administration and agency managers stood to influence the extent to which PART reviews promoted or hindered performance information use among agency managers. What the simple principal-agent perspective we offer above neglects is that the PART review process had an influence on the substance of program planning and performance measurement. PART reviews may have made, or may have been perceived as making, performance measures more or less useful to agency managers. As we mention in the previous section, managers who disagreed with the administration regarding programmatic goals or who discounted administration priorities (perhaps due to ideological differences) may have been less likely to perceive PARTinfluenced performance measures to be valid. Specifically, we offer the following hypothesis: H1: Managers who reported involvement with PART reviews perceived that performance measurement issues were impediments to performance information use to a greater extent than managers who did not report involvement with PART reviews if and only if their policy preferences differed from those of the administration. In turn, agency managers who disagreed with the performance goals and measures shaped by or provided through the PART review process should have been less likely to report using performance information, provided that information asymmetries permitted this shirking. Indeed, if this is the case, PART review could potentially have led to declines in the use of performance information. Specifically, we offer the following hypothesis: 15

16 H2: Managers who reported involvement with PART reviews used performance information to a lesser extent than managers who did not report involvement with PART reviews if and only if their policy preferences differed from those of the administration and management activities were sufficiently difficult for the administration to monitor. Empirical Approach Determining the extent to which policy preferences or political ideology affected PART reviews impact on performance information use requires measures of information use, PART implementation, and policy preferences. We employ data from a survey of agency managers to create measures of information use and PART implementation. Specifically, to create these measures we use survey items that ask agency managers to identify levels of information use, hindrances to information use, and involvement with PART reviews. To approximate differences in policy preferences or ideology, we employ a measure that categorizes agencies according to their ideological proclivities liberal, moderate, or conservative. Thus, the results we present below are from models that estimate the relationship between managers reported involvement with PART reviews, the ideological tradition or orientation of the agency in which managers work, and managers reported information use and perceptions regarding the impact of performance measurement problems on information use. Additionally, to test the robustness of our findings, we employ control variables based on a number of items that ask managers about other factors thought to influence information use. 16

17 Data The Government Accountability Office administered surveys in 1996, 2000, 2003, and 2007 to collect data on the implementation of performance management reforms in federal agencies. They administered the surveys to a random, nationwide sample of mid- and upperlevel federal employees in the agencies covered by the Chief Financial Officers Act of 1990, and, in 2007, they over-sampled managers from certain agencies to facilitate comparisons across 29 different agencies. The timing of and items in the 2007 survey also permit an assessment of PART s impact on performance management. Thus, our analysis employs the 2007 survey data. Tables 1 and 2 summarize the variables we employ (including descriptive statistics), so we do not describe most of them here. However, the manner in which we categorize measures of performance information use in each table of results, and the key measures indicating involvement with PART reviews, warrant further discussion. Aggregating all measures into a single index of use (as the GAO tends to do) is justifiable based on strong values of Cronbach s alpha. But common factor analysis indicates that it is appropriate to categorize measures of use in terms of subcategories of the broader index: program planning, problem solving, performance measurement, and employee management. The two measures we use to create indexes for each type of activity (see the bottom of Table 1) were those that most clearly loaded on the four underlying factors. That these categories capture theoretically distinct activities lends them some additional validity. [Insert Table 1 and Table 2 about here.] The indicator PART is based on an item inquiring about the extent to which respondents report being involved in PART reviews. PART is coded 1 if respondents report being involved 17

18 to a small extent or more, and zero otherwise. 2 The item used to create this measure asks about any involvement in preparing for, participating in, or responding to the results of any PART assessment. The structure of the variable is intended to reflect that the process of implementing PART affected some employees and had no impact on others. The implementation of PART was intended to create communities of agency actors who were involved with PART assessments (Gilmour 2006). Agency employees responsible for performance measurement, planning and evaluation, and budgeting processes are likely to have been directly involved in negotiating with OMB officials over PART scores. Program managers and staff whose programs were evaluated became involved in collecting agency information and responding to management recommendations offered through the PART review process. The review process led employees to invest time and effort toward performance management, enhanced awareness of appropriate performance management practices, and communicated the importance that OMB and the White House placed on performance management. As the mean of the variable in Table 2 indicates, 31 percent of managers surveyed were involved with PART reviews, indicating that relatively large numbers of federal employees were engaged in the process. The measures of agency ideology, Liberal and Conservative, are indicator variables created based on measures created by Joshua Clinton and David Lewis (2008). They employed measurement models that estimate agency ideology based on a survey of experts. The survey item read as follows: 2 We also estimated models using the continuous measure of use. The results are similar and tend to be more statistically significant. However, the dichotomous measure facilitates interpretation, so we report the results of models that employ that measure. 18

19 Please see below a list of United States government agencies that were in existence between I am interested to know which of these agencies have policy views due to law, practice, culture, or tradition that can be characterized as liberal or conservative. Please place a check mark ( ) in one of the boxes next to each agency slant Liberal, Neither Consistently, slant Conservative, Don t Know. (p5) The measure takes on a value of -1 (liberal), 0 (moderate), and 1 (conservative), which we used to create our dichotomous indicators for liberal and conservative agencies. (See Table 3 for the list of agencies by ideology.) While it would be ideal to have individual- or program-level measures of political ideology, the GAO did not collect this information. The agency-level scores should capture political or policy preferences embodied in these organizations as a result of their programs, employees, and other organizational factors. Research has shown the utility of these agency-level ideology scores in understanding PART scores and budget decisions (Gilmour and Lewis 2006a, Gilmour and Lewis 2006a), but not in terms of agency managers response to PART. The use of agency-level ideology scores also provides some reassurance that any findings that emerge are not the function of response bias or common-source methods bias. [Insert Table 3 about here.] Finally, it is important to note some things about the control variables we include in the models that test the robustness of our findings. First, all control variables were centered before model estimation to facilitate interpretation; however, the descriptions presented in Table 2 are for the variables before centering, to facilitate interpretation of the descriptive statistics. Second, we include all variables that account for manager characteristics that the GAO provided (there are two) and include variables that are thought to influence information use measures of 19

20 leadership commitment and decision-making authority, as well as measures of perceived oversight by political principals. There are a variety of additional items that inquire about potential hindrances to information collection and use in the survey, but we do not include them in the main statistical models because of the leading nature of the survey question ( to what extent, if at all, have the following factors hindered measuring performance or using the performance information? ), and because of high collinearity among many of these items. It is important to note, however, that the results are analogous if these variables are included as controls and that we analyze many of these factors explicitly in the analysis below (see Table 5). Statistical Methods We estimated a number of statistical models. Initially, for all of the analyses, we estimated hierarchical ordered probit models for each measure of use, as well as ordered probit models with errors clustered by agency. The data are ordered and non-normal and, thus, the ordered probit model is the most appropriate. However, the coefficients in ordered probit models are difficult to interpret and may be misleading for the models of performance information use, which include interactions between Liberal and PART and Conservative and PART. Additionally, the results of ordered probit models are nearly identical to those of hierarchical linear models, as well as OLS models with errors clustered by agency. Thus, to facilitate interpretation, for models of estimating information use we present the results of hierarchical linear models, which provide more information about the variability in PART s impact across agencies than OLS models with clustered errors. For the purpose of illustration, a basic hierarchical model that includes no level-1 controls and, for simplicity, includes only the 20

21 indicator for liberal agencies, is specified as follows, where i and j indicate the respondent and agency, respectively, and indicates a measure of use: Level 1: Level 2: Level 2: Plugging in level 2 equations and rearranging terms, one gets the following: The above model features four fixed effects coefficients (,,, and ) corresponding to the constant (i.e., the overall average level of information use), the impact of Liberal when respondents do not report involvement with PART reviews, the impact of PART involvement for a moderate or conservative agency (i.e., the impact for the omitted agency category), and the adjustment in the impact of PART involvement for a liberal agency, as opposed to a moderate or conservative agency. The above model also estimates random effects ( and ) corresponding to the unexplained variance of PART s impact across agencies and the unexplained variance in the dependent variable (i.e., information use) across agencies. All models were estimated using the xtmixed command in Stata 11.1 and using maximum likelihood estimation. The models also estimated the covariance between the impact of PART involvement across agencies and information use by agency. Finally, the results of ordered probit models with errors clustered by agency are reported for models that include no interaction variables. 21

22 Results The results presented in Table 4 are from hierarchical linear models estimating information use. The model in the first column employs an index that sums all of the measures of performance information use listed in the top portion of Table 1. The inclusion of interaction terms requires that the coefficients be interpreted carefully. The results indicate that, accounting only for agency random effects, involvement with PART reviews is positively associated with information use in ideologically moderate agencies (the coefficient for PART reveals its impact for managers in moderate agencies); agency ideology is unrelated to information use when a manager does not report PART involvement (the coefficients for Liberal and Conservative reveal ideology s lack of impact when a respondent does not report involvement); there is no difference between moderate and conservative agencies in terms of the impact of PART involvement (the coefficient for PART*Conservative does not reach statistical significance); and the impact of PART involvement is significantly lower in liberal agencies than it is in moderate agencies (the coefficient for PART*Liberal is negative and statistically significant). [Insert Table 4 about here.] There are two noteworthy aspects to these findings. First, the key difference is between the impact of PART in liberal and non-liberal agencies, and this is the relevant comparison for subsequent analyses. Second, when considering performance information use across all management activities, the impact of PART involvement is completely negated in liberal agencies. For example, in moderate agencies, the average level of use across all activities when no PART involvement is reported is captured by the constant, about 2.49 (between to a moderate extent and to a great extent ), and increases by the coefficient for PART (0.30) when PART involvement is reported, bringing the total to about In liberal agencies, the average 22

23 level of use by managers with no PART involvement is about 2.59 (though this is not statistically different from 2.49) and increases to 2.62 when PART involvement is reported ( ). Put differently, PART involvement does not significantly affect information use in liberal agencies, but it has a positive impact on information use in moderate and conservative agencies. The difference in the impact of PART across management activities also is noteworthy. Both interaction terms fail to reach statistical significance in the performance measurement model, indicating that the impact of PART is consistent across agencies of different ideologies when it comes to using performance information for refining program performance measures and setting new or revising existing performance goals. In other words, for those performance management activities that PART reviews monitor explicitly, agency ideology does not account for differences in the impact of PART reviews on performance management. On the other hand, the biggest disparity between liberal and non-liberal agencies in terms of the impact of PART involvement is in the problem solving model. Interestingly, this is the only activity for which there is a difference in the use of performance information by managers not involved with PART, in that managers in liberal agencies used performance information more (by 0.26) than moderate agencies. The impact of PART involvement on using performance information to identify and correct problems mitigates this disparity to some extent. Finally, another noteworthy finding is that, unlike other management activities, PART involvement has no impact on information use for employee recognition and rewarding in moderate and conservative agencies, and it may have had a negative overall impact on information use for this activity in liberal agencies (p=0.104 for a two-tailed test). The results presented in Table 5 indicate that these findings are robust to the inclusion of statistical controls. It is worth noting that, because the control variables were centered, the 23

24 constant refers to the average level of use for non-liberal agencies (and, in all but one model, liberal agencies as well) when no PART involvement is reported. The control variables are not the focus of our study, so we refrain from discussing their estimated coefficients here, except to say that leadership commitment, decision-making authority, and oversight by managers supervisors are strongly linked to performance information use across all activities. The findings regarding commitment and decision-making authority are consistent with previous studies (Dull 2009; Moynihan and Pandey 2010). [Insert Table 5 about here.] The results presented in Table 4 and Table 5 are consistent with our second hypothesis, which states that managers involved with PART and whose policy preferences differed from those of the administration used performance information to a lesser extent than those not involved if information asymmetries permitted it. The results indicate that the overall positive impact of managers involvement with PART reviews on information use is contingent on an agency being associated with a moderate or conservative ideology, but this disparity is driven by information use for management activities that are not easily monitored via the PART review process. When information use could be monitored effectively through the PART review process such as in management activities involving performance measurement differences in information use no longer correspond to agency ideology. The results clearly illustrate that the important distinction is between liberal and nonliberal agencies. The next analysis examines what it is about PART implementation in liberal agencies that accounts for the differences in use we have uncovered. The results presented in Table 6 are from ordered probit models (with errors clustered by agency) that estimate the extent to which agency managers perceive various factors as having hindered their collection or use of 24

25 performance information. Exact item wordings are provided in the table. The results reveal the estimated impact of managerial involvement with PART reviews on each factor. These PART effects were estimated separately for liberal and non-liberal agencies to facilitate comparisons. [Insert Table 6 about here.] A clear pattern emerges in the results presented in Table 6. Managers in liberal agencies involved with PART reviews agreed to a greater extent than those not involved with PART reviews that performance measurement problems hindered their collection and use of performance information. Strikingly, such effects are either non-existent or minimal for moderate and conservative agencies. It appears that difficulty determining meaningful measures, difficulty obtaining valid or reliable data, difficulty obtaining data in time to be useful, difficulty distinguishing between the results produced by the program and results caused by other factors, and existing information technology and/or systems not capable of providing needed data, were substantial impediments to performance management in liberal agencies only if managers reported involvement with PART reviews. On the other hand, there were few or no differences between managers in liberal agencies not involved with PART reviews and managers in moderate or conservative agencies (whether or not they were involved with PART reviews) in terms of the obstacles to performance management they perceived. These results suggest that the impact of ideology on information use occurred completely through the PART review mechanism that they are not attributable to inherent differences in agencies ability or willingness to use performance management practices. One potential competing interpretation is that managers involved with PART reviews were simply more knowledgeable about the performance information limitations their agencies faced. But it is not clear why such an effect would be limited only to managers in liberal agencies, and the selection 25

26 effect is perhaps more plausible in the the other direction (managers who spend their career creating performance measures are more likely to reject the claim that it is hard to measure program performance). The findings in the models of use also tend to undercut concerns about the PART variable: when information asymmetries are sufficiently great, reported levels of use in liberal agencies do not differ between managers who are involved and those not involved, and use practices do not differ by agency ideology when no PART involvement is reported. Further, managers in liberal agencies who were not involved with PART reviews did not report perceptions that differed in statistically significant ways from those of managers in moderate and conservative agencies. Indeed, the results reveal that all managers (whatever their agency s ideology and whether or not they were involved with PART reviews) had similar perceptions regarding the extent to which difficulty determining how to use performance information to improve the program affected information use. In other words, the differences in information use that we have uncovered are not attributable to differences in the ability to use performance information. Finally, fears of OMB micromanagement affected those involved with PART reviews similarly, whatever their agency s ideology. It is almost exclusively in liberal agencies that managers involved with PART reviews agree to a greater extent than managers not involved that difficulties collecting valid, reliable, and timely data inhibited the collection and use of performance information. These results are consistent with our first hypothesis that managers in liberal agencies who were involved with PART reviews are more likely to perceive that performance measures are an impediment to information use. Discussion 26

27 The results offer a coherent narrative about how political ideology and information asymmetry affect the ability of a political executive to further performance information use. The results indicate that the overall positive impact of managers involvement with PART reviews on information use may have been completely contingent on an agency being associated with a moderate or conservative ideology. Breaking down the analysis by management activity reveals that this ideological effect obtains for management activities that were more difficult for the administration to observe. Additionally, the analysis indicates that managers in liberal agencies who were involved with PART reviews agreed to a greater extent than those not involved that performance measurement problems impeded the collection and use of performance information, whereas there generally were no such differences in moderate and conservative agencies. These and other results are consistent with our hypotheses based on principal-agent theory. They provide evidence that managers in liberal agencies did not respond to the administration s approach to promoting performance management practices when PART reviews influenced performance measurement in ways they found problematic, and when information asymmetries between them and the administration permitted it. Our empirical analysis suggests that these results are driven by the PART-review monitoring mechanism, as opposed to differences in organizational or programmatic factors. Heinrich and Marschke (2010), employing a principal-agent perspective, argue that the impact of performance management reforms depends a good deal on implementation dynamics. In the course of examining these dynamics in the case of the PART, we contribute in important ways to a number of academic literatures. First, we contribute to the growing literature on the determinants of performance information use (e.g. Dull 2009; Moynihan and Pandey 2010) by examining the role of political ideology and how its impact varies across management activities. 27

28 A second contribution deals with the role of ideology in executive-branch policy implementation more generally. There is a growing literature that focuses on how political executives, particularly presidents, attempt to control executive-branch policy decisions using various mechanisms (e.g., see Moe 1985 and Lewis 2008) and the impact of control mechanisms on the behavior of agency personnel (e.g., see Brehm and Gates 1997; Golden 2000). And there is some research on the role of ideology in implementing performance management reforms (e.g., Durant 2008) and the use of the PART in particular (Gilmour and Lewis 2006a, 2006b, & 2006c). This paper is the first to provide systematic government-wide evidence that ideological factors influenced the Bush administration s success in promoting performance information use via PART reviews. Performance reforms often are promoted by political executives. Understanding how the ideological gap between executives and managers matters has real importance. While the Bush OMB promoted PART as a nonpartisan tool, ideology nonetheless impeded the administration s ability to promote performance management via PART reviews. Agency-OMB interactions related to PART reviews often focused on the issue of what constituted acceptable goals and measures (Frederickson and Frederickson 2006; Gilmour 2008). While PART assessments provided a mechanism through which OMB could make various programmatic and management recommendations, the majority of PART recommendations had to do with performance assessments, such as developing outcome measures and/or goals, and improving data collection (GAO 2005, 22). This was to be expected, since OMB examiners typically did not possess in-depth management programmatic knowledge and thus were likely to recommend their principals policy priorities and monitor compliance with these recommendations. They were less able, however, to monitor other uses of performance information by agency managers, such as the use of performance information for problem 28