Surveys. A long tradition

Transcription

1 Surveys Josh Lerner Empirical Methods in Corporate Finance A long tradition Critical to much of social science research, e.g.: Human resource management. Psychology. Marketing. Technology diffusion. Political science. But relatively little use in corporate finance. 1

2 The birth of political surveying: The Straw Poll Many early censuses (in U.S. dating to 1790; elsewhere, far earlier), but surveys far later: The straw poll : first conducted by the Harrisburg Pennsylvanian in Mail out ballots and tally returned votes. Also used as a marketing ploy. Reliability of straw polls Typically, no effort to control for respondents. Depend on people to return mail in in cards. Pierre du Pont straw poll concerning Prohibition was only returned by people who favored repealing it. People polled can be unrepresentative (haphazard sample) 1936 Literary Digest poll predicted Alf Landon (57%) would be elected president over FDR (43%). Same issues surface in Internet polling. 2

3 The birth of the modern survey Gallup in 1936 predicts FDR wins (55.7% even though h FDR won 60.8%). Used scientific quota sampling of only about 1,200 people compared to the 2 million in the Literary Digest straw poll. Illustrated power of model Though, of course, possibility of error, should be something that can quantified. Standard Error = pˆ ( 1 pˆ ) N Simple Random Sampling everyone has an equal chance of being selected Quota Samples use the census to find a certain number of people in different groups to force sample to be representative of population: This method failed in 1948: Multi stage Cluster Sampling a combination of the two approaches Types of sampling 3

4 The problem of non response One example: 56% of people contacted t responded d to the 2000 National Election Survey. 5% of households don t have phones Solution is to weight the surveys to match the census, but Census is not entirely accurate People who choose not to respond may hold different opinions than those that do, even within the same demographic category Bias Bias refers to anything that causes the estimate from the survey to differ from the true population Sampling: how representative the poll is Sampling error Non response error Survey design: Question wording Item ordering Question ordering Interviewer and response: Social desirability 4

5 Question wording CNN/USA Today/Gallup Poll. Dec. 9-11, Nationwide. N=503, MoE ± 5 "Do you think legal immigrants mostly help the economy by providing low cost labor, or mostly hurt the economy by driving wages down for many Americans?" Options rotated. Mostly Help 42% Mostly Hurt 52% Neither (vol.) 3% Both (vol.) 2% Unsure 1% Time Poll conducted by Schulman, Ronca & Bucuvalas (SRBI) Public Affairs. Nov. 29- Dec. 1, N=1,004 adults nationwide. MoE ± 3. "Overall, do you think illegal immigrants hurt or help the U.S. economy?" Options rotated Help 26% Hurt 64% Depends (vol.) 5% Unsure 5% Item ordering FOX News/Opinion Dynamics Poll. Latest: Aug. 6-7, N=900 registered voters nationwide. MoE ± 3. "...If the congressional election were held today, would you vote for the Republican candidate in your district or the Democratic candidate in your district?" If undecided: "Well, if you had to vote, which way would you lean? Rep: 39% Dem: 36% Other/Not Sure: 25% CNN/USA Today/Gallup Poll. Latest: Aug , N=689 registered voters nationwide. MoE ± 4 "If the elections for Congress were being held today, which party's candidate would you vote for in your congressional district: [rotate] the Democratic Party's candidate or the Republican Party's candidate?" If undecided: "As of today, do you lean more toward [rotate] the Democratic Party's candidate or the Republican Party's candidate? Rep: 42% Dem: 50% Other(vol.)/Undecided: 8% 5

6 Framing 1980 The US U.S. should let Communist newspaper reporters from other countries come here and send back to their papers the news as they see it 55% Yes When preceded by a question about U.S. reporters sent to Communist countries 75% Yes Problem: We often don t know all of the questions asked and in what order Other problems with polls Do they capture true feelings on sensitive issues such as race? Efforts to combine with coin flip. Interviewer bias Can control partly with fixed effects. Multiple stimuli versus balanced arguments Non attitudes and response acquiesce The surprise poll draws attention, but is it representative of the population? 6

7 Well defined best practices Advisory and technical committees. Development of multiple questions to detect gaming gaming. Pre testing. Translation and back translation. Sampling frames. Stratification. Fieldstaff training. Field staff incentivization and supervision. Post survey data processing: i.e., interviewer fixed effects. Well defined best practices (2) Documentation: Objectives Survey organization actors in planning and implementation Target population and sample design Sample design Questionnaires Other survey tools Data collection fieldwork Data processing Analysis 7

8 Skepticism in finance George Stigler and profit maximization survey. Cliff Smith and the chef story. These prejudices are deeply engrained in the field! Barriers to dissemination in finance Focus on corporations as subject of study: Limited interest in surveys. Will the right person fill out? GM R&D survey story. Interest in observable behavior. Distrust of reported motivations. Unfamiliarity with methodology. 8

9 Views of Financial Economists on the Equity Premium Welch JB, 2000 ke=rf+β[e (rm rf)] A practical problem Crucial lin valuation problems. But few systematic treatments. Most practitioners rely on rules of thumb, academics don t discuss. Many methodologies have relied on Historical average premia (but arithmetic? Geometric? Time period? Country mix?) Dividend yields? (But nonsensical answers) Theory? (But non robust and typically very low) 9

10 Survey design Survey 1: Web posting. Scattered mail out: 11 schools, UCLA colleagues, associate editors. Ambiguous questions. No reporting of response rates. Survey 2: Web posting only. Clearer question. Prompting with Ibbotson number. Year delay, with rising equity markets. Analysis Aggregate two sets of response: Need to adjust first responses for failure to correct for arithmetic/geometric confusion. Eliminate one high outlier: No clear justification. Figure 1 and Table 2. 10

11 Looking at consensus Ask economists to identify consensus pick: Higher than actual average. Lower standard deviation. Not surprisingly, belief in higher premium also translates into higher consensus estimate. Tables 4 and 5. Concerns Methodology is disappointly crude: Poor question wording. Lack of re testing. Lack of real sampling frame. Non reporting of key statistics. Suggests why surveying has bad name in Suggests why surveying has bad name in corporate finance. 11

12 The Theory and Practice of Corporate Finance: Evidence from the Field Graham and Harvey JFE, 2001 A different approach Rather than looking at a single firm, survey large number of firms. Ask about a wide variety of financial strategies. See as complement to large sample, clinical studies. 12

13 Objective of study Seek to understand firm s choices regarding: Capital budgeting. Cost of capital. Capital structure. Methodology Sent surveys to two groups: Selected ~1/4 of Financial Executive Institute members (4400): Diverse array of firms. All Fortune 500 CFOs. 9% response rate: 392 responses in all. Not great, but consistent with earlier surveys. 13

14 Addressing non respondent bias Comparing on time and late respondents: Late respondents should provide clue as to features of non respondents. Look at industry mix, public status vs. FEI population as a whole. Compare financial policy offirms firms to random samples of Compustat firms as a whole: Sample based on firm size. Capital budgeting NPV and IRR by far most common. NPV more common in larger firms. Similarly for public, dividend paying firms. Payback is as common among small firms, those with old CEOs (especially without MBAs). 14

15 Cost of capital CAPM by far most common way to compute cost of capital: Especially among large, public firms. Little use of Fama French, other factors. Few distinctions between firm, project risk. Capital structure Not surprisingly taxes, credit ratings matter. Only 44% of firms have real target capital structure. Little evidence that common hypotheses are influential: Signaling product market concerns free cash Signaling, product market concerns, free cash flow, etc. 15

16 Concerns Could do more to address selection biases? Non respondent surveys? Could do more to assess consistency of responses? Multiple questions? Final thoughts Typically, empirical research looks at actual bh behavior. In depth clinical studies ideally allow one to get behind simple responses. To what extent to do survey responses reflect reality, or just beliefs (or what would like to believe)? 16