NYSERDA CFL Expansion Plan Evaluation 2009 Baseline RDD Survey

Transcription

1 9/29/2009 NYSERDA CFL Expansin Plan Evaluatin 2009 Baseline RDD Survey The purpse f the NYSERDA CFL Expansin Plan Evaluatin is t furnish infrmatin n the effectiveness f the CFL Expansin Plan. The purpse f the 2007 Baseline RDD Survey is t furnish benchmarks fr New Yrk State, New Yrk City, and cmparisn areas Washingtn, DC, Hustn, TX, and Ohi prir t the start f the CFL Expansin Plan. Sample Target Ppulatin The target ppulatins fr the survey are telephne husehlds in New Yrk State (excepting New Yrk City and Lng Island), New Yrk City, Washingtn, DC, Hustn, TX, and Ohi. Sample Frame and Selectin The RDD sample frames fr each f the five lcatins were prvided by Survey Sampling Internatinal (SSI). The New Yrk State frame included all cunties f New Yrk excepting thse n Lng Island and in New Yrk City. The New Yrk City frame included the five brughs f the City. The Washingtn, DC frame included nly the District f Clumbia. The Hustn frame included all f Harris Cunty. The Ohi frame included the entire state excluding area cdes 513 and 283. These area cdes cver the Duke Pwer service territry where a CFL prgram is being implemented. The randm digit sampling prcedure prvides representatin f bth listed and unlisted (including ntyet-listed) numbers by randm generatin f the last tw digits f telephne numbers selected n the basis f their area cde, telephne exchange (the first three digits f a seven digit number), and bank number (the fur and fifth digits). Telephne exchanges are selected with prbabilities prprtinate t their size by cunty and by exchange within cunty. Only wrking banks f telephne numbers were selected. A wrking bank is a grup f 100 cntiguus telephne numbers that cntain at least ne wrking residential listing. Using wrking banks imprves the efficiency f the sample in lcating husehlds and it prvides the pprtunity fr husehlds with numbers in a new exchange t be included in the sample. The sample was released fr interviewing in replicates, which are randm subsamples f the larger sample. Using replicates cntrls the release f sample t ensure that all released sample numbers receive the full call prcedures and t maintain apprpriate reginal distributin f called numbers. Data Cllectin Overview f Data Cllectin Prcedures The RDD Survey was administered as a telephne interview with a husehld. Interviewers frm Braun Research cnducted the interviews using a cmputer-assisted telephne interview (CATI) survey instrument. 1

2 Survey Instrument The survey instrument gathered infrmatin n respndents awareness f CFLs, their usage f CFLs in and arund their hmes, and their CFL buying habits. The instrument als addressed ther husehld lighting devices, namely light emitting dides (LEDs) and standard incandescent bulbs. The survey was pretested with a small number f respndents t evaluate the degree t which questins were clear and understandable and t check the lgic and skip patters f the CATI survey instrument. A secnd Spanishlanguage instrument was als created and administered in New Yrk City and Hustn, TX. Survey Administratin Interviewers had a minimum effrt requirement f at least tw daytime, evening, and weekend phne calls per sample telephne number. If the interviewer reached an answering machine at the number dialed, he/she left a message n the first cntact and n every third cntact thereafter. If the interviewer reached a husehld in New Yrk City r Hustn, TX that nly spke Spanish, the Spanishlanguage interview was cnducted. Fielding began abut tw weeks prir t the hliday seasn, and was suspended fr several days including Christmas Eve, Christmas Day, Dec. 26 th, and New Year s Day. The New Yrk State study was fielded fr 37 days, New Yrk City fr 30 days, and the Washingtn, Hustn, and Ohi studies were fielded fr 25 days. Survey administratin averaged 16 minutes per cmpleted interview fr all five studies. Tables 1a t 1e shw the final dispsitin f the samples. Table 1a Survey Sample Dispsitin: New Yrk State % Partial 56 1% Refused % Nt d % Nt Quta Met 0 0% Excluded Infrmatin nt available fr cntact 0 0% Unusable number % Nt Eligible 2 ~0% Ttal % 2

3 Table 1b Survey Sample Dispsitin: New Yrk City % Partial 33 2% Refused % Nt d % Nt Quta Met 0 0% Excluded Infrmatin nt available fr cntact 0 0% Unusable number % Nt Eligible 0 0% Ttal % Table 1c Survey Sample Dispsitin: Washingtn, DC % Partial 41 2% Refused % Nt d 174 7% Nt Quta Met 0 0% Excluded Infrmatin nt available fr cntact 0 0% Unusable number % Nt Eligible 0 0% Ttal % 3

4 Table 1d Survey Sample Dispsitin: Hustn, TX % Partial 43 2% Refused % Nt d 212 9% Nt Quta Met 0 0% Excluded Infrmatin nt available fr cntact 0 0% Unusable number % Nt Eligible 3 ~0% Ttal % Table 1e Survey Sample Dispsitin: Ohi % Partial 31 1% Refused % Nt d % Nt Quta Met 0 0% Excluded Infrmatin nt available fr cntact 0 0% Unusable number % Nt Eligible 0 0% Ttal % 4

5 Table 2 shws the number f sampled cases, the number f cmpleted interviews, and the respnse rate by sample stratum. Table 2 Nnrespnsive Referred Survey Respnse Rate New Yrk State New Yrk City Washingtn, DC Hustn, TX Ohi Number f Interviews Eligible Sample Size Respnse Rate 32.8% 26.3% 35.2% 34.5% 31.9% Data Prcessing Cding The survey instrument included 22 field-cded questins, 21 pen-ended questins, and 4 clsed-ended Other (Specify) questins. Fr the field-cded questins, pre-cded respnse ptins were visible n the CATI screen and available t the interviewer t cde but they were nt read t the respndent. The interviewer had the chice f either cding the respnse int ne r mre f the pre-cded respnse ptins r cding the respnse as Other and recrding the respndent's answer verbatim. Fr the pen-ended questins, the interviewer recrded the respndent s verbatim respnse t the questin. Fr the clse-ended Other (Specify) questins, the interviewer asked a questin with pre-cded respnse ptins but als had the ability t cde the respnse as Other and enter a text string t summarize the respnse if it fell utside f the respnses listed. Fr each applicable questin, APPRISE staff reviewed the verbatim respnse and then selected ne f the pre-cded respnses r created a new cde if there were enugh similar verbatim respnses. A new cde was created if 1% r mre f respndents prvided the same answer r if a new cde was deemed useful fr inclusin in future RDD evaluatins. After reviewing the verbatim respnses frm the New Yrk State study, the first site t be cmpleted in the field, verbatim respnses were gruped int categries and cded. A standard set f cdes was develped fr the New Yrk State study then submitted t Nexus Market Research fr review. Fllwing apprval, the same cdes were used fr the fur fllwing studies. Data Prcessing The survey data were checked fr cnsistency with the CATI survey instrument, all skip pattern lgic in the instrument was checked, and additinal data cnsistency checks were run n the survey data. The survey data were cmbined with sample frame data, including respndent zip and cunty. Data were 1 Eligible sample size is calculated by adding the number f eligible respndents t the number f cases where eligibility was unknwn multiplied by the estimated eligibility rate. 5

6 delivered in a number f data file frmats, including SAS, SPSS, Stata, and Excel. All files were labeled with variable labels and value labels. The survey data was divided int three deliverable data sets: a CFL retailer data file, a CFL saturatin data file, and an RDD data file. The three sets included the respnses t different sets f questins. The CFL retailer data included questins BUY10 t BUY11-2 that asked the name and address f retailer where the respndent purchased CFLs. The CFL saturatin data included questin REC3 t that asked the respndent fr their twn, name, and phne number fr participatin in a further study. The RDD data cntained the respnses frm all the remaining questins in the study. Additinal data prcessing was dne with the RDD data fr the New Yrk State and Ohi studies. In these studies, respndents were asked t name their electric utility prvider, and if they were unable t identify their prvider, the twn r city they live in. This infrmatin was checked t verify that the infrmatin the respndent prvided was accurate, then bth the riginal respnses and crrected respnses were included in the data file. This crrectin prcess began first by back-cding all the ther specify respnses, where apprpriate, int existing respnse categries. Secnd, respndents wh identified municipal utility prviders r a rural electric assciatin (REA, Ohi study nly) were identified in the remaining verbatim respnses. These utility prviders were categrized in the crrected data as Municipals and REAs. Third, all respnses were checked t cnfirm that the utility the respndent prvided des in fact serve the lcatin f their residence. In the New Yrk State data, a listing f zip cdes and electric utility prviders that serve each zip was btained frm NYSERDA. Each f the New Yrk State respnses were then frce cleaned against this list and re-categrized where there existed a discrepancy between the utility nted by the respndent and the utility serving the respndent s zip cde. In the Ohi data, the utility nted by the respndent and the cunty in which they reside were checked against the service territry f each electric utility prvider. Fr the twelve cases in which the utility prvider given by the respndent des nt serve the respndent s lcatin, the respnse was re-categrized as a Dn t Knw. A new variable was created in the data set (called dem13_edited that reflects these edits t the utility cmpany questin. In all five studies, the BUY10 t BUY11-2 data was prcessed t highlight several facts: number f CFLs purchased by the respndents, the type f stres they were purchased frm, the name f the stres, and lcatin f the stres. This infrmatin was used t determine the number and percentage f husehlds reprting purchasing CFLs by stre name, type, and lcatin. If a respndent said that they did nt knw hw many light bulbs they bught, that number was replaced with the average f all the light bulbs bught by the respndents. The CFL retailer data was cleaned by stre name and lcatin t determine the number f discrete stres where the CFLs were purchased. The data was first transfrmed t ne recrd per stre with each recrd cntaining basic infrmatin abut the retailer. Then, an edited stre name, address, city, and state was assigned t each f the retailers. This crrected infrmatin will be used fr the CFL retailer survey. Weighting Since RDD surveys tend t under-represent certain parts f this target ppulatin (husehlds cntaining nly yung adults, husehlds with husehld heads with high schl educatin r less, and single persn husehlds fr example), relative weights were develped t furnish results that are 6

7 cnsistent with the gegraphic and demgraphic distributin f husehlds in these areas. The ppulatin parameters used t calculate the weights were derived frm the American Cmmunity Survey (ACS). Weighting the data is an imprtant means f mitigating the effects f differential nn-respnse by gegraphic r demgraphic grup and the inability f RDD samples t include husehlds that are cell phne nly husehlds that have n ther telephne besides cell phnes. Current data indicates that 14.8% f American husehlds are cell-phne nly husehlds. 2 These cell phne nly husehlds tend t be mre likely t cntain yunger adults and t be single adult husehlds. The weighting is intended t mre apprpriately represent these types f husehlds, balance the sample, and bring it int alignment with ppulatin parameters n gegraphic lcatin, husehld member age, highest level f educatin by head f husehld, and husehld size. The weighting fr the New Yrk State and New Yrk City studies were calculated in tw stages: a gegraphic weight and a demgraphic weight. These tw weights were calculated separately but then cmbined int ne final weight. The Ohi, Washingtn, and Hustn studies included nly a demgraphic weight. As cities, there was n way t subgrup the Washingtn and Hustn studies t create gegraphic weights. Fr Ohi, an explratry analysis f the rural versus urban areas f the state shwed n significant differences in sample distributin frm the actual ppulatin distributin and negated the need t create a gegraphic weight. The fllwing prcedures were emplyed: Gegraphic Weight: New Yrk State and New Yrk City Studies Only Cunties were cmbined int regins based n NYSERDA New Yrk Energy Smart Cmmunity Regins. The ppulatin parameter fr the gegraphic weighting was develped using husehld cunts by cunty frm ACS year average data. Schuyler and Hamiltn cunties were excepted frm this prcess. Data fr these cunties is frm 2000 Census. Gegraphic weights were created t bring the sample distributin acrss regins int alignment with distributin f husehlds acrss regins. Demgraphic Weight: All Studies Demgraphic data n the ages f adult husehld members, highest level f educatin f the husehld head, and husehld size was btained frm ACS 2007 data. Weight The sample data was weighted with a gegraphic weight described abve fr nly the New Yrk State and New Yrk City studies. The weighted sample distributin was cmpared n demgraphics (age, educatin, HH size) t ppulatin parameters. A demgraphic weight was calculated t bring sample distributins int alignment with 2 CDC/Natinal Center fr Health Statistics. NHIS Wireless Substitutin, July December < 7

8 ppulatin parameters. The weight at this stage is the gegraphic weight * demgraphic weight. The weights were trimmed s n individual case has t much impact n weighted data. An adjustment was calculated fr use when New Yrk City and State data are cmbined. This multiplier shuld be used nly when analysis is based n the ttal New Yrk State data (State and City data cmbined). The final New Yrk City weight shuld be multiplied by and the final NY state weight shuld be multiplied by.7612 when these tw data sets are cmbined and analyzed tgether as ttal New Yrk State (excluding Nassau and Sufflk cunties). 8