Project Outcome Evaluation: Mid-Year Survey Results Summary

Transcription

1 Prject Outcme Evaluatin: Mid-Year Survey Results Summary February 2016 Prject Outcme is aimed at transfrming strategic decisin making, planning, and advcacy effrts at public libraries by making accessible the tls and resurces they need t cllect and use utcme data. It is led by the Public Library Assciatin (PLA) and funded by the Bill & Melinda Gates Fundatin. The crnerstnes f Prject Outcme are 1) a set f easily deplyable patrn surveys crrespnding t different types f library prgrams that were develped and pilted by field experts cmprising the Perfrmance Measurement Taskfrce (PMTF), and 2) a web-based prtal fr data entry, autmated analysis and reprting, and interactive data visualizatins. During a June 2015 precnference f the American Library Assciatin Annual Cnference, Prject Outcme was launched with a 7.5-hur rientatin and training attended by 105 public library staff and leaders. Since then, participants have registered fr Prject Outcme after learning abut it via PLA cmmunicatins, nline and in-persn events hsted by PLA, r wrd f muth. As part f a multi-year prgram evaluatin, in early December 2015 ORS Impact administered an nline survey f all library staff wh registered with Prject Outcme thrugh early Nvember 2015 (N=413). The Mid-Year Survey was designed t prvide infrmatin t PLA and the PMTF that wuld help them 1) ptimize participants experience f the prject, and 2) develp new tls and resurces fr measuring lng-term patrn utcmes. Table f Cntents Key Findings... 2 Methds... 3 Respnse... 3 Abut the Survey Participants... 3 Data Cnsideratins... 3 Detailed Findings and Ptential Discussin Pints... 4 Supprting Early Engagement in Prject Outcme... 4 Alignment f Prject with Participant Needs... 6 Supprting Successful Cmmunicatins Supprting Peer Netwrks r Infrmatin Exchange Input Related t Advanced Tl Develpment APPENDIX A: Thery f Change fr Prject Outcme This mem summarizes results based n data cllected with the Mid-Year Survey. An assessment f prgram utcmes will ccur in June 2016 (see Appendix A fr intended utcmes f participatin in the Prject Outcme Thery f Change).

2 Prject Outcme Evaluatin: Mid-Year Survey Results Summary Key Findings Fllwing are six key findings based n 213 respndents t the Mid-Year Survey. 1. The rate f resurce access is high (75%) amng respnding libraries. Participants wh attended the launch were mre likely than thers t have accessed prject resurces (86% vs. 72%). Hwever, the respectable rate f access amng thse wh registered mre recently suggests that initial engagement thrugh infrmal channels (e.g., wrd f muth, visiting prject website) is als effective fr mtivating early engagement. 2. There are a number f ways prject cntent and functinality align well with participant needs; hwever, participants want imprvements in accessibility and flexibility, particularly easier access t resurces either by reducing actins required t gain access r by making them easier t find; mre flexibility in survey design, including cntent and frmatting; and mre flexibility in prtal functins, primarily related t aggregating r disaggregating data and develping the charts and reprts. 3. Mst respndents maintain awareness f prject happenings, primarily via s frm prject staff, their preferred vehicle; hwever there are pprtunities t tailr prject cmmunicatins frequency based n participant preference and imprve cmmunicatin arund key messages. 4. Participants clearly want pprtunities fr peer netwrking r infrmatin exchange, primarily t supprt utcme measurement practice and use, and particularly in at-least-partly structured nline venues. 5. Participants frm smaller libraries may need mre mtivatin r supprt t mve frm registratin t accessing resurces and pssibly survey implementatin, as well as t engage in peer netwrking r infrmatin exchange pprtunities. 6. Participant input related t advanced tl develpment suggests that in additin t ffering surveys fr measuring lng-term utcmes, building skills fr using third party data culd prvide the mst bang fr the buck. 2

3 Prject Outcme Evaluatin: Mid-Year Survey Results Summary Methds The nline Mid-Year Survey was administered ver a tw-week perid in December 2015 t all library participants wh registered fr Prject Outcme thrugh early Nvember Prject Outcme staff prvided ORS Impact with a list f registered library participants, as well as participants rles at the library, their cntact infrmatin, and the name and NCES cde f the library where they are emplyed. The latter was used t merge IMLS data n library size (i.e. library service area) with survey respnses. Respnse The survey respnse rate was 43% (213), and respnses represented just under 50% f registered libraries. Respnse was much strnger amng launch participants than amng thse wh registered fr Prject Outcme after the launch (70% vs. 36%). Respnse did nt vary by library size. Abut the Survey Participants Respndents were mst cmmnly thse wh had registered after the launch event (79%, remaining attended launch) and wrk in US public libraries (86%; remaining frm libraries in Canada). They hailed frm 33 US states and fur Canadian prvinces. They were mst cmmnly directrs (41%), fllwed by ther library leaders/managers (34%) and ther staff members (26%). Smaller libraries were mst likely t be represented by directrs and larger libraries were mre likely t be represented by ther types f managers and staff. Data Cnsideratins Fr the purpse f the survey t prvide directinal input and feedback frm participants t infrm prgram imprvement and cntent develpment the resulting data are mre than sufficient. Obtaining a higher rate f respnse, particularly amng nn-launch participants will be mre imprtant fr Year 1 fllw-up data cllectin, which will assess prgram utcmes. Respnse t sme pen-ended questins was limited. When reprting results based n pen-ended questins we nte the number f respnses and the prprtin f respnses ut f thse pssible. Because s few individuals wh wrk in smaller public libraries attended the launch, we cannt, at this pint, disentangle relatins between engagement in the prject and registratin vehicle (launch versus ther), length f time engaged, and library size. Similarly, because participant rle at the library (directr versus ther) is als related t library size, it is als difficult t disentangle relatins between rle and prject engagement. 3

4 Prject Outcme Evaluatin: Mid-Year Survey Results Summary As mre participants take advantage f different registratin vehicles, we will be in a better psitin t explre independent influences f registratin vehicle, library size, and participant rle at the library n engagement and utcmes. In the meantime, Mid-Year Survey results related t these factrs shuld be cnsidered preliminary. Detailed Findings and Ptential Discussin Pints This Mid-Year Survey prvided interesting findings and raised intriguing questins. It als prvided sme directin fr future evaluatin effrts. Belw are detailed results and emergent discussin questins related t the five tpics cvered in the survey: Supprting Early Engagement in Prject Outcme: Rates f prject web resurce access by registratin vehicle and participant characteristics Alignment f Prject with Participant Needs: Experiences navigating the prject website, accessing resurces, implementing surveys, using the data prtal, and using results Supprting Successful Cmmunicatins: Experience f prject cmmunicatins Supprting Peer Netwrks and Infrmatin Exchange: Interests and input related t venue, structure, and tpics f peer netwrks and infrmatin exchange Input Related t Advanced Tl Develpment: Capacity, likelihd f use, and needs related t lng-term utcme measurement Supprting Early Engagement in Prject Outcme The rate f resurce access is high amng respnding libraries. Participants wh attended the launch were mre likely than thers t have accessed prject resurces; hwever, the respectable rate f access amng thse wh registered mre recently suggests that initial engagement thrugh infrmal channels (e.g., wrd f muth, visiting prject website) is als effective fr mtivating early engagement. Finding 1 Prject resurce access is high amng participants. At this early stage, 75% (144) f survey respndents reprted they had accessed prject resurces. 4

5 Prject Outcme Evaluatin: Mid-Year Survey Results Summary Finding 2 The precnference launch was an effective way t prmte early engagement in Prject Outcme amng staff frm larger libraries. 1 Thse wh attended launch were significantly mre likely t have accessed web resurces (86% vs. 72%, p <.05). As stated abve, 70% f launch attendees cmpleted the Mid-Year Survey (cmpared t 36% f thers). Finding 3 Ptential participants frm libraries f all sizes d nt need t participate in an event t be mved t register fr Prject Outcme; infrmal channels (e.g., wrd f muth, visiting prject website) are als effective. Amng thse wh had nt attended launch (168), 75% learned abut it frm prject website r smene else then registered. The remaining participants learned abut it frm a prject webinar then registered (20%) r learned abut it at in-persn meeting hsted by the prject then registered (5%). Finding 4 Participants frm smaller libraries may need mre mtivatin r supprt t mving frm registratin t resurce access and pssibly survey implementatin. Staff frm smaller libraries 2 were less likely t have accessed web resurces (64% vs. 81%, p <.05) Finding 5 Resurce access is dependent n rle at the library directrs are least likely t have accessed resurces, ther managers are mst likely, and staff fall in between. 67% f directrs, 73% f staff, and 87% f ther managers had accessed web resurces (p <.05). Discussin Pints Des the differential rate f web resurce access by library size and rle have implicatins fr utreach, cmmunicatins, and/r supprt? Evaluatin nte: Future analysis shuld explre relatins between participant factrs, timesince-registered, and early engagement s we can understand better what might explain lnger lags between registratin and use f prject resurces. 1 This finding is qualified as applicable t larger libraries because almst all library attendees at the launch event were frm larger libraries. Future evaluatin will be better able t explre variatin in early engagement amng the brader ppulatin f libraries. 2 Smaller libraries refers t single-utlet libraries with a 65,000 r fewer LSA. 5

6 Prject Outcme Evaluatin: Mid-Year Survey Results Summary Alignment f Prject with Participant Needs There are a number f ways prject cntent and functinality align well with participant needs. Participants reprt the website is navigable, the hw-ts and step-by-step prcesses are especially helpful, and the surveys are easy t understand and implement successfully. Hwever, participants want imprvements in accessibility and flexibility, particularly easier access t resurces either by reducing actins required t gain access r by making them easier t find; mre flexibility in survey design, including cntent and frmatting; and mre flexibility in prtal functins, primarily related t aggregating r disaggregating data and develping the charts and reprts. Finding 6 There are a number f ways prject cntent and functinality align with participant needs. 71% f respndents find Prject Outcme website navigatin pretty easy r very easy (68% and 3%, respectively) When asked fr the types f infrmatin r resurces that were mst helpful, respndents mst cmmn respnses were related t implementatin practice, e.g., tutrials, hw-ts, step-by-step prcesses, and best practices (29% f 35 3 ); and abut the survey tls and items (23% f 35). When asked t describe smething that went well when implementing the surveys, respndents mst cmmn respnses were related t survey cntent r ease f survey implementatin (55% f 38 4 ), and 26% spke specifically abut the paper surveys. The step-by-step explanatins and links are wnderful. The sectin that detailed the... Outcme Measurement Prcess.... It was helpful fr us t knw what the prcess lked like befre prceeding. Custmers were receptive... surveys used simple language and cmmunicated utcmes with participants. Paper surveys have the mst success. Physically handing them the paper with a pen gets the mst results. [The surveys were] very easy fr staff and fr patrns. 3 This is 24% f thse wh had accessed prject resurces nline. 4 This is 57% f thse wh had implemented surveys. 6

7 Prject Outcme Evaluatin: Mid-Year Survey Results Summary Finding 7 Participants fund survey results useful, particularly fr validating prgram quality. When asked t describe smething psitive that came ut f reviewing the results the mst cmmn respnses related t hw the results validated prgram quality r a particular aspect f a prgram (48% f 24 5 ); increased their awareness mre generally, particularly related t benchmarking against the natinal data (24%); and gave them ideas fr prgram imprvement (24%). The written cmments really validated what we prvide as an incentive as well as a smething new we tried this year - having activities ut all the time rather than just sme times. Liked t cmpare ur library's averages with the natinal nes. Gives us a gd idea as t hw we're ding relative t ther participating libraries. I learned that library resurces shuld have been a part f bth prgrams that were surveyed this was cncrete feedback that was very helpful. I discvered that ne f the prgrams we're ffering is mre effective than I thught. Finding 8 Participants have an appetite fr even mre implementatin practice supprt (e.g., hw-ts, utilizatin examples), bth frm prject staff and frm peers. When asked t describe types f infrmatin r resurces that are the mst imprtant t imprve r add, the mst cmmn respnse was mre webinars, trainings, r resurces related t implementatin practice, including best practices fr measuring utcmes fr different prgrams r example items; training n survey implementatin; infrmatin n sharing results with different stakehlders; and a resurce explaining what they can and cannt d with the current surveys, prtal functinality, and results prducts (38% f 24 6 ). When asked t describe what they hped t get ut f netwrking r infrmatin exchange with ther participating libraries, the mst cmmn respnse related t implementatin practice (88% f ). When asked t describe challenges they faced when implementing surveys, the mst cmmn respnse related t patrn respnse r participatin, suggesting the need fr mre (r mre accessible) infrmatin and resurces t supprt best practices in data cllectin, in particular (38% f 29 8 ). 5 This is 56% f thse wh had reviewed results. 6 This is 17% f thse wh had accessed resurces nline. 7 This is 62% f thse wh were interested in participating in such netwrks. 8 This is 43% f thse in libraries that had implemented surveys. 7

8 Prject Outcme Evaluatin: Mid-Year Survey Results Summary Are there editable templates r ideas fr presenting the data t internal and/r external clients? I'd like t knw hw thers are using the Prject Outcme tls, hw they are using the infrmatin they gather t cmmunicate value, and what actins they are taking as a result f what they learn. The nline and linked surveys are nt getting any traffic. Hard t get peple t stp and d ANOTHER survey. Whenever yu have early literacy surveys it's hard t crral all f the caregivers t cmplete them. Need mre and better marketing materials fr prmting the surveys. Finding 9 Accessibility and flexibility/custmizability are cnsistently identified as areas needing imprvement. When asked t describe types f infrmatin r resurces they lked fr n the website but were nt able t find, the mst cmmn respnses described a difficulty finding r gaining access t what they needed (42% f 36 9 ); finding r accessing the surveys in particular (33% f 36); and understanding prtal functinality, particularly related t having multiple users within ne library system (22% f 36). When asked t describe types f infrmatin r resurces that are the mst imprtant t imprve r add, the secnd, third, and furth mst cmmn respnses, respectively, related t better access t resurces and prtal functinality (25% f ); increased flexibility arund reprting and data-sharing (21% f 24) and increased flexibility r custmizability in survey develpment (21% f 24). When asked t describe challenges they faced when implementing surveys, the secnd mst cmmn respnse related t desires fr mre flexibility in survey design (31% f ). When asked t describe challenges experienced during results review, the mst cmmn respnse related t limited functinality within the prtal, particularly arund aggregatin and disaggregatin f survey results (43% f ). 9 This is 25% f thse in libraries that had accessed nline resurces. 10 This is 17% f thse in libraries that had accessed nline resurces. 11 This is 43% f thse in libraries that had implemented surveys. 12 This is 31% f thse in libraries that had reviewed results. 8

9 Prject Outcme Evaluatin: Mid-Year Survey Results Summary It's wrth cnsidering hw many barriers yu want t have t allwing libraries t begin using the tls. [We need] easy access t infrmatin abut the prgram. Have nt yet been able t access tls r ther resurces because we have nt yet cmpleted ur prfile. Wrking in a large rganizatin, it has t be easier, mre intuitive fr multiple peple t be able t access site, create surveys, etc. If everyne using has t g thrugh such an extensive training prcess, utilizing utcme surveys, etc. becmes mre difficult.... it is just nt clear t me hw we are t use this as an rganizatin. We need t be able t have individuals respnsible fr their surveys and deadlines, and fr admin and managers t see the results. This is nt hw is seems t be structured. We had existing surveys that were required by ur prgram partners s we ended up having t ask patrns t fill ut tw ne-page surveys n the same dcument. Smetimes peple nly filled ut ne survey. It wuld have been gd t be able t cmbine the questins, but the inflexibility f the tl prevented that. I wuld like mre flexibility with the results, such as ways f cmbining the results f specific prgrams. Discussin Pints What are actins PLA can take t increase awareness and understanding f existing flexibility and functinality related t develping surveys and using the prtal? Is it wrth cnsidering ptins fr decreasing limitatins r barriers t access f particular resurces? Might it be apprpriate t develp a resurce explaining ratinales r reasns fr existing limitatins? Is it wrth cnsidering ptins fr increasing flexibility in survey design and prtal functins, particularly related t aggregatin and disaggregatin f results, and chart r reprt design? Might it be apprpriate t develp a resurce explaining ratinales r reasns fr existing functinality? 9

10 Prject Outcme Evaluatin: Mid-Year Survey Results Summary Supprting Successful Cmmunicatins Mst respndents maintain awareness f prject happenings, primarily via s frm prject staff, their preferred vehicle; hwever there are pprtunities t tailr prject cmmunicatins frequency based n participant preference and imprve cmmunicatin arund key messages. Finding 10 Respndents verwhelmingly maintain awareness f prject happenings, primarily via s frm prject staff, their preferred vehicle. 93% f respndents maintain awareness f what s happening with Prject Outcme. Of thse, 86% maintain awareness thrugh message sent my prject staff 45% thrugh checking the prject website 36% thrugh PLA cmmunicatins (e.g., e-news r PL Online) 34% thrugh regular attendance at scheduled webinars 22% thrugh Facebk ntificatins (18%) and/r checking prject Facebk page (8%) 6% thrugh thers in their library Mst respndents prefer t receive infrmatin frm the prject via (80%), fllwed by regularly scheduled webinars (12%). Finding 11 There is an pprtunity t tailr prject cmmunicatins frequency based n participant preference. In terms f frequency, half prefer t receive prject cmmunicatins as frequently as the need arises; 29% wuld prefer a mnthly digest, 19% a weekly digest, and 2% prefer t seek ut infrmatin themselves. Preferred frequency did nt differ by rle. Finding 12 Lw awareness f advanced tl develpment suggests an pprtunity t imprve cmmunicatins arund key messages. 39% f respndents were aware Prject Outcme is develping a new set f advanced tls. Respndents were significantly mre likely t be aware f advanced tls develpment if they were launch participants (61% vs. 33%, p <.001), tried t access web resurces (45% vs. 21%, p <.01), r attend regularly scheduled webinars (59% vs. 32%, p <.001). 10

11 Prject Outcme Evaluatin: Mid-Year Survey Results Summary Discussin Pints What are tactics PLA can use t maximize the chance that participants receive key messages? T raise the likelihd that participants welcme and cnsume prject cmmunicatins, des PLA want t cnsider tailring cmmunicatins frequency based n preferences cllected during registratin? Supprting Peer Netwrks r Infrmatin Exchange There is a clear interest amng participants in having pprtunities fr peer netwrking r infrmatin exchange, primarily t supprt utcme measurement practice and use, and particularly in at-least-partly structured nline venues. Finding 13 Participants clearly want pprtunities fr peer netwrking r infrmatin exchange, particularly thse wh have already demnstrated early engagement. 94% f participants reprted interest in netwrking r infrmatin exchange with ther prject participants (39% yes, 55% maybe). Of all the pen-ended questins n the Mid-Year Survey, the questins abut peer netwrking received the largest prprtin f respnses, by far. Finding 14 Participants frm smaller libraries may need mre mtivatin r supprt t engage in peer netwrking and infrmatin sharing. Staff frm smaller libraries were less likely t be certain abut interest in netwrking (smaller: yes 25% and maybe 68% vs. yes 45% and maybe 50%, p <.05), r attend an in-persn netwrking event (smaller: 33% vs. 50%, p <.05). Finding 15 Participants are mst interested in infrmatin frm peers that supprts prject-related practice, fllwed by that which supprts using and/r sharing results. When asked t describe what they hped t get ut f netwrking r inf exchange with ther participating libraries, 88% (f ) described smething related t prject-related practices; 13 This is 62% f thse wh were interested in participating in such netwrks. 11

12 Prject Outcme Evaluatin: Mid-Year Survey Results Summary 43% described smething related t analyzing r using the results, mst cmmnly sharing the results/cmmunicating library value t external audiences; 31% described smething related t utcme measurement r data cllectin, mst cmmnly wanting t learn abut ther methds fr cllecting utcmes; 18% percent wanted t hear abut challenges experienced and 11% wanted t hear abut success stries; and 8% spke t the value t cnnecting with thers arund a cmmn interest. Implementatin tips... and hw thers are utilizing in alignment with their strategic plan.... hw thers are using the Prject Outcme tls, hw they are using the infrmatin they gather t cmmunicate value, and what actins they are taking as a result f what they learn. Best practices fr measuring utcmes fr library devices. I'm always interested in hearing hw the surveys are wrking fr ther libraries and if they have implemented smething that has encuraged a higher respnse rate. Exchange f ideas n hw using, what challenges faced and hw vercme, what else is being dne in this area, hw this fits int ther effrts. I dn't think we spend enugh time brainstrming ideas tgether - we need that t address the critical issues facing ur libraries tday. Finding 16 Online venues are mst likely t attract the mst participants, including scheduled events and an nline frum that allws nging access. Amng thse wh expressed an interest in netwrking r infrmatin sharing with ther prject participants (199), 83% indicated they were likely t participate in scheduled nline event (31% very likely); 77% indicated they were likely t participate in an nline frum accessible at any time (34% very likely); and 44% indicated they were likely t participate in an in-persn event (10% very likely). Other suggestins fr venues included a listserv, telecnference r videcnference, and access t a participant cntact list. 12

13 Prject Outcme Evaluatin: Mid-Year Survey Results Summary Finding 17 In a netwrking frum participants wuld appreciate having sme structure (e.g., arund a particular tpic r case) and pprtunity fr discussin r prblem-slving. When asked t describe aspects f peer netwrking venue that wuld me crucial fr the experience meeting their needs, 22% (f ) spke t the imprtance f having sme structure t the event, such as arund a particular tpic, survey r case f fcus; 22% spke t the imprtance f there being pprtunity fr interactin, particularly small grup discussin; and a few (7%) spke t the imprtance f having a mderatr and/r access t prject staff. It's helpful t have shrt presentatins n sme tpic and then be able t dig int details with a small grup. Helps me t mve inf frm shrt term memry int relevant ideas fr the lnger term. They wuld be mderated. Questins, cncerns r tpics culd be submitted ahead f time by participants t mderatrs s that the exchange wuld be fcused. Send agenda t participants ahead f time s that they can be prepared t respnd t specific items by having data r reprt infrmatin available t share. Discussin Pints Is the level f participant-reprted interest in peer netwrking reflected in rates f attendance at netwrking and infrmatin sharing events hsted by PLA? If nt, what might accunt fr that? Are current and/r planned netwrking and infrmatin exchange pprtunities aligned with participants preferences that they prvide sme structure, maximize pprtunity fr discussin and prblem-slving amng participants, and prvide infrmatin and examples f using Prject Outcme data in their libraries and cmmunities? 14 This is 38% f thse wh were interested in participating in such netwrks. 13

14 Prject Outcme Evaluatin: Mid-Year Survey Results Summary Input Related t Advanced Tl Develpment Participant input related t advanced tl develpment suggests that in additin t ffering surveys fr measuring lng-term utcmes, building skills fr using third party data culd prvide the mst bang fr the buck. Finding 18 Of the lng-term utcme measurement methds being cnsidered fr develpment, participants are mst likely t use fllw-up surveys and third party data in bth the near and lng term. At least three quarters f participants reprt a medium r high likelihd their libraries wuld use surveys and third party data t assess lng-term patrn utcmes; under tw thirds prvide thse estimates fr interviews and fcus grups (see Figure 1). Finding 19 Participant input suggests building skills fr using third party data culd prvide the mst bang fr the buck. In additin t results related t use likelihd (see abve), mre participants reprt high r medium skills and capacity amng staff at their libraries fr using fr using surveys and third party data, which prvides a gd fundatin fr ptential supprts and valuable peer netwrking and infrmatin sharing (see Figure 1). Finding 20 There is demand particular tls and knwledge r skills related t measuring lng-term utcmes. When asked t describe tls r infrmatin that wuld be helpful fr measuring lng-term utcmes, 50% (f ) wanted tls fr tracking and/r string ther types f relevant data at the individual patrn level, e.g., t better track the library usage acrss multiple interfaces and t stre persnal infrmatin fr the purpse f fllw-up measurement; 45% wanted knwledge and skills relevant t lng-term utcme measurement and use, mst cmmnly related t interpreting and analyzing data, telling their stry with lng-term data, develping measures, and learning frm ther libraries' experiences. Respndents were mst likely t use lng-term utcme measurement results fr prgram planning (88%) and internal decisin making (87%), fllwed by making prgram adjustments (81%) and advcacy (73%). 15 This is 10% f all respndents. 14

15 Prject Outcme Evaluatin: Mid-Year Survey Results Summary The types f questins that shuld be asked when trying t measure lng term custmer utcmes. I wuld lve t see PLA lead the way in develping CRM practices/sftware fr public libraries. Data analytics t fllw library user patterns and trends. [Best practices regarding hw t] cuple ur utput measures with ur utcme measures t present the fullest picture f ur services and their impact n ur cmmunity. Discussin Pints What may PLA d in respnse t appetite fr tls t track and stre ther types f individual-level data relevant t data assessing, tracking, and making sense f lng-term utcme data? What knwledge- and skill-needs are specific t analyzing, understanding, and using lngterm utcme data? What are the best ways t fulfill thse needs? 15

16 Figure 1 Participants estimated levels f staff skills (A), staff capacity (B), library likelihd f near-term use (C), and library likelihd lng-term use (D) fr each methd being cnsidered fr advanced tl develpment (N=178) A. Estimated levels f staff skills fr using each methd being cnsidered B. Estimated levels f staff capacity fr using each methd being cnsidered High Medium Fllw-up surveys 40% 50% 90% Fllw-up surveys 23% 54% 77% 3rd party data 23% 51% 74% 3rd party data 16% 52% 68% Fcus grups 27% 45% 72% Fcus Grups 16% 43% 59% Interviews 26% 44% 70% Interviews 15% 33% 48% C. Estimated library likelihd f using in the near term each methd being cnsidered D. Estimated library likelihd f making business as usual each methd being cnsidered Fllw-up surveys High 51% Medium 35% 86% Fllw-up surveys 49% 42% 91% 3rd party data 30% 44% 74% 3rd party data 30% 47% 77% Fcus Grups 17% 47% 64% Fcus Grups 17% 46% 63% Interviews 17% 43% 60% Interviews 17% 44% 61%

17 APPENDIX A: Thery f Change fr Prject Outcme

18 Gals Outcmes Activities Resurces Meeting date: 4/1/15 Revised date: 4/7/15 Outcme Map fr Prject Outcme ( ) PLA Leadership & Staff PLA Publicatins & Cnferences Perfrmance Measures Task Frce Prject Outcme Technlgy Infrastructure Prject Outcme Surveys Materials, Webinars & Training Engage Libraries and Supprt Field-level Mvement Building Develp messages fr Beginners, Enthusiasts and Champins Push cmmunicatins ut t the field Activate and supprt champins within and external t the library field Educate/Engage stakehlders at the natinal level Supprt Outcme Measurement & Use f Outcme Data Prvide access t utcme surveys and materials designed t meet differentiated needs f Beginners and Enthusiasts Prvide training and webinars designed t meet needs f Beginners and Enthusiasts Prvide structure/venues fr cmmunities f practice amng participants Increased belief in imprtance f utcme measurement fr planning, advcacy, and cmmunity/partner engagement amng leaders/staff Increased awareness f Prject Outcme and differences between impact data initiatives amng leaders/staff Increased capacity t administer utcme surveys with quality amng leaders/staff Libraries measure utcmes thrugh Prject Outcme surveys and/r ther methds Increased capacity fr utcme-based thinking amng library leaders/staff Leaders/staff use utcme data fr Imprving patrn prgram experience and utcmes Planning and decisin making Advcacy and cmmunity engagement Supprting/engaging partners Increased understanding amng leaders/staff utcmes as distinct frm ther types f library data specific desired utcmes assciated with typical library prgram areas OUTCOMES KEY Individual Library Leader/Staff Library Library Field and Cmmunity Outcmes directly influenced by crrespnding set f activities Linchpin utcme Increased library funding and resurces Increased library cllabratin with partners in wrk tward reaching cmmn cmmunity gals Increased transparency in relatins amng library, cmmunity, and partners Public library services and prgrams are mre aligned with cmmunity needs Increased advcacy/ prmtin f utcme-based thinking and measurement by library field leaders, cmmunity partners, and natinal stakehlders at lcal, reginal, state and natinal levels Increased champining f utcme measurement by staff/leaders t ther staff within libraries Increased rganizatinal supprt/resurces fr utcme-based planning, prgram imprvement and advcacy in libraries Institutinalizatin f utcme-based thinking and increasingly sphisticated utcme measurement and use by libraries Increased actins in the library field t advance utcme measurement Cmmunities and partners have a greater sense f the value f public libraries Outcme-based perfrmance measurement is business as usual thrughut the public library field Public libraries are better funded Cmmunities thrive