Measuring Capacity and Effectiveness of NIH Public Access Policy Programming as a Model for Open Access

Transcription

1 University of New Mexico UNM Digital Repository Evidence Based Scholarly Communications Conference HSLIC Events Measuring Capacity and Effectiveness of NIH Public Access Policy Programming as a Model for Open Access Tania Bardyn Martin Brennan Paul Camp Janet Carter Sharon Farb Follow this and additional works at: Recommended Citation Bardyn, Tania; Martin Brennan; Paul Camp; Janet Carter; and Sharon Farb. "Measuring Capacity and Effectiveness of NIH Public Access Policy Programming as a Model for Open Access." (2010). This Presentation is brought to you for free and open access by the HSLIC Events at UNM Digital Repository. It has been accepted for inclusion in Evidence Based Scholarly Communications Conference by an authorized administrator of UNM Digital Repository. For more information, please contact amywinter@unm.edu.

2 Measuring Capacity and Effectiveness of NIH Public Access Policy Programming as a Model for Open Access Tania Bardyn, MLIS, AHIP Martin Brennan, MLIS, AHIP Paul Camp, Janet Carter, MLIS Sharon Farb, JD, PhD UCLA Library Louise M. Darling Biomedical Library Digital Collection Services March 11, 2010 This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

3 Objective To effectively promote the importance of the NIH Public Access Mandate as a cornerstone of open access at UCLA Library and identify faculty partners in the translational research community. 2

4 NIH Public Access Policy Workshop 8 workshops offered in 2008/09 and 2009/10 academic years Co-sponsored by Academic Senate, Academic Senate Committee on Library and Scholarly Communications, Office of Intellectual Property Administration Outreach to all faculty including translational researchers Program evaluation survey to measure learning outcomes and identify translational research partners 3

5 About UCLA Biomedical Sciences Calif Nanosystems Neuroscience Building 8

6 Translational Researchers are like alligators 8

7 6

8 library 9

9 Why a Program Evaluation at UCLA? Awards By Federal Sponsors FY (In Millions) Other Federal, 40.00, 7% National Science Foundation, 52.95, 9% US Dept of Education, 16.41, 3% National Aeronautics Space Admin., 12.33, 2% US Dept of Energy, 10.58, 2% US Dept of Defense, 28.41, 5% US Dept of Commerce, 0.36, 0% US Dept of Transportation, 0.11, 0% US Dept of Health and Human Services, , 72% 5

10 Comparison of Workshop Attendees vs. NIH Awards NIH Awards by Major Campus Area FY (in millions) NIH Workshop Attendees by Major Campus Area FY COLLEGE OF LETTERS AND SCIENCES, , 17% OTHER, , 15% COLLEGE OF LETTERS AND SCIENCES, 12, 12% OTHER, 17, 16% HEALTH SCIENCES, , 68% n=103 HEALTH SCIENCES, 74, 72% 7

11 Project Timeline Compile names of Translational Researchers at UCLA Ot Oct. - Dec Compile names of Worksop Attendees at UCLA Oct IRB Application Submitted Nov. 19, 2009 First & Only Reminder Dec. 7, 2009 Drawing & Notification of Winner Dec. 16, 2009 Extensive Review of Survey Jan. Feb Oct. 1, 2009 Oct 15, 2009 Nov. 1, 2009 Nov. 15, 2009 Dec. 1, 2009 Dec. 15, 2009 Jan. 1, 2010 Jan. 15, 2010 Feb. 1, 2010 IRB Exemption Approved Nov. 23, 2009 Close Survey Dec. 15, 2009 Open Survey Nov. 30, 2009 Compilation of Results Dec. 18,

12 Results

13 Survey Respondents by Type Research Administrators 6% Staff Librariansi 13% 4% Faculty 64% Researchers 13% 10

14 Survey Respondents by School Survey of Translational and Other Researchers' Knowledge of the NIH Public Access Policy at UCLA Answer Options Response Percent Response Count David Geffen School of Medicine 72.5% 50 Other (Library, etc.) 15.9% 11 Public Health 5.8% 4 Nursing 43% 4.3% 3 Dentistry 1.4% 1 Engineering 0.0% 0 n=69 13

15 Survey Respondents by Department Survey of Translational and Other Researchers' Knowledge of the NIH Public Access Policy at UCLA Top 10 Departments: Answer Options Response Percent Response Count 1. Medicine 16.4% 9 2. Pediatrics 16.4% 9 3. Radiology 12.7% 7 4. Psychiatry 10.9% 6 5. Library 9.1% 5 6. Obstetrics and Gynecology 7.3% 4 7. Hematology Oncology 5.5% 3 8. Neurosurgery 5.5% 3 9. Pathology and Laboratory Medicine 5.5% All Other 67% 37 n=55 14

16 Did you attend an NIH workshop at UCLA? Unsure 6% No 51% Yes 43% n=69 15

17 Are you currently involved in any NIH funded research? Unsure, 1, 1% No, 17, 25% Yes, 51, 74% n=69 16

18 Survey Demographics A. Translational Researchers 51% B. Not a Translational Researcher 41% C. Unsure 8% n=69 11

19 Medicine (Internal) 16% Pediatrics 16% Radiology 13% Psychiatry 11% Ob/Gyn 7% Hematology/Oncology 6% Pathology & Lab Med 6% Neurosurgery 6% Orthopedics 4% Ophthalmology 4% All others 11% Comparing Departments of Survey Respondents to NIH Funding Top 10 Survey Respondents 2009 Top 10 NIH Funded % Medicine (Internal) 10% Neurology 4% Microbiology/ Immunology 4% Biochemistry 4% Physiology 4% Chemistry 4% Pediatrics 4% Pharmacology 3% Public Health/ Preventive Medicine 2% Psychiatry 39% All others Source: NIH Reporter 12

20 Let s test your knowledge.

21 Which of the following is NOT one of the stated intentions of the NIH Public Access Policy? UCLA Survey of Translational and Other Researchers Knowledge of the NIH Public Access Policy Answer Options Response Percent Response Count To advance science 3.1% 2 To reduce unnecessary research 54.7% 35 To provide public access to taxpayer-funded research 4.7% 3 To improve human health 14.1% 9 I don't know 23.4% 15 n=64 18

22 Which of the following is NOT one of the stated intentions of the NIH Public Access Policy? UCLA Survey of Translational and Other Researchers Knowledge of the NIH Public Access Policy Answer Options Response Percent Response Count To advance science 3.1% 2 To reduce unnecessary research 54.7% 35 To provide public access to taxpayer-funded research 4.7% 3 To improve human health 14.1% 9 I don't know 23.4% 15 n=64 19

23 Which of the following is NOT one of the stated intentions of the NIH Public Access Policy? UCLA Survey of Translational and Other Researchers Knowledge of the NIH Public Access Policy Answer Options Response Percent Response Count To advance science 3.1% 2 To reduce unnecessary research 54.7% 35 To provide public access to taxpayer-funded research 4.7% 3 To improve human health 14.1% 9 I don't know 23.4% 15 45% got it wrong n=64 20

24 The NIH Public Access Policy applies to any final manuscript that is peer reviewed and accepted for publication in a journal on or after April 7, 2008, and fits any of the following criteria EXCEPT: UCLA Survey of Translational and Other Researchers' Knowledge of the NIH Public Access Policy Answer Options Response Percent Response Count Arises from any direct funding from an NIH grant or cooperative agreement active in Fiscal Year % 6 Arises from any direct funding from an NIH contract signed on or after April 7, % 11 Arises from any direct funding from NEA or NEH 50.8% 33 Arises from any direct funding from the NIH Intramural Program 4.6% 3 I don't know 18.5% 12 n=65 21

25 The NIH Public Access Policy applies to any final manuscript that is peer reviewed and accepted for publication in a journal on or after April 7, 2008, and fits any of the following criteria EXCEPT: UCLA Survey of Translational and Other Researchers' Knowledge of the NIH Public Access Policy Answer Options Response Percent Response Count Arises from any direct funding from an NIH grant or cooperative agreement active in Fiscal Year % 6 Arises from any direct funding from an NIH contract signed on or after April 7, % 11 Arises from any direct funding from NEA or NEH 50.8% 33 Arises from any direct funding from the NIH Intramural Program 4.6% 3 I don't know 18.5% 12 n=65 22

26 The NIH Public Access Policy applies to any final manuscript that is peer reviewed and accepted for publication in a journal on or after April 7, 2008, and fits any of the following criteria EXCEPT: UCLA Survey of Translational and Other Researchers' Knowledge of the NIH Public Access Policy 49% responded incorrectly or did not know Answer Options Response Percent Response Count Arises from any direct funding from an NIH grant or cooperative agreement active in Fiscal Year % 6 Arises from any direct funding from an NIH contract signed on or after April 7, % 11 Arises from any direct funding from NEA or NEH 50.8% 33 Arises from any direct funding from the NIH Intramural Program 4.6% 3 I don't know 18.5% 12 n=65 23

27 NIH Public Access Survey Intention t of the Policy 50% n=34 did not know the stated intention of the NIH Public Access Policy n=64 24

28 It is the author's responsibility to insure that any publication agreement, or similar copyright transfer agreement signed prior to publication, allows the final published manuscript to be submitted to NIH in accordance with the Public Access Policy. UCLA Survey of Translational and Other Researchers' Knowledge of the NIH Public Access Policy Answer Options Response Percent Response Count True 87.7% 57 False 92% 9.2% 6 I don't know 3.1% 2 n=65 25

29 It is the author's responsibility to insure that any publication agreement, or similar copyright transfer agreement signed prior to publication, allows the final published manuscript to be submitted to NIH in accordance with the Public Access Policy. UCLA Survey of Translational and Other Researchers' Knowledge of the NIH Public Access Policy Answer Options Response Percent Response Count True 87.7% 57 False 92% 9.2% 6 I don't know 3.1% 2 n=65 26

30 It is the author's responsibility to insure that any publication agreement, or similar copyright transfer agreement signed prior to publication, allows the final published manuscript to be submitted to NIH in accordance with the Public Access Policy. UCLA Survey of Translational and Other Researchers' Knowledge of the NIH Public Access Policy Answer Options Response Percent Response Count True 87.7% 57 False 92% 9.2% 6 I don't know 3.1% 2 n=65 27

31 Submission Procedures UCLA Survey of Translational and Other Researchers' Knowledge of the NIH Public Access Policy Regardless of the journal in which the manuscript is published, the author will always need to manually submit the article to PubMed Central in order to comply with the policy. Answer Options Response Percent Response Count True 92% 9.2% 6 False 80.0% 52 I don't know 10.8% 7 n=65 28

32 Submission Procedures UCLA Survey of Translational and Other Researchers' Knowledge of the NIH Public Access Policy Regardless of the journal in which the manuscript is published, the author will always need to manually submit the article to PubMed Central in order to comply with the policy. Answer Options Response Percent Response Count True 92% 9.2% 6 False 80.0% 52 I don't know 10.8% 7 n=65 29

33 Local Institutional Processes UCLA Survey of Translational and Other Researchers' Knowledge of the NIH Public Access Policy The "Tucker letter" is: Answer Options Response Percent Response Count A letter from UCLA official William Tucker, which should be included in any journal article submission 32.3% 21 A letter from a concerned citizen that sparked the formation of the NIH Public Access Policy 13.8% 9 A letter you should have received in April 2007 requiring you to comply with this policy as UCLA employees 3.1% 2 I don't know 50.8% 33 n=65 30

34 Local Institutional Processes UCLA Survey of Translational and Other Researchers' Knowledge of the NIH Public Access Policy The "Tucker letter" is: Answer Options Response Percent Response Count A letter from UCLA official William Tucker, which should be included in any journal article submission 32.3% 21 A letter from a concerned citizen that sparked the formation of the NIH Public Access Policy 13.8% 9 A letter you should have received in April 2007 requiring you to comply with this policy as UCLA employees 3.1% 2 I don't know 50.8% 33 n=65 31

35 Identifier Knowledge UCLA Survey of Translational and Other Researchers' Knowledge of the NIH Public Access Policy Which of the following identifiers must be included when citing an article in NIH applications, proposals, and progress reports that fall under the NIH Public Access Policy? Answer Options Response Percent Response Count Digital Object Identifier (DOI) 6.2% 4 Persistent URL 00% 0.0% 0 PubMed Central ID (PMCID) 56.9% 37 PubMed ID (PMID) 18.5% 12 I don't know 18.5% 12 n=65 32

36 Identifier Knowledge UCLA Survey of Translational and Other Researchers' Knowledge of the NIH Public Access Policy Which of the following identifiers must be included when citing an article in NIH applications, proposals, and progress reports that fall under the NIH Public Access Policy? Answer Options Response Percent Response Count Digital Object Identifier (DOI) 6.2% 4 Persistent URL 00% 0.0% 0 PubMed Central ID (PMCID) 56.9% 37 PubMed ID (PMID) 18.5% 12 I don't know 18.5% 12 n=65 33

37 Have you submitted an article to PubMed Central? No; I tried but was unsuccessful 3% Yes 43% No; but I haven't needed to yet 54% n=65 34

38 Have you made any attempts to retain your copyrights when publishing in an academic journal? Yes, 46% 4.6% No, 95.4% n=65 35

39 Knowledge Sharing NIH Workshop Attendees Translational Researchers No, 2, 7% Does Not Apply, 1, 4% Does Not Apply, 6, 17% Yes, 24, 89% No, 12, 34% Yes, 17, 49% 36

40 Training: Do you need further training? UCLA Survey of Translational and Other Researchers' Knowledge of the NIH Public Access Policy Answer Options Response Percent Response Count Yes, I certainly need further training 56.9% 37 No, unless the policy changes, in which case I would appreciate an update 38.5% 25 No, I need no further training at all 4.6% 3 n=65 37

41 Training: Ideal format of future training on NIH Public Access Policy and Related Scholarly Communication Issues Format of future training preferred by attendees Format of future training preferred by translational researchers Print (or electronic) step-by-step instructions, 17, 27% Other, 1, 2% In-person sessions, in a classroom setting, 13, 21% Print (or electronic) step-by-step instructions, 20, 36% In-person sessions, in a classroom setting, 8, 15% Self-guided online tutorials, 14, 23% Librarians available for consultation by request, 17, 27% Librarians available for consultation by request, 9, 16% Self-guided online tutorials, 18, 33% n=27 n=35 38

42 Training at UCLA: Mandatory? NIH Workshop Attendees Translational Researchers No, 6, 22% Not Sure, 2, 8% Yes, Mandatory, 19, 17% Not Sure, 5, 14% No, 14, 40% Yes, Mandatory, 16, 46% n=27 n=35 39

43 Recommendations from Survey Respondents I think it is more efficient for the NIH website or other external website to provide such training. The issues are the same at all universities and it is not clear why each institution should provide this information. Since the NIH requires IDs on papers in biosketches and progress reports, that affects investigators competitiveness on grants which is much stronger motivation to comply with the policy than mandated training by UCLA which will force investigators to know the policy, but not necessarily comply with the policy. policy Quote from survey respondent at UCLA, December

44 Future Training Needs The following are issues we ve identified d pertaining i to training i faculty and researcher about the NIH Public Access Policy. Do you think you need further training on this issue? (n=65) 57% (n=37)..yes 39% (n=25) 25)..No, unless the policy changes 5% (n=3)...no, I need no further training at all Method for training on scholarly communication issues: (n=65) Print or electronic step-by-step instructions (55%) Self-guided online tutorials (54%) Librarians available for consultation by request (39%) In-person sessions, in a classroom setting (28%) 40

45