1. Survey Framework on Knowledge Production Process

Transcription

1 Survey-based approaches to measuring knowledge flows and use of knowledge exchange mechanisms, OECD-KNOWINNO workshop on Measuring the use and impact of Knowledge Exchange Mechanisms Measuring knowledge flow and collaboration for scientific discovery: evidence from a large scale scientists survey in Japan and the US 15 November 2011, USPTO Sadao Nagaoka 1 1. Survey Framework on Knowledge Production Process Hitotsubashi Univ-NISTEP-Georgia Tech scientists sts survey in Japan and the US. Basic findings are reported in the following working paper: Knowledge creation process in science: Key comparative findings from the Hitotsubashi-NISTEP-Georgia Tech scientists survey in Japan and the US (joint with Igami, Walsh and Ijichi ) ( 2 1

2 Purpose of the research Developing systematic and objective data on the knowledge creation process in science at the project level (Japan and the US). Motivations of the research projects Knowledge sources that inspired the project Uncertainty in the knowledge creation process Research competition Composition of the research team Sources of funds Research outputs, e.g., research papers, students, patents, etc. 3 Bottom-up approach to explore structural characteristics in the knowledge creation process Structural characteristics in the knowledge creation process in science and in science-based innovation (3) Analyze the characteristics of the research projects. Project A Project B Project C Project D Project E (2) Collect the profiles of research through a comprehensive questionnaire survey of researchers. Paper A Paper B Paper C Paper D Paper E (1) Select research papers in all fields of science. 4 2

3 Organization of the questionnaire <Inputs> History of the Research Project Total research man-months expended on the Research Project Research funds Sources of Research Funds <Research team> Composition of the authors Number of collaborating researchers, students and technicians who are not coauthors on the paper The number of R&D personnel specifically hired for this Project <Motivation and process> Motivation for the Research Project that yielded the focal paper Research process for the Focal Paper Research method of the research project that yielded the paper <Knowledge source and management> External knowledge sources that inspired the Research Project Research management <Research Environment> Use of advanced research facilities, databases, and the Internet Research competition Threat from competition <Personal Environment> Your roles in the Research Project Family situation Educational background Research career Publication of refereed papers <Outputs> Importance of the Focal Paper in the field Importance of the Focal Paper among all the outputs of the Research Project Types of outputs from the Focal Paper Number of papers produced by the Research Project Training of researchers Follow-up research Projects in collaboration with an external organization Other outputs: research tools <Commercialization> Patent applications Internal commercialization Licensing or sales Start-up companies Standards Effects on industries and on society 5 The population of the survey Articles and letters in the Science Citation Indexes-Expanded d d (Thomson Reuters) Time window: (database year) Forward citation counts were retrieved on December 31, fields in the ESI were adopted 6 3

4 Identification of possible focal papers Highly Cited Papers (H papers) Top 1% highly cited papers in each journal field (22 fields in total) and in each database year; at least one organization of authors should be located in the target country (approximately 3,000 in total). Normal Papers (N papers) Randomly selected papers in each journal field and in each database year from the population p of the survey, excluding the above highly cited papers; at least one organization of authors should be located in the target country (approximately 7,000). 7 and The project is defined as a series of research activities in which the specified focal paper and the other closely related research outcomes were produced. Research projects from which the H papers were produced. Research projects from which the N papers were produced. 8 4

5 Implementation of the survey Survey was conducted on the Web. Japanese survey Survey launch: December 21, 2009 Initial due date: February 9, 2010 Reminders were sent twice (mid of Jan., mid of Feb.) Final due date: April 11, 2010 US survey Initial mail-outs: September November, 2010 Reminder s: November December, 2010 Second (final) reminders: January, Relation between the 22 ESI journal fields, 10 fields, and the large 3 fields 22 ESI journal fields 10 fields large fields Chemistry 1_Chemistry Materials Science 2_Materials Science Physics 3_Physics&Space_Science Science Space Science Computer Science Mathematics Engineering 4_Computer Science&Mathematics 5_Engineering Physical Sciences Environment/Ecology 6_Environment/Ecology&Geosciences Geosciences Clinical Medicine 7_Clinical Psychiatry/Psychology Medicine&Psychiatry/Psychology Medicine Agricultural Sciences Plant & Animal Science 8.1_Agricultural Sciences&Plant & Animal Science Biology & Biochemistry Immunology Life Sciences Microbiology 8.2_Basic Life Sciences Biology & Biochemistry Neuroscience & Behavior Pharmacology & Toxicology Multidisciplinary Economics & Business Social Sciences, general Either of 22 ESI journal fields was assigned Either of 22 ESI journal fields was assigned based on the analysis of the backward citations based on the analysis of the backward citations S_Social Sciences 10 5

6 Response rate, Japan All Focal Papers H papers N papers Survey Survey Response Survey Response Responded Responded Responded (A) - (B) targets rate targets rate(a) targets rate(b) 1_Chemistry % % % 4.6% 2_Materials Science % % % 5.2% 3_Physics&Space_Science % % % 6.6% 4_Computer Science&Mathematics % % % 0.5% 5_Engineering % % % 7.5% 6_Environment/Ecology&Geosci ences % % % 6.7% 7_Clinical Medicine&Psychiatry/Psycholog % % % -0.5% 8.1_Agricultural Sciences&Plant & Animal Science % % % 5.8% 8.2_Basic Life Sciences % % % -4.2% 9_Multidisciplinary(*) % % - S_Social Sciences % % % -11.7% Total 7,652 2, % 1, % 5,720 1, % 2.8% 11 Response rate, US All Focal Papers H papers N papers Survey Survey Response Survey Response Responded Responded Responded (A) - (B) targets rate targets rate(a) targets rate(b) 1_Chemistry % % % 6.6% 6% 2_Materials Science % % % -1.1% 3_Physics&Space_Science % % % 2.4% 4_Computer Science&Mathematics % % % -3.2% 5_Engineering % % % 3.4% 6_Environment/Ecology&Geosci ences % % % 0.3% 7_Clinical Medicine&Psychiatry/Psycholog % % % 1.5% 8.1_Agricultural Sciences&Plant & Animal Science % % % 3.5% 8.2_Basic Life Sciences % % % 0.7% 9_Multidisciplinary(*) % % % -14.5% S_Social Sciences % % % 5.1% Total 8,864 2, % 2, % 5,982 1, % 2.1% 12 6

7 Composition of the field of science Physical sciences, 58% Life sciences, 42% 1.Chemistry Japan 13% 7% 19% 4% 10% 6% 13% 9% 20% 2.Materials Science 3.Physics & Space Science 4.Computer Science & Mathematics Physical sciences, 47% Life sciences, 53% US 9%3% 12% 6% 8% 9% 21% 7% 24% 5.Engineering 6.Environment/Ecology & Geosciences 7.Clinical ClinicalMedicine & Psychiatry/Psychology 8.1.Agriculture Science 8.2.Basic Life Science 0% 20% 40% 60% 80% 100% Note1: Results of all respondents 13 Sector of respondents Japan 76% 16% 6% University, etc Public research org. Private firm US 4% 79% 11% 4% 2% Private and non-profit org. Other Do not know 0% 20% 40% 60% 80% 100% Note1: Results of all respondents 14 7

8 2. Knowledge Sources <Knowledge sources> The survey asked a respondent to evaluate how each knowledge was important for suggesting the research project It identified 11 categories of knowledge sources based on pretesting, - including scientific literature, colleagues or past collaborators, visiting researchers and post-doctoral students, competitors, partners in industrial-academic- government alliance as well as - experts in a different field or with a different skill (which have overlaps with the other categories). New facilities and database as knowledge source (only in Japan) 15 <Knowledge sources> Importance of external knowledge sources for inspiring the research project (%, very important) Japan (a) Scientific literature (Articles in journals, etc.) 48% 46% (b) Scientific literature (Preprints, information on websites, those 19% 14% (d) Patent literature 1% 1% (e) Conferences, workshops, and academic meetings 21% 15% (i) Colleagues in the organization (a university, a laboratory, etc.) 25% 22% (j) Visiting researchers or post-doctoral students in the organization 17% 11% (k) Past research collaborators 13% 12% (l) Competitors 8% 4% (m) Partners in industrial - academic - government alliance 8% 5% (n) Researchers in different academic fields 7% 6% (o) Researchers with different research skills (experimental 11% 8% 0% 10% 20% 30% 40% 50% 60% Note1: The results show the share of "very important" in the importance of knowledge sources for conceiving the research project. Note2: The upper figure is for the and the lower figure is for the. Note3: Results weighted by field. 16 8

9 US <Knowledge sources> Importance of external knowledge sources for inspiring conceiving the research project (%, very important) (a) Published literature (articles in journals, etc.) 48% 52% (b) Unpublished literature (preprints, information on websites, etc.) 6% 5% (c) Patent literature 1% 1% (d) Conferences, workshops or academic meetings 9% 9% (e) Colleagues in your organization (university, government lab, 15% 14% (f) Visiting researchers or post-doctoral researchers in your 12% 10% (g) Past research collaborators 11% 11% (h) Competitors 3% 2% (i) Partners in an industrial-academic-government alliance 8% 7% (j) Researchers in different academic fields 9% 8% (k) Researchers with different research skills (for example, 15% 12% 0% 10% 20% 30% 40% 50% 60% Note1: The results show the share of "very important" in the importance of knowledge sources for conceiving the research project. Note2: The upper figure is for the and the lower figure is for the. Note3: Results weighted by field. 17 <Knowledge sources> Very similar rankings of the importance of knowledge sources in Japan and the US robust findings Scientific literature t is by far the most timportant t knowledge source. - difference between invention and scientific research Colleagues in the organization come next in both countries. Visiting researchers or post-doctoral students in the organization are also important. 18 9

10 Figure Knowledge source for getting an idea for R&D projects (From inventor survey in Japan) 25% 22% 22% 20% 15% 17% 15% 14% 10% 5% 0% 3.0% 2.8% 2.4% 1.6% 19 <Knowledge sources> Location of most important knowledge sources The country location of the knowledge source (such as the location of the key researcher), when the knowledge source is either important or very important for suggesting the research projects Most sources of such knowledge are domestic in the US exception: visiting researchers In Japan, knowledge embodied in researchers and the facilities tend to be domestic, expect for competitors 20 10

11 Percent ranking domestic knowledge sources as most important (for those who rank a source as important) Japan US < Knowledge sources > (a) Scientific literature (Articles 10% 18% (a) Published literature 77% 78% (b) Scientific literature 10% 12% (b) Unpublished literature 74% 79% (d) Patent literature 52% 57% (c) Patent literature 90% 86% (e) Conferences, workshops, 26% 42% (d) Conferences, workshops, 80% 83% (i) Colleagues in the 68% 82% (e) Colleagues in your 89% 89% (j) Visiting researchers or post- 65% 66% (f) Visiting researchers or post- 43% 37% (k) Past research collaborators 62% 67% (g) Past research collaborators 66% 68% (l) Competitors 11% 22% (h) Competitors 74% 63% (m) Partners in industrial - 83% 92% (i) Partners in an industrial- 91% 86% (n) Researchers in different 68% 69% (j) Researchers in different 82% 83% (o) Researchers with different 63% 71% (k) Researchers with different 76% 79% 0% 20% 40% 60% 80% 100% Note1: The upper figure is for the and the lower figure is for the. Note2: Results weighted by field. 0% 20% 40% 60% 80% 100% 21 <Research teams> 3. Collaboration as seen from authors To identify the detailed structure of research teams - the survey asked who were included d in the authors - It also collected the following information on authors of the focal paper (up to six authors). Academic/professional positions Field of expertise (27 fields) Main skill (Theoretical, Experimental, and Clinical) Types of organizational affiliations (University, Public research organization,firm, Private and non-profit organization, Other) Origin of birth 22 11

12 27 academic fields 1. Mathematics 2. Computer science 3. Chemistry 4. Material sciences 5. Physics 6. Space science 7. Earth science 8. Environmental studies, Ecological science 9. Clinical medicine 10. Psychiatric medicine, Psychology 11. Biology, Biochemistry i 12. Immunology 13. Microbiology 14. Molecular biology, Genetics 15. Pharmaceutical science, Toxicology 16. Neuroscience, Behavioral science 17. Agriculture science 18. Botany, Zoology 19. Urban engineering 20. Electrical engineering, Electronic engineering, Information engineering 21. Mechanical engineering 22. Chemical engineering 23. Material engineering 24. Medical engineering 25. Environmental engineering 26. Economics, Management 27. Social science 28. Do not know 23 Japan Scope of authors US <Research teams> (a) Any researcher who only 28% supplied research materials 19% analyzed in the research (b) Any researcher who only 17% supplied or developed the 14% research facilities or (c) Any researcher who only 11% supplied or developed the 7% computer program or database (d) Any researcher who only 8% supplied funds used in the 7% research (a) Any researcher who only 17% supplied research materials 17% analyzed in the research (b) Any researcher who only 11% supplied or developed the 10% research facilities or (c) Any researcher who only 11% supplied or developed the 8% computer program or database (d) Any researcher who only 6% supplied funds used in the 5% research (e) Other 6% 6% (e) Other 5% 5% 0% 10% 20% 30% 40% 50% 0% 10% 20% 30% 40% 50% Note1: Note2: Note3: Note4: The choice is non-exclusive. Others are those researchers who did not provide direct contribution to the research project nor any four of the listed con The upper figure is for the and the lower figure is for the. The weighted result of natural sciences

13 <Research teams> Distributions of number of authors by field, Japan unit: persons Respondents Minimum Q1 Median Q3 Maximum Average Nat_Sci 1_Chem 2_Mat 3_Phys&Space 4_Comp&Math 5_Eng 6_Env/Eco&Geo 7_Clin_Med&Psy 8.1_Agr&Plant&Anim 8.2_Basic_Life S_Social Median Average Note1: Red boxplots indicate the distributions for the ; and blue ones for the. Left end of boxes indicate the first quartiles; and right end of boxes the third quartiles. Left end of whiskers indicate the 5th percentile; and right end of whiskers the 95th percentile. The red bands in bars indicate the medians; and rhombi in bars the means. Note2: Result of social sciences in the was not shown due to the small number of responses. 25 <Research teams> Distributions of number of authors by field, US unit: persons Respondents Minimum Q1 Median Q3 Maximum Average Nat_Sci 1_Chem 2_Mat 3_Phys&Space 4_Comp&Math 5_Eng 6_Env/Eco&Geo 7_Clin_Med&Psy 8.1_Agr&Plant&Anim 8.2_Basic_Life S_Social Median Average Note1: Red boxplots indicate the distributions for the ; and blue ones for the. Left end of boxes indicate the first quartiles; and right end of boxes the third quartiles. Left end of whiskers indicate the 5th percentile; and right end of whiskers the 95th percentile. The red bands in bars indicate the medians; and rhombi in bars the means

14 <Research teams> Scope of authors Authorship is more expansive in Japan. higher percent of authors whose contribution was providing materials or data; or who only contributed specialized facilities This is consistent with a larger size of authors per paper in Japan. 27 Collaboration across fields, origin of birth of authors, skills and sectors (Japan) Combination of academic fields 49% Combination of origins of birth 50% 32.6% 13.4% 2.3% 0.5% 51.3% 48.6% 1.4% 50.1% 27.7% 9.3% 2.1% 0.4% 60.6% 29.8% 0.3% 69.8% 39% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 2 fields 3 fields 4 fields 5 fields 1 field Combination of skills 30% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Japan + Other countries Only other countries Only Japan Combination of sectors 36% 41% 32.5% 3.3% 64.2% 36.3% 0.3% 3.9% 59.5% 26.7% 3.3% 70.0% 27.2% 0.1% 3.1% 69.7% 30% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 30% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 2 skills 3 skills 1 skill 2 sectors 3 sectors 4 sectors 1 sector 28 14

15 Collaboration across fields, origin of birth of authors, skills and sectors (US) Combination of academic fields 54% Combination of origins of birth 80% 35.5% 14.5% 3.0% 0.8% 46.2% 61.3% 18.2% 20.5% 27.1% 11.7% 3.0% 0.6% 57.5% 51.7% 19.2% 29.1% 43% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 2 fields 3 fields 4 fields 5 fields 1 field Combination of skills 37% 70% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Combination of US and abroad Only abroad Only US Combination of sectors 29% 34.0% 3.0% 63.0% 26.3% 0.1% 2.6% 71.0% 29.1% 1.7% 69.3% 0.1% 20.3% 1.4% 78.2% 31% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 22% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 2 skills 3 skills 1 skill 2 sectors 3 sectors 4 sectors 1 sector 29 Fields diversity and performance, Japan Incidence of H paper 80% 70% 1 field 60% 50% 2 fields 3 fields 40% 30% 20% 10% 0% Total research man months <=12 <=24 <=48 <=96 <=192 <=384 <=768 <=1536 <=

16 Country of birth of the first authors by the type of papers (HEIs and natural sciences) (a) N papers HEIs Japan US Japan China Other Asia Europe US Other, unknown Young scholar (297) 71.4% 10.1% 7.7% 3.4% 1.3% 6.1% Senior scholar (552) 89.5% 2.7% 2.4% 2.7% 1.1% 1.6% Young scholar (299) 2.7% 14.7% 13.7% 20.4% 37.8% 10.7% Senior scholar (307) 3.3% 6.5% 13.4% 13.7% 53.7% 9.4% (b) H papers HEIs Japan US Japan China Other Asia Europe US Other, unknown Young scholar (107) 64.5% 6.5% 8.4% 9.3% 5.6% 5.6% Senior scholar (167) 80.8% 3.0% 1.2% 6.0% 2.4% 6.6% Young scholar (132) 2.3% 17.4% 14.4% 26.5% 27.3% 12.1% Senior scholar (129) 0.0% 3.9% 10.9% 13.2% 64.3% 7.8% Note1: The sample focuses on those papers the authors of which are ordered according to the contribution of the authors to the research. Note2: Young scholars: M.A. and/or undergraduate student, PhD student, and Postdoctoral fellow. Senior scholars: Assistant professor level, Associate professor level, Professor level, and other. Note3: The results are not weighted by field. 31 <Diversity of authors> Collaboration as seen from diversity of authors Research teams have more diversified memberships in terms of specialized academic fields, specialized skills, origins of birth and types of sectors in H papers than in N papers in both countries. The US teams are significantly more diversified in the origins of birth than the Japanese teams. Given that international coauthorship in terms of the locations of affiliated organizations of the US is only modestly larger than that of Japan (24% in Japan and 29% in the US in ), the above difference largely reflects the inflow of foreign-born scholars in the US. Somewhat more cross-sector collaboration in Japan 32 16