Value Measurement: Dream, Madness, Reality. The Value Scorecard

Size: px

Start display at page:

Download "Value Measurement: Dream, Madness, Reality. The Value Scorecard"

Deborah Parrish
6 years ago
Views:

1 Value Measurement: Dream, Madness, Reality The Value Scorecard J. Stephen Town Former Director of Information & University Librarian University of York, UK CAUL V&I Forum Macquarie University, NSW Wednesday 16 th March 2016

2 Summary Background: History, Sources, Definitions The Context Rationale, Future Leadership The Case Library The Scorecard Framework Dimensions Implementation

3 The Dream? HISTORY AND SOURCES

4 A Quest for value measurement SCONUL VAMP Project Missing persuasive data for University Senior Managers to prove value, impact, comparability and worth of their institutional libraries Northumbria Library Assessment Conferences PhD by Publication (2016) Integrative Chapter Autoethnographic enquiry Eight year case of theory and application

5 Sources I Town, S. (2011). The value of libraries: the relationship between change, evaluation and role. In Baker, D. and Evans, W. (Eds.). Libraries and Society: role, responsibility and future in an age of change. Oxford: Chandos. pp Town, J.S. (2011). Value, impact and the transcendent library: progress and pressures in performance measurement and evaluation. Library Quarterly, 81(1), Town, J.S. and Kyrillidou, M. (2013). Developing a values scorecard. Performance Measurement and Metrics, 14(1), Town, J.S. (2015). Implementing the value scorecard. Performance Measurement and Metrics, 16(3),

6 Sources II Town, S. (2014). The value of people: a review and framework for human capital assessment in academic and research libraries. Performance Measurement and Metrics, 15(1/2), Town, J.S. (2015). Evidence based organizational change: a seven year case study. Library Management, 37(8/9), Town, J.S. (2015). Measures of relationship capital for the value scorecard. Library Management, 36(3), Town, S. (2016). LibQUAL+: a quality survey case study. In Atkinson, J. (Ed.). Quality and the Academic Library. Oxford: Chandos. pp

7 Value The quality or fact of being excellent, useful or desirable Rescher, 1969 Acts or series of acts are steered by multiple and changing clusters of values Williams, 1968

8 Value ambiguity Value is what a thing is worth Value is a valuable thing or quality Value is an idea Najder, 1975

9 Whenever valuation takes place values must enter in.. in evaluation an indispensable recourse to underlying values is involved values cannot be deduced from.. data or logic they have to be chosen

10 Economic valuations tend to be one off, static, historic, defensive or project bounded Values based evaluation more likely to encourage dynamism, development and culture Need for a balanced and contextual answer for library measurement

11 Library value... the value of a [library] service must ultimately be judged in terms of the beneficial effects accruing from its use (Orr, 1973, p. 318) In other words, libraries produce an effect beyond use transactions. The summit of library performance measurement is therefore the provision of evidence [of] libraries demonstrating transcendent benefit

12 The distinction between Quality and Value R. H. Orr. (1973). MEASURING THE GOODNESS OF LIBRARY SERVICES: A GENERAL FRAMEWORK FOR CONSIDERING QUANTITATIVE MEASURES. Journal of Documentation. 29 (3), p318.

13 Reductionism? Abbott, Christine (1994). Performance Measurement in Library and Information Services. London: Aslib, The Association for Information Management. p19.

14 Lancour, 1948 ( and Thompson, 1974) Three phases of academic and research library development: Storehouse Service Educational

15 Madness? RATIONALE AND THE FUTURE

16 HE Measurement future There is no alternative but to play the impact game looking for more indicators of learning [and research] outcomes there are some great individual stories, but we want to put more quants around them Madeleine Atkins, CEO, HE Funding Council (England)

17 UK HE Measurement future Government now requires a Teaching Excellence Framework (similar to REF) Libraries should provide a single compelling measure of contribution to teaching excellence This should preferably be based on reliable big data combination and linked to outcome Stephanie Marshall, CEO, HE Academy

18 ARL Research Futures Scenarios 2030 Global followers Research entrepreneurs Reuse and recycle Disciplines in charge

19 The Dynamic Environment (after Shore, E.) The end of research library coherence ( ) Coherence will only be regained above campus scale through partnership Innovation required to re-achieve coherence, through Digital infrastructure Sustainable funding Publication form control

20 Reality Multiple cross pressures for performance assessment and measurement

21 A response to the context Information level management and strategy Intelligence in social and technical solutions Impact proof Innovation delivery Infrastructure sustainability Intimacy with markets and partners

22 LEADERSHIP PERSPECTIVE

23 Value and work Value creation is the objective of every enterprise, every worker and every leader Cameron, KS et al (2006)

24 Leadership role Advocacy: telling the story (Van House, 1995) Performance imperative (transformational) Direction setting as a key feature of adding value (Zaccaro & Klimoski, 2001) Vision is ideological and stated in values-related terms (House & Shamir, 1993)

25 Some assumptions Leadership vantage point alone has access to all relevant data The task of leadership is to build value for the organization Value depends on values so will be contingent on choice Built value is expressed as (surplus) capital Libraries are more than instrumental services and processes; transcendent effect is key

26 University of York THE CASE LIBRARY

27 Scorecard ( and QAA SED, 2006) Storehouse discourse of worth Human capital disengaged (Staff Survey, 2008) Conservative low-risk momentum culture (Miller & Friesen, 1982, and ARL ESP, 2008) University information strategy awaited Low satisfaction (NSS) Impact confined to Information Literacy Financial confidence Some relationships with departments written off Some external partnerships and contracts (but RLUK membership failure)

28 Values jam (2009)

29 Values revisit (2012)

31 Dimensions and implementation THE VALUE SCORECARD

32 The Value Scorecard

33 Matrix perspective

34 Intent Hospitable to all measurement methods and typologies Reconciles good and goodness Resolves reductionism Centered on values, so can be varied according to the hardest financial or softest social crosspressures No requirement for monetary value estimation or single composite index (Sveiby, 2010)

35 Dimension I LIBRARY CAPITAL

36 The Library Capital dimension Tangible capital Intangible capital Collections value data Services value data Environments value data Valuation and link to values Correlation between this, and impact, and transcendent effects Meta-assets value data Organizational value data KM & intellectual value data Correlation between this, and impact & capital growth

37 Reference analytics

38 Increasing asset value at meta-level

39 Library Capital HUMAN CAPITAL

40 Human Capital Enablers (4 C s) Capacity Minus confounders Absence, turnover Capability Talent: raw & growth Critical mass Climate of Affect Engagement Empowerment Culture of momentum Enablement Programme capability Maturity Outcome proofs Market fit Sustainability Market related impact Strategic fit (over time) Quality & Improvement New product development Contribution to Productivity Creativity Competitive impact Service development Reputational investment

41 York Information structure

42 People development metrics

43 People development metrics

44 Too many approvals are needed for routine decisions in my Department* *Results adjusted for negative phrasing (ie low score is bad, high score is good)

45 The pace of change in the University over the past three years has been positive by Tenure and Age

46 ClimateQUAL vs UK and US Mean Team Psychological Empowerment Organizational Withdrawal Organizational Commitment Organizational Citizenship Behaviors Climate for Continual Learning Task Conflict 7 Climate for Customer Service Interpersonal Conflict Climate for Deep Diversity, 6 Task Engagement Climate for Deep Diversity, Valuing Climate for Racial Diversity Climate for Gender Diversity Climate for Diversity of Ranks Climate for Sexual Orientation Authentic Leadership Co-worker Support for Innovation Leader-Member Relationship Quality Distributive Justice Job Satisfaction Procedureal Justice Climate for Teamwork, Structural Interpersonal Justice Climate for Teamwork, Benefit of Informational Justice Climate for Psychological Safety York UK Mean US Mean

47 Dimension II VIRTUE

48 Virtue definition Ancient Greek arete Related to excellence governs social interactions and good citizenship Social value James Neal (2011) Virtuous cycle or helix

49 Virtue components Impact Instrumental educational and research impact Long term social capital development Improvement Quality & continuous improvement Self-improvement Integrity Equity Fair dealing Partnership Lowering transaction costs through higher trust Investment aim

50 Customer Service Excellence Award assessed: March 2014 Achieved with two Compliance plus scores: Customer Insight Delivery

51 York UK NSS result trends

52 York UK LibQUAL+ result trends

53 Dimension III MOMENTUM

54 Measurement dimensions Momentum = mass x velocity Alignment proof Progress data Correlation between capital development through projects and time Volume, pace, culture, and meta-assessment of organizational capability

55 Information Strategy

56 Strategy Programmes

57 Programme momentum

58 Project momentum

59 Dimension IV RELATIONAL CAPITAL

60 Measurement dimensions Relationship Capital = sum of all relationships Goodwill value increase assessment Relational data (=strength?) Correlation between this, direction, activity, impact & capital growth

61 Relationship Capital - 6 Cs Awareness & Fit Strength & Process Consciousness General audit of relational spaces (7 market model) Congruence Degree of fit of relationship activity to parent institution (gap analysis) Communities Assessment of strength level across all relations CRM data Strength index Communication Measures of the process of communication for relationship development Communications audit Down to individual level

62 Relationship Capital market model

63 Relationship Capital sub-frame Return on relationships Causality Specific outcomes of positive relationships on academic process, innovation, finance, quality & staff development Comeback Specific ensuing returns to the Library of repeat benefits of relationships Transaction costs saved through Trust

64 Measuring relational strength Item Does the ALL regularly attend Board of Studies? Is there a specific departmental library committee? Does the ALL regularly attend other departmental meetings (e.g. undergraduate committee)? Is the ALL on departmental staff mailing lists? Does the department keep the ALL up-to-date with general developments in the department? Is the ALL in regular contact with the Library Rep (in person, phone or ) How is the relationship between the ALL and the Library Rep? Does the Library Rep regularly attend Library Committee? Does the department regularly ask for feedback on the library on module feedback? To what extent is information literacy training embedded in the UG programme? To what extent is information literacy training embedded in the PGT programme? Does the ALL deliver PGR training? Ratin g

65 Departmental action plans: Data

67 Engagement narrative [The Department] have for a long time had a fairly arm s length engagement with the Library though I believe that a significant number of students and staff have been regular and frequent users of the Library both for physical and electronic resources. Over the past few years engagement has increased with more active liaison between the Library and the department both via contact with the Library Representative and via attendance at BoS and Staffstudent Consultative Committee.

69 DATA COLLECTION INSTRUMENT

70 Value & Balanced Scorecards

71 Distinctions Balanced Scorecard Value Scorecard Balanced Simple Performance Discrete measures Single targets Direct data Unbalanced to suit context Complex Insight & prediction Connected Correlation aim Narrative & advocacy

72 Initial Template (BS plus VS)

73 York levels of input Service Unit template reporting (Quarterly) Internal initiatives Action plans from surveys, feedback, lean Strategy projects National & collaborative initiatives & services UK Customer Service Excellence standards ( & RLUK) LAMP; CCM; Research data; OA Publications LibQUAL+; TechQUAL; ClimateQUAL Corporate data (analytics) HR institutional data; UoY Staff Survey; benchmarks Library and other system data

74 Strategic KPIs ( clear departmental KPI toolkit ) 1. Meet need 2. Resource 3. Innovate 4. Engage 5. Align 6. Risk satisfaction measures financial measures new services relational strength University fit failure avoidance

75 The Scorecard score? Virtue Work to do on impact Library Capital Human good progress; other tangible mixed Relationships Progress; CRMs in sight; partnership importance Momentum Good progress on innovation & use in advocacy Overall need to move to correlations

76 Conclusions There are intelligent [libraries] and stupid [libraries] intelligent groups gather information better and adapt better to reality thus we find social intelligence Luis Anglada (2007) quoting Marina (2004)

LibQUAL+ and ClimateQUAL at York. Stephen Town Director of Information University of York

LibQUAL+ and ClimateQUAL at York Stephen Town Director of Information University of York SUMMARY LibQUAL+ results Trends, themes, actions ClimateQUAL results People, themes, actions Strategic contribution