Network Rubric Workbook

Network Rubric Workbook

Contents Dimensions of an Effective Network i Network Rubric Purpose.. ii Network Workbook Purpose... ii How to Use the Rubric.. iii Glossary of Key Terms... v Alternative Rating Scales... vi Rubric Context. 1 Dimension 1 Network Management 2 Dimension 2 Sustainable Service Systems.. 5 Dimension 3 Data Driven Decision Making 7 Dimension 4 Policy Expertise and Advocacy.. 9 Dimension 5 Knowledge Management and Dissemination... 11 Action Plan. 13 Learning from Experience... 14 Community Partnership Model.. 15 Table of Contents

Dimensions of an Effective Network Dimensions of an Effective Network Page i

Network Rubric Purpose The KnowHow2GO network rubric (co created with the AED evaluation team and refined with feedback from Lumina Foundation for Education and Paul Vandeventer) is intended to be used as a tool to: help KnowHow2GO sites self assess their work and progress along a research based continuum of network development; drive and coordinate technical assistance (TA) activities at the site and national levels. As discussed at the July 2010 KnowHow2GO Learning Community (KLC) meeting, the rubric also can be used to: inform current and potential network members and supporters of the characteristics of an effective KnowHow2GO network develop collective understanding of and buy in to what network development entails set priorities and an action plan for addressing those priorities help network members define and commit to clear roles and responsibilities. We suggest that you use the rubric for self assessment and reflection. This rubric is a tool that sites and TA providers can use to determine areas of strength and room for growth. The rubric was created to articulate a common understanding of the qualities thought to be important for groups working together to improve college access and success. Network Workbook Purpose The Network Rubric Workbook was created in response to feedback from the July 2010 KLC participants and from ongoing TA calls with sites as they ve piloted the rubric. The workbook includes a modified version of the rubric and a selection of tools and resources to help sites and technical assistance partners use the rubric to full effect. Network Rubric Purpose; Network Workbook Purpose Page ii

How to use the Rubric Getting Familiar with the Rubric Assumptions and Context In front of the rubric, there is space entitled Assumptions and Context. This space is for brief notes on site issues that have influenced your assessments, and describe the network's lead organization and/or network member groups; stage of development; and sources of information (documents, site visits, interviews, surveys, etc). An example might be, "Our network has just been launched, so many of the assessments are based on information from our logic model and implementation plan." Or, "The implementation plan has resulted in numerous staff and organizational changes some of which are still in process." This section should be no more than a few sentences. Dimensions, Attributes, Examples of Evidence Each dimension has associated attributes (numbered cells, column one). Below each attribute are potential sources of evidence to support any assessment (under heading Evidence Includes ). These are intended to be examples of evidence that are the most important ways of knowing whether the network is functioning well in the five dimensions (network management, sustainable service systems, data driven decision making, policy expertise and advocacy, and knowledge development and dissemination), While we think these attributes are important based on research, our experience, and site feedback, they are not necessarily exhaustive. The blank boxes labeled Additional Attribute are included in each dimension so that you can include other attributes that are significant to your network. Completing the Rubric Using Evidence to Assess the Attributes under each Dimension For each attribute, review the sources of evidence in order to give an overall assessment of the attribute. Ratings are from 0 (unknown or has not happened yet) to 4 (occurs consistently with high quality). Alternative rating scales can also be used (see alternatives on page vi). The assessment should be based on the evidence you can provide (presence, or lack thereof, significance, etc.), and not on any mathematical calculation. Again, this is a qualitative assessment tool that sites can use to gauge where it is on a continuum in order to help determine where and in what manner to focus resources and refine plans. How to use the Rubric Page iii

Adding Comments, Strengths, Challenges To the right of each attribute, there is an empty text box for entering any information you think is relevant to the assessment provided. This could be details on the available evidence, any contextual information, or even recommendations for specific next steps. Please be concise, and keep comments to more than a few sentences. The rubric is not intended to capture any and all information that may be useful, just the most significant pieces. For example, because a number of sites have just begun their work in building a network, ratings in that area will probably be low. In this case, it is helpful to include a note such as Network development did not commence until January 2010, and full membership has not been determined. Use of Results and Frequency of Assessment The results a completed rubric are intended to be used initially by multiple audiences. The national partners will share and consider results in order to better align and coordinate TA resources and delivery. TA liaisons will use the completed rubric to help guide conversations and planning with individual sites about programmatic and TA priorities. In conjunction with the network survey conducted by the evaluation team, the site should use the rubric to self assess their network development, as well as to inform priorities and potential revisions to their logic model and implementation plan. Because it is a developmental tool, the rubric is to be used to reassess the work regularly; at a minimum, the rubric should be revisited twice a year, more if useful, for example because of a significant change affecting the network. How to use the Rubric Page iv

Glossary of Key Terms Network 1 : many different organizations working in concert as partners pursuing a common social or civic purpose over a sustained period of time. Networks can be categorized as cooperating (low member risk/minimal investment in collective activity); coordinating (low to moderate member risk/increasing investment in collective activity); and collaborating (higher member risk/high investment in collective activity). Sustainability 2 : the capacity of an organization or a network to thrive over the long term. This involves developing a strategy, building capacity, and securing the resources in a way that meets the current needs while positioning an organization and/or its goals to respond to the needs of the future. Data Driven Decision Making 3 : the systematic and purposeful collection, analysis, and communication of quantitative and/or qualitative data aligned with goals and purposes in order to inform and improve upon current and future activities and actions. Policy 4 : a plan of action adopted by an individual or social group. This may include an explicit line of argument rationalizing the course of action. Types of policies and key policy makers include legal decisions [judges, magistrates, and arbitrators]; legislative constitution, laws/referenda, bills [public and elected officials]; fiscal annual budget acts and regulations [trustees and members of the legislative and executive branches of government]; regulatory executive orders, regulations, and administrative rules [cabinet officers, and agency and executive agencies]; organizational tenure and appointment/articulation agreements, code of ethics, and policy manual/standards [organization board and executive staff]. Advocacy 5 : a process whereby people mobilize to communicate a specific message to a targeted group of people. This includes a broad range of practices that may or may not change the policies, positions, or programs of any type of institution. Knowledge Management 6 : the process that creates or locates knowledge and manages the dissemination and use of knowledge within and between organizations. It comprises knowledge acquisition and dissemination, and the use or responsiveness to knowledge. It goes beyond the accumulation of data and information and should connect, bind and involve individuals and/or organizations by creating a shared context as an outcome of shared experiences, reflection, and dialogue. 1 Vandeventer, P. and Mandell M. (2007), Networks that Work. Community Partners, Los Angeles, CA. 2 National Center for Community Education in collaboration with the Afterschool Alliance. The Road to Sustainability Sustainability Workbook, undated. 3 McREL. Sustaining School Improvement: Data Driven Decision Making, 2003, http://www.mcrel.org and Rand Corporation. Making Sense of Data Driven Decision Making in Education, Rand Education Occasional Paper, 2006, http://www.rand.org 4 http://wordnetweb.princeton.edu/perl/webwn?s=policy. http://www2:slac.stanford/edu/policy.definitions.asp. W.K. Kellogg Foundation. Guidelines for Informing Public Policy, undated. http://www.wkkf.org. 5 Kellogg Foundation. Understanding Your Organization s Advocacy Rights: Do s and Don ts for Nonprofits, undated and Ritu R. Sharma. An Introduction to Advocacy Training Guide. USAID, undated. 6 Darroch, J. (2003) Developing a Measure of Knowledge Management Behaviors and Practices, Journal of Knowledge Management, (5), 41 54; and Fahey, L and Prusak, L. (1998) The Eleven Deadliest Sins of Knowledge Management, California Management Review, 40. Glossary of Key Terms Page v

Alternative Rating Scales Below are alternative scales for rating the status of your network. Use the one that feels most appropriate for your network development. Alternative Rating Scales Page vi

Rubric Context RECORD THE FOLLOWING INFORMATION EACH TIME YOU COMPLETE THIS RUBRIC (IDEALLY TWICE A YEAR OR MORE AS APPROPRIATE) Network Site Reflection Date/Period Participants Assumptions & Context (e.g. current network status, rationale for conducting the assessment, other assumptions or contextual issues) Rubric Rating Scale (indicate which scale is being used) Rubric Context Page 1

Dimension 1 Network Management Purpose, Membership, and Structure Key Attributes Related to the Network Dimensions Rating Comments, Strengths, Challenges 1.1 The network members have a shared purpose This purpose addresses inequities of college access and success for low income, first generation students. Evidence includes: a. Purpose: The network members have agreed on a shared purpose addressing inequities of college access and success for low income, first generation students. b. Priorities, Tasks and Activities: Network members operationalize their shared purpose by identifying, agreeing to, and routinely reviewing current priorities based on pertinent data; and they use planning tools to clarify, and periodically update, these priorities, tasks and activities. Key tools include: i. A logic model presenting the expected outcomes aligned with the network s purpose, and proposed activities and outputs. Outcomes are realistic given the needs (supported by data), context and resources. ii. An implementation plan aligned with the logic model that includes details for carrying out the tasks and activities and a timeline. c. Member motivation: Members can describe how the network purpose complements, and adds value to the work of their own organizations. Members also understand how the advantages of membership offset their investment of resources (e.g., time, in kind contributions, membership dues). Dimension 1 Network Management Page 2

1.2 Network membership includes the stakeholders necessary to achieve the network s purpose, priorities, tasks and activities and proposed outcomes. Evidence includes: a. Appropriate variety of members: Members represent diverse sectors (e.g., Community Based Organizations (CBOs), education, Local Education Funds (LEFs), private sector, government, etc) in relation to network purpose, priorities and activities. In addition, there is a clear process for selecting and orienting network members. b. Mutual respect, trust, understanding: Members are committed to the network s shared purpose. They participate with a sense of equal partnership with other members and share a strong sense of mutual respect and reciprocity as well as a willingness to surface and address conflict successfully. 1.3 There are structures and systems that enable network members to collaborate toward achieving their shared purpose. Evidence includes: a. Clear roles and responsibilities: All members clearly understand their roles and responsibilities and how to carry them out. Membership agreements, MOUs, and other accountability mechanisms are used to formalize these understandings. b. Processes, structures, and tools: The network has written policies, protocols, taskand activity specific work groups to foster participation and enhance network decision making. c. Effective internal and external communication: Network members use formal and informal mechanisms (e.g. regular updates and report back protocols for sharing work group progress and results) to promote productive internal communication Dimension 1 Network Management Page 3

and open discussion of issues. Members also use effective external communication to describe network purpose, priorities, tasks and activities to important publics and stakeholders. d. Shared/Distributed Leadership: Within the network, one or more members demonstrate leadership roles in terms of their understanding and knowledge of postsecondary access and success, as well as their meeting facilitation skills and their ability to unite members into effective action. Sustainability: There is a resource development plan describing how network members will work to solicit the necessary human and financial resources (leveraging cash, in kind resources and volunteers) to support network management and operations now and after Lumina Foundation/other grants end. Additional Attribute (please describe): Dimension 1 Network Management Page 4

Dimension 2 Sustainable Service Systems Key Attributes Related to the Network Dimensions Rating Comments, Strengths, Challenges 2.1 Network members structure their own organizational college access and success systems in ways that enhance the availability, quality, distribution, and coordination of services for the network s target population, as well as promote their own sustainability. Evidence includes: a. Network members routinely assess their organization s capacity to deliver services deemed critical to the network s target population and necessary to achieving the network s purpose. b. Network members review and work jointly to shape the distribution of services across the target populations, avoiding redundancies, where possible, and bringing attention to specific service gaps in ways likely to successfully reduce them. 2.2 The network supports the increase and improvement of college access and success services that member providers offer to the target population. Evidence includes: a. Network members plan collaboratively and undertake tasks and activities to enhance individual organizational effectiveness and promote visibility and public awareness. Network members communicate effectively to increase the demand for services and connect target populations to appropriate resources. b. Network members can describe ways that network participation has led to better cooperation and coordination appropriate to their shared purpose. These include increased numbers of students in the targeted populations served; the more Dimension 2 Sustainable Service Systems Page 5

effective and efficient use of organizational resources; increased resources available to network members; and/or increased availability of high quality services to target populations. 2.3 Network members ensure network sustainability and coherence. Evidence includes: Network members periodically review the network s purpose and proposed outcomes, as well as the services and resources provided by members related to the network s purpose. In addition, members monitor the alignment of services to the network s and evaluate the progress in meeting proposed outcomes. Additional Attribute (please describe): Dimension 2 Sustainable Service Systems Page 6

Dimension 3 Data Driven Decision Making Key Attributes Related to the Network Dimensions Rating Comments, Strengths, Challenges 3.1 Network members use data to inform network purpose, priorities, tasks and activities, as well as to develop membership composition and network policies and practices. Evidence includes: a. Network members know about key external data sources. b. Network members collect and aggregate data concerning college access and success in the geographic area they serve, and periodically review and analyze data to inform network structure, priorities, tasks, activities, and proposed outcomes. c. Network members have developed and routinely share with current and new members data that describe a college access and success related need specific to their target populations. These data are used in the network s logic model, written materials, and funding proposals (those developed by both the network and the individual member organizations), and other public communications. 3.2 Network members use data from external sources and/or internal evaluations to measure progress toward achieving their shared purpose. Evidence includes: a. Network members have prepared a plan for documenting the tasks and activities engaged in collectively by network members. Documentation identifies details of tasks and activities and what network members have learned as a result of them. b. Network members develop and use an evaluation plan and process to assess effectiveness of tasks and activities. The plan describes who will do the work, the Dimension 3 Data Driven Decision Making Page 7

data to be collected, and how data will be analyzed. c. Network members share and analyze evaluation results, considering them in determining and deciding future priorities, tasks and activities. 3.3 The network members use data to build public understanding and will. Evidence includes: Data are used effectively in network communications to foster public understanding of college access and success issues in the community and to build the support necessary to address these issues. Additional Attribute (please describe): Dimension 3 Data Driven Decision Making Page 8

Dimension 4 Policy Expertise and Advocacy Key Attributes Related to the Network Dimensions Rating Comments, Strengths, Challenges 4.1 Network members identify and show strong knowledge of public policies or policy gaps affecting post secondary access and success, including policies influencing K 12 postsecondary alignment and financial aid. Evidence includes: a. Network members have organized task or work groups to address policy issues needing attention, and these groups have recommended both collective network and individual organizational responses to influence policy decisions. b. Network members have identified stakeholders and allies beyond the network, and they know how to reach them and organize to influence them in ways that strengthen policy decisions in favor of target populations. c. Network members demonstrate understanding of policy and political complexity. d. The network logic model reflects proposed policy activities and desired outcomes. Dimension 4 Policy Expertise and Advocacy Page 9

4.2 Network members engage and motivate decision makers and stakeholders to act favorably in support of or opposition to identified policies. Evidence includes: a. Network members identify and reach decision makers and stakeholders related to network and organizational policy priorities. b. Network members demonstrate an understanding of what policy levers they need to pull to bring about change to change statutes, affect funding, and influence agency mandates, etc. c. Network members have reached agreement on a policy issue or issues, and members speak with one voice about them. d. Network members use data to build support around a policy. e. Network members can rally and build public opinion, including spokespeople for policy change. Additional Attribute (please describe): Dimension 4 Policy Expertise and Advocacy Page 10

Dimension 5 Knowledge Development and Dissemination Key Attributes Related to the Network Dimensions Rating Comments, Strengths, Challenges 5.1 Network members have defined the knowledge and understanding they need to attain/gain, aligned with KnowHow2GO goals. They have also defined how they will disseminate this knowledge to stakeholders and the public. Evidence Includes: a. Network members have assessed what we know, what we need to know, and how we will learn it. Network members know they will fill gaps in and/or deepen the knowledge they need to communicate effectively about the network s shared purpose, priorities, and activities. b. Network members regularly communicate with each other through formal and informal means including newsletters, e mails, phone calls, meetings, and (other) electronic methods (e.g., website, webinars, and Ning). c. Network members can name a person or work group responsible for communications. d. The network has developed a communications plan pertaining to programmatic issues; and review and update the plan regularly. The plan includes communication goals. Dimension 5 Knowledge Development and Dissemination Page 11

5.2 Network members consider and share best practices and evaluation results arising from their work and from the larger field (such as groups addressing college access and success in the state, regional and local networks and partnerships; the KLC; and other similar or related initiatives). Evidence Includes: a. Network members share knowledge gained at local, state, regional or national arenas (beyond the KLC). b. Network members plan activities based on best practices in the field as appropriate to members capacities and the site. c. Network members have an agreed upon process for sharing new knowledge and results from the field with one another. d. Network members participate in and contribute to the KLC, Ning, etc. including serving on planning groups, sharing learning, solving problems, and advising network members at other sites, etc. e. Network members provide resources to other KH2GO sites and/or reach out to other sites as well as to KnowHow2GO technical assistance and resource providers to address challenges and provide helpful input. Additional Attribute (please describe): Dimension 5 Knowledge Development and Dissemination Page 12

Action Plan GOAL TIMEFRAME What needs to be done? Who will do it? What resources are needed? How will you assess progress? GOAL TIMEFRAME What needs to be done? Who will do it? What resources are needed? How will you assess progress? Action Plan Page 13

Learning from Experience What worked and why What didn t work and why Concluding Date Documenter(s) Learning from Experience Page 14

Community Partnership Model The Community Partnership Model grew out of the Lumina funded Partnerships for College Access and Success (PCAS). 7 PCAS was an initiative managed by AED that engaged eight communities in developing broadbased partnerships to address systemic changes in college access and success. PCAS information and tools can be found in the PCAS toolkit (see below). The lessons learned from the PCAS initiative were significant drivers in the development of the KnowHow2GO Network Rubric. A subset of the KnowHow2GO sites is currently engaged in scaling this model up from a local context to a statewide network. While not every KnowHow2GO network will pursue this model, this is one example of a framework that has been implemented in a variety of contexts. The Community Partnership Model seeks to engage all relevant stakeholders in the arena of college access and success in many cases facilitating coordination or collaboration between individuals and organizations that have never worked together before. The model necessitates a strong lead organization facilitating, managing, and nurturing the strategic alignment of significant state and local groups and individuals. The model assumes that all state and local actors play a critical role in addressing improved college outcomes for low income, first generation students. This requires that school districts and postsecondary institutions work in consort and with parents, civic organizations, and businesses to build momentum and create the political will for change. 7 Academy for Educational Development (2009). Partnerships for College Access and Success: Using Partnerships as a Strategy. A Technical Assistance, Toolkit and Resource Guide, Washington, D.C. Community Partnership Model Page 15