Examiner Tip Sheet Independent Review Welcme t Independent Review! The purpse f Independent Review is fr each team member t independently evaluate the applicant s prcesses and results by identifying areas f strength and pprtunities fr imprvement (OFIs) relative t the Baldrige Excellence Framewrk. The fllwing pages cntain helpful tips and step-by-step instructins fr cmpleting yur Independent Review. Many examiners have fund this stage f the evaluatin prcess t be bth challenging and valuable. As yu immerse yurself in gathering evidence and suggesting insights t help yur applicant rganizatin imprve, yur wn persnal learning n hw t apply the Baldrige framewrk will add tremendus value t yur rganizatin and enhance yur persnal effectiveness in yur wn rle. A few things t remember befre yu begin 1. Start early! Set a schedule that wrks fr yu and stick t it. Finish yur Independent Review at least tw weeks prir t the cnsensus meeting t allw time fr categry leaders t synthesize. 2. If yu are assigned t a Level 2, 3 r 4 team, yu will access yur screbk via the BOSS sftware. If yu are assigned t a Level 1 team, the Level 1 screbk can be fund n the Examiner Resurces page f TNCPE s website. 3. Yur gal is t help the applicant take the next step. Fcus n what is imprtant t them. 4. Yur teammates are ging t rely n yur Independent Review wrk. Make sure it is as clear and cncise as pssible (make nte f page numbers, figure numbers, etc. fr quick reference). 5. Infrmatin is nt always in the designated area; yu may need t pull frm ther sectins f the applicatin. 6. Remember: We are evaluating the applicant nly against the Baldrige Excellence Framewrk. 7. Be sure t refer t the resurces that are available n the Examiner Resurces page f TNCPE s website. An index f resurce pages is available in the back f yur training ntebk. 8. If yu have truble dwnlading dcuments frm Examiner Resurces, try updating yur web brwser (e.g. Internet Explrer, Mzilla FireFx, etc.) r cntact TNCPE fr additinal assistance. 9. Yu can always cntact TNCPE fr help at (800) 453-6474 r cntact@tncpe.rg. Trust the Prcess!
Steps Lgging n t BOSS Accessing the Baldrige Excellence Framewrk Lgging Yur Time Saving Yur Wrk Actins Access BOSS at https://tn.alliancebss.rg. *BOSS Tip: Bkmark this link t make BOSS easier t access ging frward. The 2015-2016 Baldrige Excellence Framewrk has been laded int the BOSS system fr yur reference as yu cmplete yur review. The apprpriate sectins f the Baldrige Criteria appear as a separate panel n the individual item review wrksheets, as well as the key factrs screen. Yu can als dwnlad a PDF f the Baldrige Excellence Framewrk bklet frm the Team Files tab n the very tp menu. This cpy f the Framewrk is intended fr examiner use nly. By dwnlading the PDF, yu agree nt t distribute the Framewrk, in whle r in part, t ther individuals r grups either internal r external t yur rganizatin. Within the Criteria, retrieve definitins f key terms (shwn in SMALL CAPS) by clicking n them. The BOSS sftware will nt autmatically lg yur time. T manually track yur hurs wrked n the applicatin, select the Lg Time icn frm the very tp menu in BOSS. Once entered, yu cannt edit yur time. BOSS des nt have an aut save feature. Click n the SAVE icn t save yur wrk. Be sure t save ften t avid lsing yur wrk! BOSS will timeut after 60 minutes f inactivity. Yur wrk will nt be saved if yur sessin times ut and yu did nt click the SAVE buttn. Be sure t save yur wrk befre stepping away frm yur cmputer. Spell Check This versin f BOSS uses the brwser spell check. Fr example, if Internet Explrer is yur brwser, misspelled wrds will be underlined in red and can be crrected by using a right-click t view ther spelling ptins. Cpy/Paste Yu can cpy and paste frm dcuments, the Criteria, and glssary terms. Shrtcuts include: CTRL C (cpy) and CTRL V (paste). If cpying and pasting frm a Wrd dcument, check that the text des nt exceed the character cunt in BOSS. Nte: BOSS des NOT allw yu t drag/drp text, pages, files, etc. Fnt Size T increase fnt size, use CTRL + (plus) and t shrink fnt size use CTRL (minus). Using either f these will shift the layut f yur view. T return t the default fnt size, use CTRL 0 (zer).
Step 1: Review the Baldrige Excellence Framewrk (1 hur) Nw that yu are ready t begin yur assignment, the first step is t review the Baldrige Excellence Framewrk. Criteria fr Perfrmance Excellence (pages 4-29) Scring System and Scring Guidelines (pages 30-35) Criteria Respnse Guidelines (pages 36-38) Cre Values and Cncepts (pages 39-43) *Tip: Fr ease f reference, TNCPE recmmends placing a paperclip r tab n the Glssary f Key Terms (pages 47-54). Step 2: Read the Applicatin (3-5 hurs) Step 3: Identify Key Factrs Read the entire applicatin frm beginning t end. As yu read, make nte f the applicant s key prcesses and any results yu wuld expect t see related t these prcesses. Cnsider highlighting key wrds in each paragraph, r using separate highlighters fr A-D-L-I r the different categries. It s kay t mark up and make ntes n the applicatin find the system that wrks fr yu! Als nte any initial thughts yu have regarding gaps in the applicant s respnses r things t watch fr as yu begin yur evaluatin. Create a list f key factrs fr the applicant. A key factr is an attribute f an rganizatin r its envirnment that influences the way the rganizatin perates and the key challenges it faces. Examiners use key factrs t fcus their assessments n what is mst imprtant t the applicant. Access the Key Factrs Screen in BOSS by clicking the Key Factrs tab. Read the Criteria questins fr items P.1 and P.2. As yu read the Org. Prfile, which respnds t these questins, identify key factrs (rganizatinal attributes that influence hw the rganizatin perates). A majrity f key factrs will be fund in the Org. Prfile. Hwever, yu may als find them in the applicant s respnses t the Criteria items, the Intent t Apply frm, etc. Enter these key factrs int BOSS via the fllwing steps: Click Add New Key Factr frm the Key Factrs screen. Select the crrect area t address (P.1a, P.1b, etc.) frm the drp-dwn bx fr the key factr yu have identified. Prvide a key factr name in the apprpriate bx (e.g. Emplyee Prfile r Strategic Challenges). Enter the entire key factr in the bx labeled Key Factr Descriptin. Click Save. *BOSS Tip: Yu may add key factrs t yur screbk at any time thrughut yur evaluatin, as this is an iterative prcess. If yu edit a key factr, yu will need t delete the ld key factr in the items where it was referenced and re-assciate the newly edited key factr t each item. *Tip: Review sample key factrs n the Examiner Resurces page f TNCPE s website.
Step 4: Evaluate each item in Categries 1-7 (2-3 hurs per item) Cmplete the six-step item evaluatin prcess fr each item and recrd yur bservatins in the BOSS sftware. 1. Read the Criteria Read the Criteria item t refresh yur memry and help yu understand the item requirements. Fr example, when evaluating item 2.1, read Criteria item 2.1. 2. Assign key factrs Determine the mst relevant key factrs fr the item. Review the master list f key factrs yu develped frm the Org. Prfile. Cnsidering what the Criteria is asking, which f these rganizatinal attributes wuld have the greatest influence n the applicant s respnses t the item requirements? Access the Independent Review Item Wrksheet Screen by clicking n the tab labeled Item Wrksheets; then select the item yu will evaluate. On the Item Wrksheet Screen, chse the Relevant Key Factrs panel. Use the Add Relevant Key Factr Buttn t select the fur t six key factrs that are mst relevant fr the item. Step #1 Select the key factr yu wish t add (these will be selected frm the master list f key factrs that yu created earlier). Step #2 Fr each key factr, yu have the ptin f chsing the entire key factr (select all) r chsing nly a prtin f it (select text). T chse a prtin f a key factr, highlight the text yu wish t include and click select text. Fr example, yu may want t chse nly the strategic challenges related t the applicant s custmers when evaluating item 3.1 Vice f the Custmer. Repeat these three steps until yu have assigned fur t six key factrs t the item. 3. Read the relevant sectin f the applicatin Fr example, when evaluating item 2.1, read the applicant s respnse t item 2.1. Fr prcess items (Categries 1-6), identify prcesses the applicant uses t address item requirements r any gaps between the Criteria and the applicant s respnse. Fr results items (Categry 7), grup key results and identify any gaps between the Criteria and the applicant s respnse.
4. Analyze the applicant s prcesses and results t identify cmments. Prcesses (A-D-L-I) Evaluate the prcesses yu have identified in the applicatin using the prcess evaluatin factrs: Evaluate the applicant s Apprach (A) by cnsidering whether the applicant s prcess is apprpriate, effective, and systematic (well-rdered, repeatable, and uses data). Evaluate the extent f the applicant s Deplyment (D) by cnsidering whether the apprach yu identified is applied cnsistently and by the apprpriate wrk units. Evaluate the applicant s cycles f Learning (L) by cnsidering hw the apprach has been refined r imprved, hw it encurages innvatin, and whether imprvements are shared acrss ther wrk units and prcesses. Evaluate the extent f the applicant s Integratin (I) by cnsidering whether the apprach yu identified is aligned with the rganizatin s needs identified in the Org. Prfile and hw the prcess interacts with the applicant s ther prcesses (e.g. Have the crrect measures been chsen? Des this apprach prvide utput r data that shuld be used by anther prcess?). Results (Le-T-C-I) Evaluate the applicant s key results using the results evaluatin factrs: Evaluate the applicant s Perfrmance Levels (Le) by cnsidering its current perfrmance n a meaningful measurement scale. Perfrmance levels permit evaluatin relative t past perfrmance, prjectins, gals and apprpriate cmparisns. Evaluate the applicant s perfrmance Trends (T) by cnsidering the directin and rate f change f its results. Generally, a minimum f three histrical data pints is needed t begin t ascertain a trend. Evaluate the applicant s Cmparisns (C) by cnsidering its perfrmance relative t its cmpetitrs, rganizatins prviding similar prducts and services, industry averages, r best-in-class rganizatins. The maturity f the rganizatin shuld help determine which cmparisns are mst relevant. Evaluate the extent f the applicant s Integratin (I) by cnsidering which results measures address imprtant custmer, prduct/service, market, and actin plan perfrmance requirements identified in the Org. Prfile and in prcess items; include valid indicatrs f future perfrmance; and are harmnized acrss prcesses and wrk units t supprt rganizatin-wide gals. *Tip: Review the Prcess & Results Item Evaluatin Factrs dcuments n the Examiner Resurces page f TNCPE s website.
Based n yur analysis, identify the 6-10 mst critical strengths and OFIs fr the applicant and utline them in BOSS. Fr prcess items (Categries 1-6), begin by reviewing the prceses and gaps yu nted during Step #3. Cnsider whether any f the gaps yu identified are imprtant t the applicant. If s, these may be ptential OFI cmments. Fr results items (Categry 7), begin by reviewing the results and gaps yu nted during Step #3. Cnsider whether any f the gaps yu identified are imprtant t the applicant. If s, these may be ptential OFI cmments. Next lk at the analysis yu cmpleted abve using A-D-L-I (prcess items) and Le-T- C-I (results items). Fr prcess items, identify areas f strength (evidence f systematic appraches, evidence f apprpriate deplyment, evidence f cycles f learning, evidence f integratin) and areas f pprtunity (lack f an apprach, incmplete deplyment, unclear cycles f learning, n evidence f integratin). Fr results items, identify areas f strength (gd rganizatinal perfrmance levels, beneficial trends, relevant cmparisns, r evidence f integratin) and areas f pprtunity (lack f perfrmance levels, missing r adverse trends, little t n cmparative data, r n evidence f integratin). Using the applicant s key factrs, determine which f these strength and OFI cmments may be mst critical, and enter them int the BOSS system: On the Item Wrksheets screen select the crrect item and click n either the Strengths r Opprtunities fr Imprvement panel. Click Add Strength r Add Opprtunity fr Imprvement. Select the key factrs relevant t the bservatin using the prvided checkbxes (yu will nly be able t chse frm the 4-6 key factrs yu assigned t the item). Select an Item Reference in the next bx, such as a(1), then click Save Changes. Check the Duble bx t flag high-pririty strengths and OFIs that yu feel are the mst critical t the applicant. In the Cmment bx, write the tpic sentence f yur strength cmment. In the As evidenced by bx, list the specific evidence yu bserved in the applicatin. Use figure numbers where apprpriate t refer t evidence charts, tables, and graphics fund in the applicatin. Use the prvided checkbxes t indicate which evaluatin factrs yur strength r OFI references. Fr example, checking Apprach and Deplyment fr an OFI bservatin indicates that apprach and deplyment are lacking. Click Save after cmpleting each bservatin and repeat these steps t enter each f the 6-10 strength and OFI cmments yu have identified.
5. Scre the item Use the Scre panel t determine a scring range and final scre fr the item yu evaluated: Review the Criteria requirements. Review the strength and OFI cmments yu drafted fr the item, particularly nting any ++ r -- bservatins. Nte the balance and imprtance f these cmments relative t the item requirements and key factrs. The Scring Guidelines have been prvided in BOSS. Fllw these steps in assigning an item scre: Fr prcess items, begin by reading the descriptin f the 50-65% range in the Prcess Scring Guidelines. Cnsider whether this range accurately describes the applicant s Apprach (A), Deplyment (D), Learning (L), and Integratin (I) fr the item. Fr results items, begin by reading the descriptin f the 50-65% range in the Results Scring Guidelines. Cnsider whether this range accurately describes the applicant s Levels (Le), Trends (T), Cmparisns (C), and Integratin (I) fr the item. Mve up r dwn the six ranges until yu find the best fit. The best fit range takes int accunt the descriptins f all the individual evaluatin factrs. Cnsidering the applicant s perfrmance hlistically, click n the mst apprpriate range frm the drp-dwn bx at the bttm f the screen. The applicant des nt need t demnstrate all the characteristics in the selected range; rather, the scre is based n a hlistic view f the fur evaluatin factrs in aggregate. Finally, determine a percentage scre that is a multiple f 5 fr the item. Select the best fit percentage scre in the Item Scre bx and click Save. 6. Write tw feedback-ready cmments fr the item: ne strength and ne OFI The purpse f a cmment is t tell the applicant what it des well r what is needed t imprve. Gd cmments cnvey a single thught and prvide the applicant with meaningful (value-added) feedback that is easy t understand. Use this prcess t draft yur cmment: Be sure yur cmment is grunded in the Criteria. Start with a tpic sentence r main pint f yur cmment, which links t an evaluatin factr (A-D-L-I r Le-T-C-I). Give ne r tw supprting examples frm the applicatin. If yur OFI refers t a gap (smething missing frm the applicatin), tell what is missing (e.g. The applicant lacks cmparative data fr many f its peratinal effectiveness measures ). Finish with a s what statement t indicate why this cmment is imprtant r relevant t the applicant. The s what is usually based n a key factr.
Type yur cmment in the bx labeled Feedback Ready Cmment n the Strength r Opprtunity fr Imprvement panel. Indicate an item reference and whether the cmment is a duble-strength r duble-ofi in the bxes t the right. Click Save. *Tip: Review sample cmments n the Examiner Resurces page f TNCPE s website.