Achieving Non-Inclusive Cache Performance with Inclusive Caches Temporal Locality Aware (TLA) Cache Management Policies

Transcription

1 Ahieving Non-Inlusive Che Performne with Inlusive Ches Temporl Lolity Awre (TLA) Che Mngement Poliies Amer Jleel, Eri Borh, Mlini Bhndru, Simon C. Steely Jr., nd Joel Emer Intel Corportion, VSSAD Msshusetts Institute of Tehnology (MIT) {mer.jleel, eri.borh, mlini.k.bhndru, simon..steely.jr, Abstrt Inlusive hes re ommonly used by proessors to simplify he oherene. However, the trde-off hs been lower performne ompred to non-inlusive nd exlusive hes. Contrry to onventionl wisdom, we show tht the limited performne of inlusive hes is mostly due to inlusion vitims lines tht re evited from the ore hes to stisfy the inlusion property nd not the redued he pity of the hierrhy due to the duplition of dt. These inlusion vitims re inorretly hosen for replement beuse the lst-level he (LLC) is unwre of the temporl lolity of lines in the ore hes. We propose Temporl Lolity Awre (TLA) he mngement poliies to llow n inlusive LLC to be wre of the temporl lolity of lines in the ore hes. We propose three TLA poliies: Temporl Lolity Hints (TLH), Erly Core Invlidtion (ECI), nd Query Bsed Seletion (QBS). All three poliies improve inlusive he performne without requiring ny dditionl hrdwre strutures. In ft, QBS performs similr to non-inlusive he hierrhy. Keywords inlusion; non-inlusion; exlusion; replement I. INTRODUCTION As CMPs beome widespred nd the gp between proessor nd memory speeds ontinues to widen, it is impertive tht proessor rhitets design n effiient nd high performing he hierrhy. One of the key design hoies for multilevel he hierrhy is whether or not to enfore inlusion [6, 27, 7]. While inlusion gretly simplifies the he oherene protool [9, 6], it limits performne when the size of the lrgest he is not signifintly lrger thn the sum of the smller hes. In suh senrios, CPU rhitets resort to non-inlusive [27] or exlusive [7] he hierrhies. This pper fouses on improving inlusive he performne without srifiing its benefits. The inlusion property requires tht the ontents of ll the smller hes of multi-level he hierrhy be subset of the lst-level he (LLC) [6]. When line is evited from the LLC, inlusion is enfored by removing tht line from ll the hes in the hierrhy. We refer to he lines invlidted in the smll hes s result of inlusion s inlusion vitims. The smll hes, from here on referred to s ore hes, hide the temporl lolity from the LLC when they servie requests. Sine replement stte is only updted to Most Reently Used (MRU) on he hits, the LLC replement stte of hot lines onstntly servied by the ore hes deys to lest reently used (LRU) in the LLC. As result, the hot lines beome ndidtes for evition in the LLC. The number of inlusion vitims drmtilly inreses when multiple pplitions ompete for the LLC or when the LLC is not signifintly lrger thn the sum of ll the ore hes. A stright forwrd mehnism to eliminte inlusion vitims would be to remove the requirement tht ore hes be subset of the LLC. Suh he, known s noninlusive he [0], llows he lines to reside in the ore he(s) without lso being duplited in the LLC. In doing so, non-inlusion inreses the effetive pity of the he hierrhy [27, 29]. Unfortuntely, non-inlusion elimintes the nturl snoop filter benefit tht n inlusive LLC provides, thus breking the oherene benefits tht ome with inlusivity [6]. While snoop filters [5, 22, 23] n be used in ddition to the LLC, suh strutures inrese the hrdwre overhed [29] nd verifition omplexity [9]. It would be idel to design he hierrhy tht redues (if not elimintes) the frequeny of inlusion vitims while providing the oherene benefits of inlusion. Thus, our gol is to bridge the performne gp between inlusion nd non-inlusion by improving the mngement of n inlusive LLC. It is widely held belief tht the primry benefits of noninlusive he hierrhy ome from the inrese in the effetive hing pity [, 27, 29]. However, we show tht the first order benefit of non-inlusion is the elimintion of inlusion vitims nd not the extr he pity. In light of this observtion, we illustrte tht inlusive hierrhies n perform similr to non-inlusive hierrhies by preserving hot lines in the ore hes nd extending the life time of these lines in the LLC. We propose Temporl Lolity Awre (TLA) inlusive he mngement poliies to redue the frequeny of hrmful inlusion vitims. The suess of TLA poliies requires identifying lines tht hve high temporl lolity in the ore hes nd preventing LLC replement of these lines until their temporl lolity is exhusted in the ore hes. We propose three TLA he mngement poliies where the identity of hot lines is either onveyed to, derived by or inferred by the LLC. Temporl Lolity Hints (TLH) onvey the temporl lolity of hot lines in the ore hes by sending hints to the LLC to updte its replement stte. With the sme temporl informtion s the ore hes, the LLC is signifintly less likely to hoose vitim tht will use n inlusion vitim. Erly Core Invlidtion (ECI) derives the temporl lolity of line before it beomes LRU in the LLC. The LLC hooses line erly nd invlidtes tht line in the ore hes, while leving the line in the LLC. By observing the ore s subsequent request, the LLC derives the temporl lolity.

2 Core Request Core Request Core Request Miss Flow Fill Flow L LLC vitim evit bk-invlidte Miss Flow Fill Flow L LLC vitim Miss Flow Fill Flow L LLC vitim vitim fill Memory Memory Memory () Inlusive Hierrhy (b) Non-Inlusive Hierrhy () Exlusive Hierrhy Figure : Summry of Che Hierrhies. Query Bsed Seletion (QBS) infers the temporl lolity of line in the LLC by querying the ore hes. The LLC selets line for replement nd queries the ore hes for pprovl. The LLC uses the informtion provided by the ore hes to infer the temporl lolity nd selet vitim. QBS employs the heuristi tht the ore hs better informtion bout temporl lolity, nd it is best suited to propose whih lines n be evited. Our evlutions on 2-ore, 4-ore, nd 8-ore CMPs show tht the three TLA he mngement proposls improve inlusive he performne with QBS performing similr to non-inlusive he. We lso show tht the proposed poliies do not require ny dditionl hrdwre strutures nd only require extr messges in the system. The orgniztion of the pper is s follows: Setion II provides more in depth motivtion for the work; Setion III introdues TLA he mngement; Setion IV desribes our experimentl methodology, followed by our experimentl results in Setion V. Setion VI disusses relted work nd we summrize in Setion VII. II. MOTIVATION Figure illustrtes the different types of he hierrhies. An inlusive he hierrhy requires tht the ontents of the smller he levels be subset of the LLC. When line is evited from the LLC, inlusion is enfored by bkinvlidting (i.e., removing) tht line from the smller he levels (if present). The pity of n inlusive he hierrhy is equl to the size of the LLC. Inlusive hes re benefiil beuse they serve s nturl snoop filter. When n LLC lookup results in miss, no snoops need to be sent to the ore hes beuse they re gurnteed to not be present in the ore hes. A non-inlusive he hierrhy, on the other hnd, does not gurntee tht the smller he levels be subset of the LLC. As suh, the pity of non-inlusive he hierrhy rnges between the size of the LLC nd the sum of ll levels in the he hierrhy. Finlly, n exlusive he hierrhy requires tht the ontents of ny he level re not replited in ny other he level. Exlusive hierrhies first fill he lines t the smllest level in the hierrhy nd fill them into subsequent lrger he levels upon evition from the smller he levels. Exlusive hes require higher bndwidth sine even len vitims from the ore hes hve to be written to the LLC. The pity of n exlusive he hierrhy is equl to the sum of the sizes of ll the hes in the hierrhy. Unfortuntely, non-inlusive nd exlusive hes inrese the pity of hierrhy by srifiing the nturl snoop filter benefits of inlusive hes. In generl, the hoie of he hierrhy is dependent on the rtio of the size of smll hes (i.e., ore hes) to the LLC. Figure 2 ompres the performne of non-inlusive nd exlusive LLCs to n inlusive LLC for vrious ore he to LLC size rtios. The figure shows tht inlusive hes typilly perform worse thn non-inlusive nd exlusive hes. When the LLC size is more thn 8X lrger thn the ore hes, the performne of non-inlusive nd exlusive LLCs re similr to n inlusive LLC. However, when the LLC shrinks, both non-inlusive nd exlusive LLCs perform signifintly better thn inlusive LLCs. Modern dy Intel miroproessors (like the Intel Core i7 proessor) hve :8 he rtio nd implement inlusive LLCs [4] while AMD miroproessors (like the Phenom II proessor) hve :4 he rtio nd implement non-inlusive LLCs []. The figure shows tht inlusive LLCs with :8 rtio hve n verge of 3% (mx 2%) worse performne thn non-inlusive/exlusive LLCs. Similrly, inlusive LLCs with :4 rtio hve n verge of 8% (up to 33%) worse performne thn non-inlusive/exlusive LLCs. For suh he rtios, it is better not to enfore inlusion. Both noninlusive/exlusive LLCs improve performne over inlusive LLCs by eliminting inlusion vitims nd inresing the effetive hing pity of the hierrhy. Sine we nnot hnge the requirement tht inlusion duplite lines in the Performne Reltive to Inlusion Exlusive LLC : 2 : 4 : 8 : 6 Rtio of Core Che Size to LLC Size Figure 2: Performne of Non-Inlusive nd Exlusive LLCs.

3 Next Ref () Bseline Inlusive Che (b) Temporl Lolity Hints (TLH) () Erly Core Invlidtion (ECI) (d) Query Bsed Seletion (QBS) (t = 0) b I L: Hit! L2: - b I L: Hit! --- L2: LRU updte b I L: Hit! L2: - b I L: Hit! L2: - d (t = ) b I L: reple - L2: reple I b I L: reple - L2: reple I b I L: reple - L2: reple I L2: ECI b I L: reple - L2: reple I d d d I d (t = 2) d b L: Hit! L2: - d b L: Hit! --- L2: LRU updte d b L: reple I - L2: Hit! d b L: Hit! L2: - e (t = 3) d d b L: reple d - L2: reple d d b L: reple d - L2: reple b d d b L: reple d - L2: reple b L2: ECI d d b L: reple d - L2: reple b move to MRU (t = 4) e d e d b L: reple d - L2: reple b e e d L: Hit! --- L2: LRU updte Figure 3: Illustrtion of Temporl Lolity Awre Che Mngement Poliies. e e d L: Hit! L2: - e e d L: Hit! L2: - LLC, we fous on eliminting inlusion vitims to improve inlusive he performne. Inlusion vitims mnifest on CMPs whenever pplitions with working set sizes tht fit in the ore hes onurrently exeute with pplitions tht hve working set sizes tht re greter thn or equl to the size of the LLC. In suh situtions, the ore hes hide ll the temporl lolity of the ore he fitting (CCF) pplitions from the LLC. Consequently, the LLC replement stte of he lines belonging to the CCF pplitions dey towrds LRU when LLC fitting (LLCF) or LLC thrshing (LLCT) pplitions exerise the LLC. As result, the CCF pplitions suffer from inlusion vitims even though they hve high temporl lolity in the ore hes. The problem of inlusion vitims is not intrinsi to CMPs lone. It n lso mnifest in single-ore, single-threded system where non-unified instrution nd dt hes shre unified LLC. For exmple, streming pplition whose instrution working set fits in its instrution he n suffer from inlusion vitims when referenes to non-temporl dt evit the instrution working set from the LLC. Mintining inlusion nd voiding inlusion vitims for ode lines is pplible for rhitetures tht require ode lines to be oherent due to self-modifying ode (e.g. x86). Inlusive he performne n be improved if the LLC knew of the temporl lolity of lines in the ore hes. To ddress this problem, the next setion proposes Temporl Lolity Awre he mngement poliies for LLCs. III. TEMPORAL LOCALITY AWARE (TLA) MANAGEMENT In multi-level hierrhy, the ore hes t s filter nd hide the temporl lolity from the LLC. Thus, the LLC only observes the referenes tht miss in ll levels prior to the LLC. For lines tht hve high temporl lolity in the ore hes, the replement stte of the line in the LLC n dey to LRU. The line eventully gets evited from the LLC nd due to the inlusive hierrhy, the line is lso evited from the ore hes (n inlusion vitim). To illustrte this problem, onsider the following unfiltered he referene pttern:..., b,,,, d,, e,, f,,.... Note tht the referenes to '' hve high temporl lolity. Figure 3 illustrtes two-level inlusive he hierrhy with fully ssoitive 2-entry L he nd fully-ssoitive 4- entry LLC. Both the L nd LLC strt with ll entries invlid. The boxes nd rrows represent the LRU hin, with the MRU line on the left nd LRU line on the very right. The figure strts fter the referene to. Figure 3 illustrtes the behvior of n inlusive he hierrhy. The figure shows tht in L '' is the MRU entry nd '' is the LRU entry. In the LLC, '' is the MRU entry, nd the LRU entry is n invlid line, I. The next referene to '' hits in the L he nd updtes '' to the MRU position. Sine '' hits in the L he, only the L replement stte is updted nd the LLC replement stte stys unhnged. The next referene, 'd', misses in the hierrhy nd reples lines '' nd 'I' in the L nd LLC respetively. Line '' is now the LRU line in both the L nd LLC.

4 L hit (updte LRU) TLH (updte LRU) L LLC b I L LLC b I Vitim Bk Invlidte Erly Core Invlidte L LLC b I Yes! No! Evit it Updte LLC LRU, Choose next LLC vitim, repet query Cn LLC Evit This Line? MRU Next Vitim LRU MRU Figure 4: Temporl Lolity Awre (TLA) Che Mngement Poliies. Next Vitim () Temporl Lolity Hints (TLH) (b) Erly Core Invlidtion (ECI) () Query Bsed Seletion (QBS) LRU MRU Next Vitim LRU The next referene to '' hits in the L he. Agin, '' is moved to the MRU position of the L he nd no replement updtes our in the LLC. The next referene 'e' misses in both the L nd LLC. Sine '' is now the LRU entry in the LLC, it is hosen for evition, even though it is the MRU line in the L he. Due to the enforement of inlusion, line '' is n inlusion vitim nd is removed from the L he leving n invlid entry behind (not shown in the figure). Line 'e' tkes the ple of '' in the LLC nd the invlid entry left behind by '' in the L he. Line 'e' is promoted to the MRU position in both the hes. The next referene, '', misses in both the L nd LLC. For the bove referene pttern, line '' misses to memory on every ninth referene to the L he, despite its high temporl lolity. If the LLC were non-inlusive or if the temporl lolity of '' ws not hidden from the LLC by the L, '' would never hve been bk-invlidted nd suffer ore he miss. Reduing the hrmful effets of inlusion requires tht lines with high temporl lolity in the ore hes not be hosen for evition in the LLC until their temporl lolity in the ore hes is exhusted. This implies tht the LLC replement stte should be mde wre of the temporl lolity of lines in the ore hes. A. Temporl Lolity Hints (TLH) Inlusion vitims n be eliminted by onveying the temporl lolity of lines in the ore hes to the LLC. A diret mehnism would be to inform the LLC of the unfiltered temporl lolity. On hit in the ore hes, temporl lolity informtion n be onveyed to the LLC by issuing Temporl Lolity Hint (TLH) (see Figure 4). The hint is non-dt request tht updtes the replement stte of the line in the LLC. The heuristi used is tht if line is hot in ore he, it is worth preserving in the LLC. We illustrte the use of TLHs in Figure 3b. At the L he, the replement behvior remins the sme s the bse se. For every L he hit, the line is moved to the MRU position in the L he nd TLH is sent to the LLC. The LLC uses the hint to promote tht line to the MRU position in its LRU hin. Thus, in our exmple referene pttern, L he hits on '' updte the LRU stte in both the L nd LLC. Thus, when the referene to 'e' ours, '' is not in the LRU position nd is not evited from the LLC. By onveying the temporl lolity of, TLHs prevent inlusion vitims. TLHs n signifintly redue the number of inlusion vitims beuse they keep the LLC replement stte up-todte with the ore hes. However, the downside is tht they send request to the LLC for every hit in the ore hes. Optimiztions n be mde by filtering the number of TLHs sent from the ore hes to the LLC. For exmple, in multilevel hierrhy, the seond level he n issue TLHs insted of the first level he. Alterntively, the L he n issue TLHs for non-mru lines. However, even with filtering of TLHs to the LLC, the number of requests to the LLC is extremely lrge nd does not sle well with inresing number of ores. Thus, for the purpose of this study, we evlute the use of TLHs s limit study to determine the potentil performne of voiding inlusion vitims ltogether. Insted of using TLHs, we propose lterntive low overhed solutions: Erly Core Invlidtion (ECI) nd Query Bsed Seletion (QBS). B. Erly Core Invlidtion (ECI) Erly Core Invlidtion (ECI) is method for deriving the temporl lolity of line. The intent is to invlidte line from the ore hes erly (but retin the line in the LLC) nd observe ore's response to derive the line s temporl lolity. If the line is hot in ore he, the ore will re-request it. If this request rrives before the line gets evited from the LLC, it will be hit. As with ll hits, the temporl lolity t the LLC uses replement stte updte. If ore does not rereferene the line, or the re-referene tkes too muh time, the line will be evited by the next miss to tht set. Sending n ECI to the ore hes is reltively stright forwrd. As illustrted in Figure 4b, upon miss in the LLC, the stndrd vitim flow hppens normlly vitim is seleted nd evited from the LLC nd ore hes to mke room for the inoming line. ECI dds nother step to the miss pth by seleting nother vitim (the next LRU line) nd sending n ECI to the ore hes to hve tht line evited. The line still remins in the LLC 2 though. We illustrte the behvior of ECI in Figure 3. Unlike TLHs no updte trffi to the LLC is required. When the referene to 'd' misses in both the L nd LLC, the miss t the LLC is hndled s usul by eviting the LRU line, I. With ECI, the next LRU line, '', is lso evited from the L he (it is still retined in the LLC). This results in 'd' being MRU in L nd LLC, nd 'I' nd '' being LRU in the L nd LLC. Like the Core i7, diretory is mintined with eh LLC line to determine the ores to whih bk-invlidte must be sent [3]. 2. The diretory bits of the LLC line re updted s usul.

5 respetively. The next referene to '' (t t=3), misses in L but hits in the LLC. This updtes the replement stte of '' in the LLC. By premturely eviting line '' from the ore hes nd then observing the subsequent request for, the LLC derived tht '' ws hot, nd ptured the temporl lolity of ''. Note, however, tht ECI is time sensitive. The ore must re-referene the line within window of opportunity. For instne, hd new line been referened t t=3 before the rereferene to, '' would hve been evited. A line hs to be hot reltive to the time window, time window tht will vry depending on the level of ontention t the LLC. With ECI, the number of lines tht need to be invlidted in the ore hes on eh LLC miss n be either one or two. If n erly invlidted line hs not been re-referened, it is not in ny ore hes. Thus, when the next miss ours, tht line is hosen s the vitim, nd beuse it is not in ny ore hes, the bk-invlidte request is not required. Only the ECI needs to be sent out for the next LLC vitim. In ontrst, if n erly invlidted line is re-referened by ore before LLC evition, its LLC replement stte is updted. Thus, the next miss fter suh hot line resue will need to find vitim to evit, whih will inur norml invlidte s required by inlusion long with the ECI to derive the temporl lolity of the next potentil vitim. Note tht ECI hppens in the shdow of the miss to memory, nd thus is not time ritil. As ompred to TLHs, ECI is lower trffi solution to derive line s temporl lolity. This is beuse the number of TLHs is diretly proportionl to the number of ore he hits (whih is high) while ECI request trffi is diretly proportionl to the number of LLC misses (whih is orders of mgnitude smller thn the number of ore he hits). There re trde-offs though. The erly invlidte is predition tht the line will not be needed by the ore hes. If the predition is inorret, the sheme llows time window (until the next miss to tht set) for the ore hes to orret the predition by re-requesting the line. If the ore re-requests the line in tht time window, then the downside is tht wht would've been hit in the ore hes hs beome hit in the LLC. The dded lteny of getting the dt from the LLC insted of the ore hes is trded ginst extending the line's lifetime in the LLC. The more signifint downside is when the re-request for the line misses the time window. The onsequene is n LLC miss tht must be servied from min memory. The time window for ore to re-referene n erly invlidted line restrits the bility of the LLC to fully derive the temporl lolity, nd thus limits the potentil benefits of ECI. C. Query Bsed Seletion (QBS) To ddress the drwbks of ECI, we propose Query Bsed Seletion (QBS). Like ECI, QBS is lso initited from the LLC. Insted of invlidting line erly nd hving the ores re-request it, QBS voids invlidting lines in the ore hes ltogether. QBS queries the ore hes nd hs them indite if line is resident in the ore hes nd uses tht informtion to infer the temporl lolity of those lines. Lines resident in the ore hes re inferred to hve high temporl lolity nd re not evited from the LLC. As illustrted in Figure 4, on n LLC miss, the he ontroller selets the LRU line s the vitim nd queries the ore hes to determine if the line n be evited from the LLC. If the line is not present in ny of the ore hes, QBS evits this line to mke spe for the inoming line. However, if the urrent vitim line is present in ny of the ore hes, QBS updtes the line's LLC replement stte to MRU nd extend its lifetime in the LLC. Sine vitim still needs to be seleted, new vitim is hosen nd the proess repets. The QBS vitim seletion proess is typilly hidden by memory lteny. Should the dt rrive from memory before QBS selets vitim, the he ontroller n either wit for QBS to finish seleting vitim or fore QBS to selet the next vitim. Alterntively, the he ontroller n limit the number of queries issued on n LLC miss. When the mximum is rehed, the next vitim line is seleted for replement nd no further queries re sent to the ore hes. We illustrte the behvior of QBS in Figure 3d. When the referene to 'e' misses in the LLC, QBS queries the ore hes to determine whether it n evit the first potentil LLC vitim, ''. When the ore hes re queried for '', the ore responds tht the line is present in the L he. Consequently, the LLC replement stte of is updted to MRU nd query for the next vitim, 'b', is sent. Sine b is not resident in the L he, the ore llows 'b' to be repled from the LLC. Thus, when '' is re-referened, it hits in the L he. QBS hs prevented '' from being n inlusion vitim. By preventing the LLC from eviting lines tht re still resident in the ore hes, QBS prevents hot lines from beoming inlusion vitims. QBS ddresses the time window problem of ECI nd voids LLC misses. IV. EXPERIMENTAL METHODOLOGY A. Simultor We use CMP$im [6], Pin [9] bsed tre-driven x86 simultor for our performne studies. Our bseline system is 2-ore CMP. Eh ore in the CMP is 4-wy out-of-order proessor with 28-entry reorder buffer nd three level he hierrhy. Without loss of generlity, we ssume single-thred per ore. The L nd L2 hes re privte to eh ore. The L instrution nd dt hes re 4-wy 32KB eh while the L2 he is unified 8-wy 256KB. The L nd L2 he sizes re kept onstnt in our study. We support two L red ports nd one L write port on the dt he. The bseline lst-level he (LLC) is unified 6-wy 2MB he tht is shred by both the ores in the CMP. We ssume bnked LLC with s mny bnks s there re ores in the system. Like the Intel Core i7, we model non-inlusive L2 he nd only the lst-level he enfores inlusion 3. All hes in the hierrhy use 64B line size. For replement deisions, the L nd L2 hes use the LRU replement poliy while the LLC uses the Not Reently Used (NRU) replement poliy 4 [2, 4]. We model strem prefether tht trins on L2 he misses nd prefethes lines into the L2 he. The prefether hs 6 strem detetors. The lod-to-use 3. Modern proessors [, 2, 3] use non-inlusive L2 hes. If the L2 were inlusive, TLA poliies n be pplied t the L2 he to perform similr to non-inlusive hes. We verified this in our studies. 4. The inlusion problem is independent of the LLC replement poliy. The problem ours with LRU replement s well s more intelligent replement poliies (e.g. RRIP [4]). We verified this in our studies.

6 TABLE I. MPKI of Representtive SPEC CPU2006 Applitions In the Absene of Prefething st bzi l de gob h26 hmm lib mf per pov sje sph wrf xl L MPKI (64KB) L2 MPKI (256KB) LLC MPKI (2MB) ltenies for the L, L2, nd LLC re, 0, nd 24 yles respetively. We model 50 yle penlty to min memory nd support 32 outstnding misses to memory. The he hierrhy orgniztion nd ltenies re bsed on the Intel Core i7 proessor [3]. The proposed poliies do not rely on the speifi ltenies used. We hve verified tht the proposed poliies perform well for different ltenies inluding pure funtionl he simultion. We ompre the poliies using both the throughput 5 nd he performne metris. We model n interonnet with fixed verge lteny. Bndwidth onto the interonnet is modeled using fixed number of MSHRs. Contention for the MSHRs models the inrese in lteny due to dditionl trffi introdued into the system. All trnstions, inluding those for ECI nd QBS, use existing dt pths nd ontend for the MSHRs. We do not model bndwidth limittions for the Temporl Lolity Hints (TLH). TLHs serve s limit study on potentil performne improvement if temporl lolity informtion from ll levels in the hierrhy were vilble t the LLC. We lso ompre the performne of the proposed TLA poliies to non-inlusive nd exlusive he hierrhies. Che ltenies re identil to the bseline inlusive he hierrhy for both these he hierrhies. A non-inlusive he hierrhy is modeled by not sending bk-invlidtes to the ore hes when repling lines from the LLC. To model n exlusive he hierrhy we modify both the hit nd miss pths. Lines re invlidted in the LLC upon he hits. As for the miss pth, new lines re inserted into the ore hes first. These lines re inserted into the LLC only fter they re evited from the ore hes. Exlusive hes typilly require higher LLC bndwidth beuse even len vitims from the ore hes must be inserted into the LLC. We do not model this inresed bndwidth, nd thus our results for exlusive hes re optimisti. B. Benhmrks The SPEC CPU2006 benhmrks were first grouped into three different tegories bsed on their L, L2, nd LLC he hit behvior. The first tegory is those pplitions whose working set fits in the ore hes. We refer to these pplitions s ore he fitting (CCF) pplitions. The seond tegory is those pplitions whose working set fits in the LLC. We refer to these pplitions s LLC fitting (LLCF) pplitions. Finlly, the third tegory is those pplitions whose working set is lrger thn the LLC. We refer to these pplitions s LLC thrshing (LLCT) pplitions. Of ll the SPEC CPU2006 benhmrks, we seleted five from eh 5. We ompred the performne of the TLA poliies on both the weighted speedup nd hmen-firness metris. Sine the TLA poliies do not introdue ny firness issues, they perform similr to the throughput metri for both weighted speedup nd hmen-firness metris. tegory to over the spetrum of hit/miss behvior in the different levels of the he hierrhy. The 5 representtive SPEC CPU2006 benhmrks were ompiled using the i ompiler with full optimiztion flgs. Representtive regions for the SPEC benhmrks were ll olleted using PinPoints [20]. Tble I lists the 5 SPEC CPU2006 benhmrks nd their misses per 000 instrutions (MPKI) in the L, L2, nd LLC when run in isoltion. The MPKI numbers re reported in the bsene of prefether. Bsed on the MPKI vlues in Tble I, delii, h264ref, perlbenh, povry, nd sjeng ll hve smll miss rtes in the L2 he. This implies tht these benhmrks hve working sets tht fit into the ore hes. Thus, for our bseline he hierrhy, we n lssify these pplitions s CCF pplitions. The benhmrks gobmk, libquntum, mf, sphinx3, nd wrf ll hve working set sizes tht re lrger thn the bseline LLC size. This is beuse these pplitions hve lmost s mny misses s there re esses to the LLC. Thus, for our bseline he hierrhy, these pplitions n be lssified s LLCT pplitions. Finlly, the benhmrks str, bzip2, lulix, hmmer, nd xlnbmk ll hve working set sizes lrger thn the L2 he but benefit from the LLC. Thus, for our bseline he hierrhy, we n lssify these pplitions s LLCF pplitions. In generl, CCF pplitions re vulnerble to inlusion relted evitions when onurrently exeuting with LLCT or LLCF pplitions. To study the effets of inlusion, we rn ll possible two-threded ombintions of the 5 SPEC benhmrks, i.e. 5 hoose 2 05 worklods. To provide insights on when TLA poliies re benefiil, we seleted 2 worklod mixes (listed in Tble II) to showse results. However, we provide results for ll 05 worklods. We simulted 250 million instrutions for eh benhmrk. Simultions ontinue to exeute until ll benhmrks in the worklod mix exeute t lest 250 million instrutions. If fster thred finishes its 250M instrutions, it ontinues to exeute to ompete for he resoures. We only ollet sttistis for the first 250 million instrutions ommitted by eh pplition. This methodology is similr to existing work on shred he mngement [5, 2, 25]. TABLE II. Worklod Mixes Nme Apps Ctegory Nme Apps Ctegory MIX_00 bzi, wrf LLCF, LLCT MIX_06 hmm, xl LLCF, LLCF MIX_0 de, pov CCF, CCF MIX_07 de, wrf CCF, LLCT MIX_02 l, gob LLCF, LLCT MIX_08 bzi, sje LLCF, CCF MIX_03 h26, per CCF, CCF MIX_09 pov, mf CCF, LLCT MIX_04 gob, mf LLCT, LLCT MIX_0 lib, sje LLCT, CCF MIX_05 h26, gob CCF, LLCT MIX_ st, pov LLCF, CCF

7 V. RESULTS AND ANALYSIS A. Temporl Lolity Hints A Limit Study Figure 5 shows the throughput normlized to the bseline inlusive he hierrhy when ll requests t different levels in the hierrhy send TLHs. As reminder, we do not model ny bndwidth onstrints when sending TLHs to the LLC. The first three brs in the figure show the performne when the L instrution he lone sends TLHs (TLH-IL), the L dt he lone sends TLHs (TLH-DL), nd when both L instrution nd dt hes send TLHs (TLH-L). The x-xis represents the different worklod mixes nd the br lbeled All represents the geomen of ll 05 worklods used in the study. TLH-IL nd TLH-DL lone improve throughput by 5% or more for the worklod mixes MIX_08, MIX_09, MIX_0, nd MIX_. For these worklod mixes, the performne improvements re dditive when both the IL nd DL hes send TLHs (i.e., TLH-L). This is beuse these worklod mixes re omposed of CCF pplition tht is running onurrently with n LLCT or LLCF pplition. Sending TLHs from the ore hes prevents LLCT nd LLCF pplitions from eviting the he lines of CCF pplitions from the LLC. This llows TLHs to improve the performne of n inlusive he by s muh s 3%. The figure lso shows tht homogeneous worklod mixes of CCF pplitions (MIX_0 nd MIX_03) or worklod mixes tht hve ombintion of LLCT nd LLCF pplitions (worklod mixes MIX_00 nd MIX_02) reeive no benefits from TLHs. For suh mixes, inlusion vitims re not problem beuse either ll benhmrks hve working sets tht fit in the ore hes or ll hve working set sizes tht re lrger thn the ore hes. On verge, TLH-IL, TLH-DL, nd TLH-L improve throughput by 2.6%, 2.5% nd 5.2% respetively. Figure 5 lso presents the performne of TLHs when only the L2 he hits send TLH to the LLC (TLH-L2). On verge, TLH-L2 hs roughly two-thirds the performne of TLH-L. For exmple, MIX_0 observes only 7% improvement in throughput with L2 generted TLHs ompred to 24% improvement from L generted TLHs. MIX_0, onsists of benhmrks libquntum nd sjeng. From Tble I, we see tht libquntum hs no lolity in ny of the hes while sjeng hs good L he lolity. Thus, TLH-L2 is unble to refresh the LRU stte of the lines belonging to sjeng in the LLC. Figure 5 lso illustrtes tht the performne of TLH-L nd TLH-L2 is not dditive when both the L nd L2 hes send TLHs (TLH-L-L2). In ft, TLH-L provides the bulk of the performne improvements when ll levels in the hierrhy send TLHs. Figure 5 shows tht TLH-L bridges 85% of the gp between inlusive nd non-inlusive he hierrhies while TLH-L2 only bridges 45% of the gp between inlusive nd non-inlusive hes. Figure 5 lso shows the s-urve for the throughput improvement of TLH- L, TLH-L2, nd non-inlusive LLC for ll 05 two-ore worklod mixes. The s-urve is sorted bsed on non-inlusive he performne. In the figure, x represents TLH-L2 nd tringles represent TLH-L. The figure shows tht TLH-L losely trks the performne of non-inlusion for the bulk of the worklods. These results show tht the performne of non-inlusive he hierrhy n be hieved by llowing n inlusive LLC to be wre of the globl temporl lolity of lines tht re resident in the LLC. While TLH-L bridges the gp between inlusive nd non-inlusive hes, the number of LLC requests inrese by lmost 600X. In omprison, TLH-L2 inreses the number of requests to the LLC by bout 8X. We onduted sensitivity study on the frtion of hits tht n send hints to the LLC. When %, 2%, 0%, nd 20% of ore he hits in the L he generte TLHs, they bridge the gp between inlusive nd non-inlusive he hierrhies by 50%, 60%, 75%, nd 80% respetively. However, even when % of hits in the L he send TLHs, the number of LLC requests still inreses by 6X or more. Thus, while TLH-L n potentilly bridge the performne gp between inlusive nd non-inlusive he hierrhies, the high bndwidth demnds required by TLHs require n lternte low bndwidth tehnique to llow the LLC to derive or infer the globl temporl lolity. B. Erly Core Invlidtion Figure 6 presents the performne of ECI for the different worklod mixes. The figure shows tht ECI improves performne by more thn 7% for four out of the 2 worklods. The four worklods orrespond to the sme worklods tht benefit from TLHs, i.e. worklods where CCF pplition is onurrently exeuting with LLCT or Performne Reltive to Inlusion.3.2 TLH-IL TLH-DL TLH-L TLH-L2 TLH-L-L2 Performne Reltive to Inlusion.3.2 TLH-L TLH-L2 MIX_00 MIX_0 MIX_02 MIX_03 MIX_04 MIX_05 MIX_06 MIX_07 MIX_08 MIX_09 MIX_0 MIX_ All (05) Figure 5: Performne of Temporl Lolity Hints (TLHs). Worklod ID

8 Performne Reltive to Inlusion.3.2 ECI Performne Reltive to Inlusion.3.2 ECI MIX_00 MIX_0 MIX_02 MIX_03 MIX_04 MIX_05 MIX_06 MIX_07 MIX_08 MIX_09 MIX_0 MIX_ All (05) Figure 6: Performne of Erly Core Invlidtion (ECI). LLCF pplitions. On verge, ECI bridges 55% of the gp between inlusive nd non-inlusive hes. To illustrte this, Figure 6 lso presents the s-urve for ECI for ll 05 worklods. Aross ll worklods, ECI provides up to 30% improvement in performne with the worst se outlier losing only.6% performne. Unlike TLHs, ECIs re initited from the LLC to derive the temporl lolity of lines. The dditionl bk-invlidte requests required by ECI on verge is less thn 50% (in the worst se it doubles). While the inrese in bk-invlidtes might seem signifint, the extr messges introdued by ECI re reltive to the number of LLC misses. On verge, our studies show n LLC miss rte of 5 misses per 000 yles. ECI inreses the number of bk-invlidtes from 5 trnstions per 000 yles to 7 trnstions per 000 yles. Sine the number of trnstions is smll to begin with, the inrese in bk-invlidte requests from the LLC is negligible nd n esily be sustined by the ores. As our studies show, the inrese in bk snoop trffi from the LLC to the ores does not impt performne. The first order benefit of ECI is from voiding misses to min memory (Che misses re disussed in Setion D). C. Query Bsed Seletion Figure 7 shows the performne of QBS when pplied to different hes in the hierrhy. QBS-IL nd QBS-DL ensure tht he lines resident in the IL or DL hes re not evited by the LLC. QBS-L ensures tht he lines tht re resident in ny L he re not evited by the LLC. QBS- Worklod ID L2 ensures tht he lines resident only in the L2 re not evited by the LLC. Finlly, QBS-L-L2 ensures tht he lines resident in ny he in the hierrhy re not evited by the LLC. The figure shows tht QBS-IL onsistently performs better thn QBS-DL. Sine instrution he misses stll the front-end of the pipeline from supplying instrutions to the bk-end, on verge, it is more importnt to retin ode lines in the LLC until their temporl lolity is exhusted in the instrution he. On verge, QBS-IL improves performne by 2.7% nd QBS-DL improves performne by.6%. QBS-L is dditive of QBS-IL nd QBS-DL nd improves performne by 4.5%. Figure 7 lso shows tht QBS-L onsistently performs better thn QBS- L2 QBS-L2 improves performne on verge by.2%. Agin, this is beuse the L hes filter out the temporl lolity from the L2 hes. Finlly, Figure 7 shows tht QBS- L-L2 outperforms non-inlusion on verge. To illustrte this, Figure 7 lso presents the s-urve for QBS for ll 05 worklods. On verge, QBS improves performne by 6.5% while non-inlusion improves performne by 6.%. We lso evluted versions of QBS tht limit the number of queries tht the LLC n mke before seleting vitim. We evluted query limits of, 2, 4, nd 8 for QBS nd observed 6.2%, 6.5%, 6.6% nd 6.6% performne improvements respetively. The bseline inlusive he hierrhy n be thought of hving 0 queries. Performne does not signifintly improve beyond two queries beuse the totl pity of the on-hip hes is equl to the pity Performne Reltive to Inlusion.3 QBS-IL QBS-DL QBS-L QBS-L2 QBS-L-L2.2 MIX_00 MIX_0 MIX_02 MIX_03 Figure 7: Performne of Query Bsed Seletion (QBS). MIX_04 MIX_05 MIX_06 MIX_07 MIX_08 MIX_09 MIX_0 MIX_ All (05) Performne Reltive to Inlusion.3.2 QBS Worklod ID

9 % Fewer LLC Misses Reltive to Inlusion MIX_00 TLH-L TLH-L2 ECI QBS Exlusive LLC MIX_0 MIX_02 Figure 8: Che Performne Reltive to Inlusion. MIX_03 MIX_04 of two wys of the LLC. On verge, sending only one query is suffiient to hieve the bulk of the performne benefits. When QBS only sends one query, like ECI, our studies showed 50% inrese in bk-invlidte requests from the LLC to the ore hes. As we disussed for ECI, the 50% inrese in bk-invlidte requests is negligible beuse the trffi is smll to begin with nd the inrese in bkinvlidte requests from the LLC to the ores n be esily sustined. Our studies show tht the inrese in trffi from the LLC to the ores does not impt performne. Like ECI, the first order benefit of QBS is from voiding LLC misses. D. Effet of TLA Mehnisms on Che Misses Sine inlusion vitims require re-fething hot lines from memory, we use LLC miss redution s the metri to mesure the goodness of the proposed TLA poliies. While redution in L he misses n lso be ompred, they re not pplible to ECI or L2 TLH beuse they both evit lines from the L he. However, they both void requests to memory by preserving hot lines in the LLC. Thus, we present redution in LLC misses sine this metri is pplible to ll the TLA poliies. We verified tht the TLA proposls redue L he misses where pplible. Figure 8 presents the redution in LLC misses for the different TLA poliies, non-inlusive he, nd exlusive he when ompred to the bseline inlusive he. The br lbeled All represents the verge redution in LLC misses for ll 05 worklod mixes. On verge, L TLH redues LLC MIX_05 MIX_06 MIX_07 MIX_08 MIX_09 MIX_0 MIX_ ALL(05) % Fewer LLC Misses Reltive to Inlusion QBS Worklod ID misses by 8.2%, L2 TLH by 4.8%, ECI by 6.5%, QBS by 9.6%, non-inlusive hes by 9.3%, nd exlusive hes by 8.2%. On verge, QBS performs better thn ECI beuse QBS elimintes the time window problem ssoited with ECI by updting the replement stte instntly insted of witing for the line to be resued by the ore 6. This llows QBS to perform similr to non-inlusive hes without the omplexity of non-inlusion. Figure 8 lso presents n s-urve tht ompres redution in he misses ross ll 05 worklod mixes for QBS. Like non-inlusion, QBS redues he misses by s muh s 80%. These results gin emphsize tht non-inlusive hes primrily ddress inlusion vitims nd not extr pity. Only exlusive hes tke dvntge of the extr he pity. E. Summry of All TLA Poliies Figure 9 summrizes the performne of the TLA mehnisms proposed in this pper ompred to the bseline inlusive he. The figure shows tht QBS performs similr to non-inlusive he hierrhy. To ensure tht the performne benefits of the TLA mehnisms re only from reduing inlusion vitims, we lso evluted the performne 6. A version of QBS tht bk-invlidtes lines from the ore hes but updtes the LLC replement ws lso evluted. This modified QBS version is similr to ECI in tht lines re evited from the ore hes. We find tht the modified QBS poliy performs similr to the proposed QBS mehnism. This implies tht the bulk of QBS benefits re from voiding memory lteny (not LLC hit penlty). Performne Reltive to Inlusion.3.2 TLH-L TLH-L2 ECI QBS Performne Reltive to Non-Inlusion.3.2 TLH-L TLH-L2 ECI QBS Exlusive LLC MIX_00 MIX_0 MIX_02 MIX_03 MIX_04 MIX_05 MIX_06 MIX_07 MIX_08 MIX_09 MIX_0 MIX_ All (05) MIX_00 MIX_0 MIX_02 MIX_03 MIX_04 MIX_05 MIX_06 MIX_07 MIX_08 MIX_09 MIX_0 MIX_ geomen () Bseline Inlusive Che (b) Bseline Non-Inlusive Che Figure 9: Performne of TLA Che Mngement Poliies In the Presene nd Absene of Inlusion.

10 Performne Reltive to Inlusion TLH-L TLH-L2 ECI QBS Exlusive LLC :2 :4 :8 :6 Rtio of Core Che Size to LLC Size Figure 0: Slbility of TLA Poliies to Different Che Rtios. impt of these mehnisms on non-inlusive he (where inlusion vitims do not exist). If there were some other benefits besides reduing inlusion vitims, we would expet benefits from TLA mehnisms on non-inlusive he. Figure 9b summrizes the performne of the TLA mehnisms normlized to the performne of non-inlusive LLC. The results show tht TLA he mngement poliies improve the performne of non-inlusive he by only 0.4.2%. These results show tht the TLA mehnisms provide insignifint improvement in performne for noninlusive he. However, they signifintly improve inlusive he performne. Thus, we onlude tht the first order benefits of non-inlusive he re primrily from voiding inlusion vitims. Finlly, the figure shows tht on verge, n exlusive he hierrhy hs 2.5% better performne thn non-inlusive he hierrhy. This shows tht eliminting inlusion vitims n signifintly bridge the performne gp between inlusive nd exlusive he hierrhies. One inlusion vitims re eliminted, the remining gp between inlusive nd exlusive hes is primrily due to the extr pity in the hierrhy. F. Slbility to Different Che Rtios Figure 0 presents the slbility of the proposed TLA mehnisms for different ore he to LLC rtios. The figure shows the performne of the TLA mehnisms for MB, 2MB, 4MB, nd 8MB LLC. The L2 to LLC rtios for these he sizes re :2, :4, :8, nd :6. In generl, reduing the LLC size while keeping the ore hes onstnt requires better LLC mngement. When the LLC is not signifintly lrger thn the ore hes, exlusive or non-inlusive hes signifintly improve performne ompred to the bseline inlusive hes. Both TLH-L nd QBS signifintly bridge the gp between inlusive nd non-inlusive hes. We find tht TLH-L does not perform s well s QBS for the :2 rtio beuse hot lines servied by the L2 he were suffering from inlusion vitims. We found tht TLH-L-L2 performs similr to QBS for this onfigurtion. Unlike TLHs (whih re imprtil due to the required mount of bndwidth), we find tht low bndwidth solution suh s QBS mthes noninlusive he performne for the different he rtios. G. Slbility to Different Number of Cores Figure presents the performne of QBS when the totl number of ores in the CMP re inresed. We reted Performne Reltive to Inlusion QBS (4-ore CMP) QBS (8-ore CMP) Worklod ID Figure : QBS Performne with Inresing Core Count. ore nd 8-ore worklod mixes nd evluted QBS performne on CMP where the rtio of the ore he to LLC is mintined t :4. The 4-ore CMP hs 4MB LLC while the 8-ore CMP hs n 8MB LLC. The results show tht QBS improves performne of 4-ore CMP with n inlusive he by 8.% on verge. Non-inlusive nd exlusive he hierrhies for the 4-ore CMP improve performne by 8.3% nd.4% respetively. Similrly, for n 8-ore CMP, QBS improves verge performne by 9.5% while non-inlusive nd exlusive LLCs improve performne by 0.% nd 3.6% respetively. These results show tht QBS sles with inresing number of ores nd onsistently bridges more thn 95% of the performne gp between inlusive nd non-inlusive he hierrhies. H. Hrdwre Overhed The proposed TLA poliies require very little hrdwre overhed. They ll use existing dt pths in the system to llow the LLC to beome wre of the temporl lolity of lines in the ore hes. TLH requires new messge lss tht does not return dt but merely updtes the LLC replement stte. Sine TLH is not slble due to lrge bndwidth requirements, TLHs re not prtil solution espeilly sine ECI nd QBS serve s lterntive low bndwidth solutions. ECI requires dditionl logi to preselet the next vitim in the he nd invlidte it from the ore hes. Similrly, QBS uses the existing dt pth to determine whether or not he line n be evited from the LLC. The dditionl logi required is stte-mhine to send queries to the ore hes nd responses to the LLC. As the number of HW-threds on-hip ontinue to inrese, both QBS nd ECI re slble solutions tht re effetive for ny inlusive he hierrhy orgniztion. VI. RELATED WORK Mnging multi-level he hierrhies hs been n importnt reserh re for severl dedes [26, 24,, 7, 8]. Ber et l first introdued the onept of multi-level inlusive he hierrhies for simplifying he oherene [6]. Jouppi et l proposed exlusive hes to redue LLC onflit misses nd to lso inrese the effetive he pity of the hierrhy by not repliting lines in the LLC. Exlusive hes perform best when the sum of the ore hes is not signifintly lrger thn the size of the LLC [7, ]. Non-inlusive he hierrhies hve been proposed in the ontext of ggressive

11 prefething [0, 28] nd in the ontext of better he mngement poliies [28, 2]. Zhrn [27, 28] proposed the use of globl replement to improve the performne of inlusion. The proposed replement poliies were only evluted for single-threded worklods running in isoltion. The results showed tht while globl replement redued the number of inlusion vitims, the observed performne benefits were negligible. Our studies replited their results for single-threded worklods, however, our results indite tht globl temporl lolity indeed benefits CMPs. Grde et l [3] followed up on Zhrn s work nd deonstruted globl replement for single-ore nd multi-ore proessors. They evluted the potentil for globl replement in inlusive hes by nlyzing the miss strem of non-inlusive LLC. Speifilly, for their non-inlusive LLC, they mesured the number of times n LLC vitim ws resident in the ore hes upon evition (i.e. potentil inlusion vitim). The problem with there pproh is tht the potentil inlusion vitim ours only one for hot line beuse suh hot line ontinues to reeive hits in the ore hes nd is never re-filled in the LLC. The potentil for globl replement should hve been mesured in the presene of n inlusive he hierrhy. Flether et l [2] observed inlusion vitims in the ontext of diret mpped network hes nd proposed three solutions to ddress the problem. The proposed solutions inlude inresing the he ssoitivity, vitim he [8], or mking the LLC non-inlusive nd using snoop filter (lled tg he in the pper) to ese he oherene. They showed tht inresing the network he ssoitivity nd vitim hes redue the negtive effets of inlusion vitims. The proposed mehnisms require dditionl hrdwre strutures (e.g. vitim he nd snoop filter). Our work differs from the work of Flether et l in tht we do not require ny dditionl hrdwre strutures. The Erly Core Invlidtion (ECI) proposl effetively uses n in-llc vitim he insted of n externl vitim he. For our bseline system, we ompred the performne of both ECI nd QBS to n inlusive LLC bked by 32-entry vitim he. We found tht the 32-entry vitim he improves verge performne by only 0.8% while ECI nd QBS improves verge performne by 4.5% nd 6.5% respetively. There hs lso been extensive reserh on mnging shred hes in CMPs [4, 5, 25]. Most of the prior reserh work fouses on how to effiiently prtition the shred lstlevel he of CMP. When multiple pplitions ompete for the shred LLC, the proposed poliies dynmilly provide more he to pplitions tht benefit from the LLC nd less to pplitions tht do not. However, the proposed poliies only trget he prtitioning nd do not ddress the problem of inlusion vitims. We ompred the TLA poliies in the presene of intelligent he mngement poliies [4, 5] nd find tht we hieve similr performne improvements. VII. SUMMARY Inlusive hes re desirble beuse they simplify he oherene. However, inlusive hes limit performne due to inlusion vitims. In ontrst, non-inlusive hes eliminte inlusion vitims but ome t the expense of inresing oherene omplexity. This pper improves inlusive he performne by mking these ontributions:. We show tht better mnged inlusive he provides equl or better performne thn non-inlusive he. Speifilly, we show tht the bulk of non-inlusive (nd exlusive) he performne over inlusive hes is due to voiding inlusion vitims nd not the extr hing pity in the hierrhy. Inlusion vitims our beuse n inlusive LLC is unwre of the temporl lolity of hot lines in the ore hes. 2. We propose Temporl Lolity Hints (TLH) s mehnism to onvey the temporl lolity of lines referened in the ore hes to the LLC. We show tht TLHs sent by the L he signifintly redue the number of inlusion vitims. TLHs, however, signifintly inrese on-hip bndwidth. 3. We propose Erly Core Invlidtion (ECI) s low bndwidth tehnique used by the LLC to derive line s temporl lolity. Unlike TLHs, ECI is only triggered on LLC misses. ECI selets the next potentil LLC vitim nd invlidtes tht line from the ore hes while retining it in the LLC. ECI employs the heuristi tht if the next potentil LLC vitim line is hot, it will be servied by the LLC (insted of the ore hes) on the subsequent ess using n LLC replement stte updte. However, ECI limits performne when the subsequent ess ours fter miss (to the sme set). 4. We propose Query Bsed Seletion (QBS) s n lterntive to ECI. QBS employs the heuristi tht the ore hes re best suited to inform the LLC on the temporl lolity of line. In the pper, we show tht lines resident in the ore hes hve high temporl lolity nd should not be evited from the LLC. In doing so, QBS llows inlusive hes to pproh the performne of non-inlusive hes. For our bseline 2-ore CMP, we show tht TLHs issued from the L he bridge 85% of the gp between inlusive nd non-inlusive hes. ECI bridges 55% of the gp nd QBS performs similr to non-inlusive hes. Our studies show tht when non-inlusion must be enfored due to trget ore he to LLC size rtios, (e.g. :2 or :4), the QBS tehnique llows hip designers to mintin the oherene benefits of inlusion while performing similr to non-inlusive he. This is espeilly importnt sine hip designers tody re willing to srifie oherene benefits for the performne improvement of non-inlusive LLCs []. Using QBS, we report 0-33% performne improvement for 25 of the 05 worklods on 2, 4 nd 8 ore systems with :4 hierrhy nd 6% performne improvement on systems with :2 hierrhy. As the number of ores shring n LLC inreses, he ontention inreses nd ddressing inlusion vitims beomes growing onern. ECI nd QBS re slble nd perform better with inresing ore ounts. Both introdue extr messges into the system tht do not require signifint dditionl bndwidth or dditionl hrdwre strutures.