Missing items. Sp eci al Sessi on f or EECCA an d SEE cou n t r i es. Geneva, Switzerland. Randi Johannessen, Statistics Norway.

Transcription

1 Missing items Sp eci al Sessi on f or EECCA an d SEE cou n t r i es Geneva, Switzerland Randi Johannessen, Statistics Norway raj@ssb.no

2 Missing items -roadmap Pr oblem Reason Opt ion Method Tar get Missing items Tem porar y Seasonal Omit/ Estimate prices Permanent Estimate prices Replacement Omit, carry forward, imputation Carry forward, imputation Find a substitute, quality adjustment Sampling issues Replace the shop, Follow the new location Balanced number of prices in compilation Shop closed Price collection Central office 2

3 Temporary missing items A missing item (price observation) that will be available in (near) future Item is a good or service that consumers purchase in a specific shop How to handle: Omit the specific item in current and the price reference period Carry forward the last observed price Impute Overall or class mean HICP regulation; impute first and second month, find a replacement the third month. 3

4 Imputation Overall imputation Use the average price change of all other similar items (price observations) in the same period and in the same geographic area If few price observation within the geographic area use the whole country Mainly food and beverage items Class mean Use the price changes for only those varieties of the missing item that are assumed to be similar Mainly non-food items 4

5 Overall imputation Item: wheat bread Month t Month t- 1 Shop A 2 2 Shop B 2,5 2,5 Shop C 2 1,5 Shop D Missing 2 Imp. price, Shop D 2,1666 Average 2,

6 Class mean imputation Item: men's shirt Month t Month t- 1 Shop A Shop B Shop C Shop D Shop E Shop F Shop G Missing 15 Imp. Price, Shop G 15,9375 Average, shop A, C, E

7 Seasonality in a CPI The prices of some items vary through the year, often following a seasonal pattern Consumers buy items in varying quantities, often following a seasonal pattern Items not available for purchase in shops (or only in negligible quantities) in certain months of the year, in a typical seasonal pattern Sometimes called strong seasonal items Examples: Vegetables, fruits, fish, clothing, footwear 7

8 Treatment (strong) seasonal items Fixed weights approach The weights of the items remain fixed during the year Carry forward or impute non available prices Variable weights approach The weights of the items vary during the year within a specific COICOP group The seasonal item will have zero weight off-season 8

9 Estimated prices -seasonal items Carry forward the last observed price until product reappear Bring price back to normal and carry forward until the product reappear Bring price back to normal and impute the missing price until the product reappears This is the treatment in the HICP regulation 9

10 Carry forward last observed price Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan In season price Carry forward price 10

11 Bring back to normal? The price in the last month of the season may be considered too low, if products are on sale E.g. summer and winter clothing The last price before the sales period may be considered a normal price The average observed price in the past season may be considered a normal price E.g. if prices are volatile also within the season (fruits, vegetables) 11

12 Carry forward average price Bring back to «normal» 1. month (e.g. average price) Change from average price Change from last observed price in season Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar In season price Imputed price Average price 12

13 Imputation Impute the prices of off-season items by using the price development of in-season items within the same COICOP group All-seasonal imputation Counter-seasonal imputation 13

14 Imputation based on items in season Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan In season price Imputation Price increase 14

15 All-seasonal imputation Use price development of all items Coicop 0116 Coicop Fr u it Fresh fruit Bananas Apples Pear s Gr apes Melon (seasonal item) Oranges 15

16 Counter seasonal imputation Use price development of seasonal items in-season Coicop 0312 Coicop Clothing Men's clothing Socks T-sh irt Sweater, cotton Su it Sh irt, lon g sle eves Shirt, short sleeves Je a n s Pyjamas Sh or t s Summer trousers (Seasonal item) 16

17 All-or counter seasonal imputation? Counter-seasonal useful for products groups that contain both seasonal and nonseasonal items Typically two seasons for the seasonal items E.g. summer and winter clothing All-seasonal may be preferred in other cases 17

18 Ppermanent missing items An item permanent disappear from the shop How to handle: Price collector should find a replacement (new item) that serves the same utility to consumers Identify any differences in quality between the old item and the replacement Quality adjustment if needed (central office) Calculating a new price reference price of the replacement (new item) Impute or simply carry back 18

19 Quality adjustment Direct com parison Assume no relevant quality difference between the old and new item Explicit The monetary value of the quality difference is determined directly This value is used to calculate what price the replacement would have in the price reference month Different approaches; hedonic regression, expert judgement, package size adjustment, option pricing Implicit The price difference between the old and new item is assumed be due to quality changes Different approaches; bridged overlap/other overlap methods, link-to-show no change Further reading: 19

20 20 Source: CENEX HICP Quality Adjustment

21 Shop closed or changed location If a shop goes out of business or refuses to participate it should be replaced with a same type of shop A shop changes location: should the price collector follow the new location? Is it within the same stratum? E.g. geographic area, type of shop Sampling size need to be considered Following the shop to the new location may be inconvenient 21

22 Fi nal remarks Price collectors should identify the real causes for missing prices Central office decide how to handle and methods Rejected prices: In general, the prices reported by the price collectors should be accepted Correction of prices, e.g. unusually high or low price change requires specific information Rejected price due to validation procedures use the rules for missing items Totally new items requires update of the basket (re-sampling) 22