Modifications to the Job Openings and Labor Turnover Survey

Transcription

1 Modifications to the Job Openings and Labor Turnover Survey Charlotte Oslund Bureau of Labor Statistics 2 Massachusetts Avenue, NE, Suite 4840 Washington, D.C Abstract The Job Openings and Labor Turnover Survey (JOLTS) at the Bureau of Labor Statistics produces monthly estimates for job openings, hires, and separations. The estimates are published by industry, by region, and for the private sector and government. The goal of the JOLTS data is to measure current labor market dynamics as well as trends over time. The data series, with 11 years of data covering two recessions, exhibit clear trends and demonstrate economic concepts, including leading indicator properties for turns in the economy. During the past 10 years, the JOLTS program has graduated from a brand new program to an established program entering a period of evaluation and change. This paper will discuss improvements that have been implemented, including a birth/death model, a quality control process unique to the JOLTS hires and separations data, quarterly birth samples, a new data collection system, and new electronic modes of collection. Key Words: job vacancies, job openings, hires, separations, turnover, labor statistics, labor dynamics, birth samples, economic trends 1. Introduction Development of the Job Openings and Labor Turnover Survey began in 1999 and data collection began in Since the beginning of the program over a decade ago, many changes have been made to the JOLTS survey, including methodology changes, additional published data, and new data collection modes. This paper explains the original survey design of JOLTS and the changes implemented over time. The hope is that other surveys can learn from the JOLTS experience and also share with JOLTS what they have learned in their survey work. 2. Basics of JOLTS JOLTS is a monthly panel survey with a sample size of approximately 16,400 private business and government establishments 1. The nationwide sample covers nonfarm businesses of all sizes in the 50 states and the District of Columbia. Excluded from the scope are agricultural establishments, private households, and religious organizations. The data items JOLTS collects are: employment, job openings, hires, quits, layoffs and discharges, other separations, and total separations. The job openings data element is a stock measure capturing the number of jobs open 1 An establishment is typically a single physical location whereas a firm includes multiple locations of the business.

2 on the last business day of the reference month. The hires and separations are flow measures capturing the number of hires and separations that occurred throughout the full reference month. Capturing data on voluntary and involuntary separations is essential for the economic use of the data. Quits are generally voluntary separations initiated by the employee and layoffs and discharges are generally involuntary separations initiated by the employer. Other separations include retirements, separations due to disability, and transfers to other locations of the same business. Since these other separations are not as easily designated as voluntary or involuntary, they are counted in their own category. Employment is not published by JOLTS since the official BLS employment estimate is from the Current Employment Statistics (CES) program. JOLTS uses employment to ensure that data are collected for the correct location and also to interpret the other JOLTS data elements relative to employment during data collection and data review. Total separations is calculated as the sum of quits, layoffs and discharges, and other separations if the respondent provides all three data items. If the separations breakouts are not available, the respondent may report one figure for total separations, which is allocated to the separation breakouts during estimation. Published estimates include rates and levels for all the JOLTS data elements by industry, region, and ownership (private sector and government). Seasonally adjusted data for job openings, hires, quits, and total separations is published for many, but not all, of the industries. Seasonally adjusted layoffs and discharges estimates are available by ownership (total nonfarm, private, and government) and by region, but not by industry at this time. Other separations seasonally adjusted estimates are only available by ownership at this time. The JOLTS data series begins with December The JOLTS program is managed from the national BLS office in Washington, D.C., and the data are collected by contract interviewers in one data collection center based in Atlanta, Georgia. The data collection center is managed by federal government staff. 3. Original JOLTS Survey Design The original survey design for JOLTS was a typical design with frame development, sample selection, unit and item nonresponse adjustment, point estimation, and variance estimation. Two not-so-typical steps were also included: ratio-adjustment of JOLTS employment to CES employment and the resulting annual readjusted when CES benchmarks to the frame. Much of the original sample design remains the same today with a few parts slightly modified and with many new steps added. 3.1 Sampling Frame The JOLTS sampling frame is the BLS Quarterly Census of Employment and Wages, which combines the State Unemployment Insurance (UI) files with the Unemployment Compensation for Federal Employees (UCFE) information. The CES program adds permanent random numbers, which reduces the overlap between the JOLTS sample and other BLS establishment survey samples. As a final step, JOLTS adds a frame of railroad establishments obtained from the BLS Occupational Employment Survey. The final result is the JOLTS sampling frame.

3 3.2 Sample Selection The sample is a stratified random sample from the frame sorted by ownership, region, industry, and size (employment). Originally, the sample was parsed into 18 noncertainty panels so that each establishment was in the sample for 18 months. Later, the time in sample was lengthened to 24 months to save money, as is discussed later in this paper. One certainty panel is selected for establishments with over 5,000 employees or with a substantial portion of the cell s employment. The certainty units stay in the survey continually unless the employment declines below the cutoff or no longer comprises a large enough portion of the cell employment. The original six employment size classes remain today: 1-9; 10-49; ; ; 1,000-4,999; 5, Nonresponse Adjustment As many surveys do, JOLTS chose to account for unit nonresponse through weight adjustments. The weight of the nonresponding units is transferred to the responding units in the sampling cell. The nonresponse adjustment factor (NRAF) is the new weight divided by the original weight and is a multiplicative factor used in point estimation. Item nonresponse is managed through nearest neighbor item imputation, another common approach. Both methods are still used in the current JOLTS estimation, although the item nonresponse has been slightly modified to cap the number of times a respondent can be used as a donor. 3.4 Ratio-Adjustment One somewhat unique step in the JOLTS estimation is the ratio-adjustment of the JOLTS weighted employment to the CES weighted employment. This adjustment allows JOLTS to benefit from the larger CES sample size (approximately 141,000 businesses and government agencies representing about 486,000 establishments). The ratio of the weighted employment values is the benchmark factor, which is applied to JOLTS job openings, hires, and separations during estimation. The goal is for the benchmark factor (BMF) to remain between 0.9 and 1.1 so that the JOLTS values are not adjusted by more than 10 percent up or down. Additionally, if no pattern of under- or over-estimation exists, the range of the benchmark factor should be symmetrical around a mean of 1.0. On average, over the last two years, the private sector BMF has ranged from 0.55 to 1.48 with a mean of 0.98, and the average government BMF has ranged from 0.90 to 1.50 with a mean of These ranges indicated JOLTS under-estimates employment more than it over-estimates, but on average, the BMF mean is near Point Estimation Once the NRAFs and BMFs have been determined, the point estimates for levels are calculated using a Horvitz-Thompson estimator: Where π i is the probability of selection and Y i is the response from the i th sample unit. Note that π i -1 is the weight of the i th sample unit. More simply, the estimate = SampledWeight*NRAF*BMF*DataValue. The levels are calculated at the industry level then summed to obtain supersector, ownership and total nonfarm estimates. The regional estimates are derived from the national total using ratios of employment by region from the CES employment figures. The rates are calculated from the levels. In all but job openings, the rate is the level divided by the CES employment for the

4 estimation cell, times 100. The interpretation is the percentage of employment that was hired/separated during the reference month. For job openings, the rate is the job openings level divided by the sum of the job openings level and employment, times 100. The interpretation of the job openings rate is the percentage of all jobs filled or unfilled that remain open on the last business day of the reference month. Use of the Horvitz-Thompson estimator has remained throughout the history of the JOLTS program. 3.6 Variance Estimation The variance methodology selected in the beginning and continuing to the present is the modified balanced repeated replication known as Bob Fay s method of balanced half samples. 3.7 Annual Benchmarking Due to the unique process of ratio-adjusting the JOLTS employment to the CES employment, JOLTS must retabulate estimates each year when CES benchmarks to the frame. Each year, the most recent two years of JOLTS data are retabulated and published in March. 4. Modifications to JOLTS over Time 4.1 Addressing the JOLTS/CES divergence In 2002, when JOLTS had just over a year s worth of estimates, it was discovered that the JOLTS implied employment change of hires minus separations was not tracking the CES employment change over time. The JOLTS implied employment growth was much larger than the CES growth. The difference between the surveys employment was termed the divergence. The problem was addressed by training the interviewers and the respondents on the concept of employment changes being supported by hires and separations data. Also, fields were added to the data collection system that displayed the monthly difference of [(hires separations) overthe-month employment change]. Also added to the data collection system was the cumulative difference that summed the differences across months so the interviewer could quickly see if the respondent regularly reported inaccurate data. Lastly, a coding system was developed for the interviewers to indicate the reason for the difference. Analysis of these codes reveals that about 14% of the time, the use of part-time or on-call workers was the reason for the employment change not equaling the difference in hires and separations in a particular month. A change in employment without hires and separations to match is a valid occurrence in JOLTS due to parttime and on-call workers who may not work the pay period but who are not separated. The above steps greatly lowered the divergence, but did not cure the problem. Further action was taken in 2009 and is discussed later in this paper. 4.2 Introduction of Median Standard Errors Although variances for the point estimates, over-the-month change, and over-the-year change had been calculated all along, they were highly variable. In 2004, JOLTS began providing median standard errors for the point estimates. To date, the median is recalculated annually at annual benchmark time using one year of standard errors. Median standard errors are available upon request, though very few data users request them. Median standard errors are just one way to address the issue of variable variances; JOLTS may look into other methods in the future. 4.3 Introduction of Outlier Treatment: Winsorization With a small sample size and resulting large weights, an outlier can have a very large effect on the JOLTS estimates. The outlier effect is especially strong in the layoffs and discharges estimates and the other separations estimates since reported values tend to be small and one large

5 value weighted up can vastly inflate the estimate. In 2005, the program instituted Winsorization in which extreme values are replaced with the highest acceptable value. For example, if a respondent reports employment of 217 and hires of 203, the hires rate is 94 percent. If the unit has a large weight and/or is used as a donor in imputation, the hires level may be unduly affected. If the cutoff rate for hires is 80 percent, then the hires value is cut to (80 percent). A small number of values are Winsorized each month and the resulting estimates are less volatile. 4.4 Introduction of Seasonal Adjustment In 2004, once JOLTS had three years of data, seasonal adjustment procedures began. Ideally, seasonal adjustment would have five years of data, but data users prefer adjusted numbers. JOLTS chose concurrent seasonal adjustment using the X-12 ARIMA software 2. The same method is used today but with several changes that are discussed later. At 10 years of data, new diagnostics were utilized, including sliding spans and revision history. 4.5 Modifications to Sampling Pressure to contain data collection costs in 2006 lead JOLTS to look for ways to reduce workload in the data collection center. One way was to extend the time in sample for each unit from 18 months to 24 months. The 6-month extension balanced the program costs with respondent burden and potential for increased refusals. A major expense is the interviewers time to locate new sample units and convince them to report data. Once the respondent begins providing data, the collection is inexpensive relative to the enrollment process. The extended sample time cut the number of new sample units to be enrolled each month from about 800 to 600. The time in sample has remained at 24 months since Removal of Units no Longer in Certainty Status For the first five years of the survey, new certainty units were added to the sample but never removed if they fell below the employment cutoff. The number of certainties grew to over 1,600 and took more and more data collection time. Beginning in 2006, the annual sample dropped certainties that no longer met the employment requirements. The first year this strategy was used, 400 of the 1,600 certainty units were dropped. 4.7 New Data Collection System As technology evolved and BLS management encouraged system consolidation, a new data collection system was designed to serve both the JOLTS and CES programs. Implemented in 2008, the new system uses an Oracle database with a Windows interface, both BLS standards. The result is a more user friendly system that is more easily modified when needed. Previously, JOLTS used the Blaise software 3, which served the program s needs but BLS had no programmers trained in the language. The new system has been well-received by the interviewers and the national office staff alike and is still in use today. 4.8 Divergence Round 2: The Alignment Procedure The training and additional codes and calculations implemented in 2002 helped reduce the CES/JOLTS divergence problem, but not enough. Further investigation identified the problem to be under-reporting of separations in JOLTS, especially in the temporary help industry and in state education units (specifically colleges and universities). It seemed that accurate hires data were 2 The X-12 ARIMA seasonal adjustment software is produced, distributed, and maintained by the U.S. Census Bureau. 3 The Blaise software was designed by Statistics Netherlands and is used to design computer-assisted questionnaires and to process survey data.

6 available but accurate separations data were much more difficult for respondents to provide. Temporary help agencies didn t have clear records of when an employee no longer wanted temporary placements, and colleges and universities have trouble tracking student workers and adjunct professors. After extensive research, JOLTS developed a procedure in which the JOLTS data are aligned to better match the CES employment change over time. The procedure was dubbed alignment. The steps include seasonally adjusting the JOLTS data and comparing the JOLTS implied employment change (hires minus separations) to the CES seasonally adjusted over-the-month employment change. By seasonally adjusting the data first, the JOLTS seasonal pattern, which differs from CES, is preserved. The hires and separations are then adjusted to better match the CES employment change. Assuming that establishments with additional hires also have additional job openings, the job openings level is increased by a portion of the hires increase if an upward adjustment occurs for hires. To preserve the relative contribution of hires and separations to total churn (hires + separations), both hires and separations are adjusted for the divergence. If the JOLTS implied employment change exceeds the CES employment change, separations are added to JOLTS and hires are subtracted. Likewise, if the JOLTS implied employment change is less than the CES employment change, separations are subtracted and hires are added. Once the adjustments have been made to the levels, the data is returned to nonseasonally adjusted status by dividing out the seasonal adjustment factors. The resulting estimates are called nonseasonally adjusted aligned (NSA-Aligned) and those are the published NSA estimates. The series is then re-seasonally adjusted to produce the seasonally adjusted estimates. In addition to the alignment procedure, JOLTS staff also began hand reviewing data for all responding temporary help and state education establishments. Separations, and sometimes hires, are added to the microdata as needed before the alignment process. Units with data problems are identified and recontacted for improved data (usually separations). If the respondent cannot provide more accurate data, the problem data is no longer collected. Figure 1 below illustrates the cumulative divergence for 2007 and 2008 before and after the alignment procedure was implemented. 4.9 Introduction of Quarterly Birth Samples As researchers studied the JOLTS data, they began to suspect that the estimates were under counting hires and separations. Specifically, the newest establishments ( births ) which post high openings rates and hire rapidly, along with units less than 1 year old, were not in the JOLTS sample due to the length of time to construct a frame and select a sample that is inherent in sample survey work. To capture more of these quickly growing new businesses, JOLTS began sampling births on a quarterly basis in Any new unit on the frame is eligible to be sampled. Sampled birth units are immediately added to the data collection system for enrollment.

7 2500 Figure 1: Cumulative Divergence, Total Nonfarm (JOLTS Hires - Seps) - CES EmpChange Before alignment Dec. Jan. Feb. Mar. Apr. May June July Aug. Sept. Oct. Nov. Dec. Month After alignment 2007 After Alignment 2008 After Alignment 2007 Before Alignment 2008 Before Alignment 4.10 Introduction of a Birth/Death Model Between annual samples and even between quarterly birth samples, businesses are created and destroyed. In 2009, JOLTS also implemented a birth/death model to model the hires of brand new establishments and to model the separations of units going out of business ( dying ) between the annual samples. These birth and death establishments are thought to have higher rates of hires and separations than other older establishments. The births and deaths are modeled by adding the JOLTS churn (hires and separations) to the QCEW quarterly employment change. When estimating current months for which QCEW data are not yet available, the births and deaths are modeled by applying the CES over-the-year employment change to the available QCEW to model the employment change for the missing quarters. The modeled births and deaths are then added to the original hires and separations estimates to produce the final estimates of hires and separations. Similar to adding hires and separations during alignment, when hires are added by the birth/death model, a proportional number of job openings are also added. The addition of the birth/death model resulted in a notable increase in the job openings, hires, and separations estimates. The model remains in use today with a few updates discussed later in this paper Enrolling Low-Age Units First A final change in 2009 to further capture new businesses was combining the lowest age (least time in business) units in the annual sample into one panel and placing that panel in data collection before the panels of older units. By rolling the youngest units into collection first, the odds increase for capturing their data during their expansion years. For the past three years, this practice has remained Introduction of Peak/Trough Analysis Not all new products are specifically geared toward data users. In the busy year of 2009, JOLTS began statistical peak/trough analysis. The output provided a welcome statistical tool to help analyze the data when writing the press release. Based on the points of inflection (peaks and troughs), the JOLTS data appeared to exhibit leading economic indicator properties approaching the Great Recession of When compared to employment, job openings, hires, and quits

8 Hires and Separations Job Openings all peaked and began falling well before the recession officially began. Additionally, the troughs of the JOLTS data elements corresponded with the end of the recession and turned upward before employment. More recessions will determine whether the JOLTS data can accurately signal a coming turn in the economy. Figures 3 and 4 below illustrate the turning points of JOLTS and CES. 5,500 Figure 3. Job openings and employment Seasonally adjusted, in thousands 140,000 5,000 4,500 Employment 138, ,000 4,000 3,500 3,000 2,500 2,000 Job Openings 134, , , , ,000 Employment 6,000 5,500 Figure 4. Hires, total separations, and employment Seasonally adjusted, in thousands 140, ,000 5,000 4,500 4,000 3,500 Employment Hires Total Separations 136, , , ,000 Employment 3, ,000 Hires Total separations Employment 4.13 New Modes of Collection The data collection methods available at the beginning of the survey included computer assisted telephone interviewing (CATI), fax, and touchtone data entry (TDE). In 2004, JOLTS added

9 collection and in 2010 added web collection. Reporting data on-line became popular with respondents very quickly and web collection soon brought in over 60 percent of the collected data. With the web response rates higher than the other collection modes, the overall JOLTS response rates soon began trending upwards from 64 percent in August 2010 to 70 percent in May Figure 5 below provides the response rates and proportion of data for each collection method as of April Note that each new sample unit is required to report via CATI for the first six months before rolling to self-reporting. Further, respondents who do not remember to report data on their own are moved back to CATI. Therefore, the CATI response rate is not comparable to the response rates for the other modes of collection. Figure 5: Response rates by collection method Collection Number of Units Number of Units Percent of Response Rate Methods Assigned Collected Collected Data CATI 7,744 2,858 26% 41% Web 7,780 7,053 64% 92% TDE % 84% % 85% Fax % 72% US Mail 5 2 1% 100% Total 16,932 11, % 69% 4.14 Modifications to the Birth/Death Model The birth/death model introduced in 2009 was improved in a few ways in First, it better tracked UI breakouts and consolidations of businesses, which are due to reorganizations or changes in data reporting. The improved tracking helped differentiate true births and deaths from reorganizations. Second, the model parameters were updated to reflect the 2007 recession. Lastly, the government sector was dropped from the model since true births in government units are extremely rare; changes are most likely due to reorganization. The effect of the updated birth/death model was to slightly lower job opening, hires, and separations estimates. 5. New Products Produced Along the Way In addition to the many changes throughout the JOLTS history, new data products have been regularly introduced. In 2006, annual estimates were introduced for hires and separations (not job openings since they are a stock measure and cannot be accumulated across months). New seasonally adjusted industries were released in 2007, 2009, and Seasonally adjusted layoffs and discharges by ownership were begun in 2009 with regions arriving in 2012 along with other separations by ownership. Every year, in preparation for the annual benchmark time, the seasonal adjustment diagnostics are reviewed to see if any new seasonally adjusted series can be published. In 2010, publication was expanded from just the news release to include supplemental materials. The graph and bullet-point format allows the JOLTS staff to combine JOLTS data with other data series and analyze the time series, neither of which is done in the news release since it focuses on current month JOLTS data. These new graphs illustrate the uses of JOLTS data and help the user understand the data better. The supplemental graphs have been popular with data users and are

10 updated on the JOLTS website each month on the news release day. These materials are posted under the title, JOLTS Graphs and Highlights on the JOLTS website. Looking for products beyond ownership, industry, and regional data, JOLTS began producing experimental estimates by establishment size class in The estimates have since been available upon request. The establishment-based size class estimates were possible without any methodology changes or further data collection since the current sample design stratifies by establishment size. Data users soon began requesting size class estimates by size of firm (rather than by size of establishment), the research for which is currently underway. One challenge has been that without CES employment by size of firm, JOLTS does not have targets for the ratioadjustment step of estimation. Instead, the distribution of firms by size class on the QCEW is applied to the current month CES estimates to derive a total for ratio-adjustment. The goal is to begin releasing size of firm data in In the pursuit of new data products, JOLTS contracted out an employer records check project, which tried to determine what other types of data employers could report to JOLTS. Of particular interest was if open jobs were due to growth or to changing structure of the workplace, whether open jobs were difficult to fill, and whether businesses encourage turnover as a way of holding down payroll costs. Whereas the contacted businesses kept plentiful records on wages, fewer than 25 percent had data on the length of time to fill a vacancy or the reason for not filling a vacancy. Not surprisingly, less than half the businesses tracked data on temporary help or on-call workers, an especially difficult segment of workers for which to capture accurate data. Future products for JOLTS could include more seasonally adjusted data (more industries, finer breakouts of industries), finer geographic detail, industry data by region, length of time to fill vacancies, reasons for not filling vacancies, and information on job openings and hires with regards to part-time or full-time or other characteristics. Several of these products are not feasible with the current sample design or sample size, but some are possible and are being studied. 6. Conclusion In just a short time, the JOLTS survey was developed and initiated and soon began publishing. That was just the beginning. Over the next decade-plus, many changes were made to revise methodology, introduce new methodology and processes, utilize advances in technology, and produce new data products. More changes will undoubtedly be made as the program continues through its second decade. References William G. Cochran, Sampling Techniques, 3 rd Edition (John Wiley & Sons, 1977) C.S. Dippo, R.E. Fay, D.H. Morgenstein, Computing Variances from Complex Samples with Replicate Weights, ASA Proceedings (1984): Steven J. Davis, R. Jason Faberman, John C. Haltiwanger, and Ian Rucker, Adjusted Estimates of Worker Flows and Job Openings in JOLTS, in Labor in the New Economy, ed. Katharine Abraham, Michael Harper, James Spletzer (Chicago: University of Chicago Press 2010),