“Gold Standard” Study of Workforce Programs Shows Interim Impacts
How well have workforce programs served their customers? The U.S. Department of Labor’s Employment and Training Administration (ETA) is trying to answer this question by overseeing one of the most rigorous studies ever done of federally-supported workforce programs.
ETA recently released 13 publications summarizing interim findings of the “Workforce Investment Act (WIA) Adult and Dislocated Worker Programs Gold Standard Evaluation,” including an impact report on whether different types of services improved employment outcomes. Although the Workforce Innovation and Opportunity Act (WIOA) has since replaced WIA, the evaluation still has relevance for the current workforce system, which has maintained the same basic service structure.
The interim report assessed program impacts 15 months after enrollment — when many participants in training were still taking their courses — so it does not draw conclusions about the impact of training. But the report suggests that other WIOA services may have positive impacts, including:
- The availability of WIA-related training funds increased the percentage of customers who entered training programs.
- Participants receiving intensive services (e.g. career counseling, in-depth skills assessments) made more money and worked more hours than those receiving only core services (e.g. online tools, brief assistance from staff).
- Customers with access to training and intensive counseling were significantly more likely to choose an occupational training program and much less likely to choose an education degree-type program when compared to the group that only received core services.
- Estimates of program impacts were similar for adults and dislocated workers.
To make the evaluation’s results more generalizable and meet the goal of providing national estimates, analysts randomly selected 28 local workforce investment areas in 19 states. In most instances, all eligible adults and dislocated workers in the local areas were randomly assigned to one of three research groups categorized by access to different levels of WIA-funded services: core services, but not with intensive or training services; core and intensive services, but not training; and all WIA services (or full WIA) – with core, intensive, and training services. Over 35,600 customers were assigned to the study.
The final report will draw from a 30-month evaluation period that should reveal more definitive results.
The evaluation used data from registration forms completed by customers who agreed to participate in the study, follow-up surveys of those customers, data collected by American Job Centers on service receipt, and financial data from local areas. For the 30-month results, researchers will use data on earnings, new hires, and Unemployment Insurance benefit receipt from the National Directory of New Hires.
The project has also been examining whether benefits from WIA intensive and training services exceeded their cost, and how WIA Adult and Dislocated Worker programs were implemented (such as the role of performance measures in service delivery, how local areas spent formula funding, and how they used sector strategies to narrow skills gaps). All WIA Gold Standard Evaluation publications are available on the Mathematica Policy Research website. ETA plans to release its final report on the evaluation in early 2018.