Before President Obama signed the Workforce Innovation and Opportunity Act (WIOA) into law yesterday, he announced:
So the bill I’m about to sign will give communities more certainty to invest in job-training programs for the long run. It will help us bring those programs into the 21st century by building on what we know works based on evidence, based on tracking what actually delivers on behalf of folks who enroll in these programs.
To ensure that $15 billion of what the federal government spends on training programs each year is used effectively, the Obama administration has created a “job-driven checklist.” The seven-point checklist will “ensure that what’s working best becomes what all Americans can expect when they participate in a training program.” Among the checklist items are work-based learning opportunities, such as on-the-job training and internships, and developing regional partnerships to ensure that local job-training providers are coordinating with other public and private agencies. This bureaucratic checklist will be engrained in all major job-training programs funded by the WIOA.
Earlier this year in National Affairs, my colleague Stuart M. Butler and I challenged the notion that the federal government can replicate social programs thought to be successful through checklists. The original success of a program being replicated is never a simple matter of easily traceable cause and effect that can be turned into a rudimentary checklist. The assumption is that following a checklist will allow administrators of other programs to produce the successful results in different settings. If replication by checklists were effective, then the federal government would have no trouble in replicating successful programs on a wider scale.
The replication of Center for Employment Training (CET) programs is a classic example of the inability of the federal government to duplicate “what works.” Based on the significant positive results of a 1992 experimental evaluation of one center in San Jose, Calif., that provided job-training services to youth, the federal government expanded the CET program across the nation. Twelve sites were later evaluated, and none of the results approached anything like those of the San Jose program. Even the sites deemed to have replicated the CET model with high fidelity could not come close to matching the success of the original program. Not only did these replications fail to increase the employment and earnings of participants, but the young men who participated in CET experienced declines in employment, earnings, and number of months worked.
Despite the lack of success among federal job-training programs, the Department of Labor and three other departments released a selective synthesis of job-training evaluations to support the notion that federal job-training programs can be effective. By “selective,” I mean that the report focuses solely on examples of particular, often local job-training programs that have some evidence of success. Omitted are the numerous large-scale evaluations of federal job-training programs that used the “gold standard” methodology of random assignment. Why omit these scientifically rigorous evaluations? Because these studies find federal job-training programs to be ineffective.
While the idea of replicating job-training programs thought to be successful through checklists may seem like wise and prudent policymaking, replication of these programs has a poor track record. When it comes to helping Americans increase their job skills and find good jobs, no one should expect a checklist created by bureaucrats in Washington, D.C. to be a silver bullet.
— David B. Muhlhausen is a research fellow in empirical policy analysis in the Center for Data Analysis at the Heritage Foundation and author of Do Federal Social Programs Work?