One program director’s experience with cost-benefit analysis: An interview with Mindy Tarlow
Mindy Tarlow is the executive director and chief executive officer of the Center for Employment Opportunities (CEO), whose headquarters is in New York City. In February, the U.S. Department of Health and Human Services (HHS) and the nonprofit research organization MDRC published a report on a three-year evaluation of CEO, including a benefit-cost analysis* conducted by the Vera Institute of Justice’s Cost-Benefit Analysis Unit.
Could you give a brief overview of the Center for Employment Opportunities?
The Center for Employment Opportunities does only one thing: provide employment opportunities for people with recent criminal convictions. Most clients come to CEO during the fragile period immediately after release from prison, when they are reentering their communities. We work with them for over a year, on average, and operate in an immediate, comprehensive way, putting people to work in paid transitional work groups. These groups provide structure and income, as well as a skill-building opportunity and platform for entering the full-time labor market, with the support of CEO’s team of job development professionals.
Following full-time job placement, CEO has a team of job retention specialists that provide a wide variety of services that help people keep their jobs—help with child support, help navigating through a difficult part of their work or a difficult thing outside of employment that keeps them from focusing on work. We refer them to the right services so they can stay focused on their jobs. CEO also offers financial incentives for staying employed, as well as re-placement services for anyone who has lost a job. We recently started a series of training programs to help people be more competitive, earn certificates, and advance their careers.
We started years ago as a demonstration project of the Vera Institute of Justice and spun off as a freestanding organization in 1996. In 2009, our work began to grow outside New York City. We opened offices in upstate New York with federal stimulus funding, and more recently, with support from the federal Social Innovation Fund, we opened offices in Oklahoma and California. Today, we operate seven offices outside New York City.
Your organization is one of four sites in the U.S. Department of Health and Human Services’ Enhanced Services for the Hard-to-Employ Demonstration and Evaluation Project. That meant a three-year evaluation of CEO—and a benefit-cost analysis was just one piece of that. What did the evaluation and benefit-cost findings show?
CEO is the only site in the Hard-to-Employ project that focuses exclusively on people with criminal records. HHS and MDRC chose CEO because our model was mature and, based on their criteria, seemed ready for a random-assignment evaluation. The study started in 2004. We enrolled people in random assignment over an 18-month period in New York City and there were 977 people split between the treatment and control groups. MDRC followed them for three years to determine the effect CEO was having. The impact evaluation found that CEO had a statistically significant impact on all measures of recidivism (subsequent arrests, convictions, and incarceration) for the three-year period for those recently released from prison, the core group of people targeted by CEO.
The study showed that recidivism rates among CEO participants decreased by as much as 26 percent. The benefit-cost analysis showed that financial benefits for taxpayers, victims, and participants exceed the costs—that for every $1 invested, CEO generates between $1.26 and $3.85 in benefits. Interestingly, the employment impacts, which were quite large during the reentry period CEO concentrates on, and were substantial throughout the first year of the program, faded in later years in contrast to the recidivism results, although there were some ongoing positive impacts for recently released people in years 2 and 3.
Why was it important for you to have a benefit-cost analysis accompany an outcome evaluation?
You’re talking to someone who worked at the New York City Office of Management and Budget for a decade! This is me putting my money where my mouth is. I have always felt that it’s the responsibility of all of us to do things that are socially and fiscally responsible. I have always been personally invested in that.
In our field of community corrections, what we do is part of the economic solution, not part of the economic problem. It’s our responsibility to prove that—to show government and others in positions to make budget decisions that by investing in an organization like CEO, you can save money that can be reinvested in other priorities. It’s incumbent on us to prove it. I see that as part of my mandate and part of my job. You go as far as you can to show that what you’re doing has a financial benefit to the city and state.
What would you tell other program directors about the benefit-cost analysis (BCA) process?
Among a couple of other big organizational assets, the evaluation and the benefit-cost analysis are essential components of the platform for our growth. But they took a long time to complete and get formally published—and also presented some organizational challenges.
For both the random assignment evaluation and the BCA, when you’re the one being evaluated, you have a much softer voice than you might have expected. Though it is perfectly appropriate and important to the integrity of the evaluation to have a firewall between the people doing the evaluation and those being evaluated, I was sometimes frustrated about not being able to participate as fully as I wanted to as the results were being interpreted, particularly given my background.
I also worry that there may be a lack of appreciation for how hard it is to actually show an impact when you engage in evaluation that’s this rigorous. And I think there’s a lot of skepticism about benefit-cost analysis in general. Everybody steels themselves for people poking holes in these analyses. But you can still make a very declarative statement of the benefit of your program if you do the benefit-cost analysis based on a solid evaluation and include a sensitivity analysis that demonstrates the costs and benefits using a range of assumptions.
The urgency to show a return on investment is only going to get bigger. It’s the wave of the future, whether it’s justice reinvestment or “pay for success” or other performance-based financing. Work with people who know how to do this and who you trust—and jump into the deep end of the pool.
*The MDRC report on CEO uses the term “benefit-cost analysis,” which is also used throughout this interview.