Why the CVCA’s newest VC impact study is truly ground-breaking

Canada’s Venture Capital and Private Equity Association (CVCA) and Industry Canada published a new study last week, showing that  VC-backed companies grow faster in terms of their assets, number of employees, revenues, sales, wages and R&D expenditures, relative to non-VC-backed companies.

Critics will likely make the usual points about the study — that it was designed for marketing purposes,  produced by consultants who were hired by industry associations, and based on partial, and therefore unreliable, data. At best , they will say, it’s an illustration of what venture capital can produce when it is successful.

(Full disclosure: I directed such a study, sponsored by the CVCA, in 2009.)

However, what is interesting about the latest study is that these criticisms do not apply at all. A little history of VC and PE impact studies can help explain this.

Because VC-backed companies are private, the main challenge for impact studies is access to data.

To address this challenge, a first generation of studies, generally sponsored by industry associations, relied on surveys of VC-backed companies that were conducted with the help of association members. As rigorous as they could be, they had some significant limitations:

  • They were not randomized, as response rates were usually 15-30%, and responses were subject to various biases, most notably survival biases (i.e. there was a good probability that failing companies would be under-represented);
  • Responses were self-declaratory, so their quality and accuracy could not be controlled;
  • In order to obtain responses, these surveys asked for only a limited set of variables – usually employment and revenues, for a limited number of years (usually two, from beginning to end);

The other main challenge in a VC impact study is the construction of control groups to which VC-backed companies can be compared. As they do not have access to detailed data on a set of comparable companies, studies only produce either gross impacts or comparisons with sectors of the economy, such as ICT, life sciences or even the whole private sector – which are not real comparables.

To address these issues, academics designed a second generation of studies in which they:

  • Linked VC or PE databases to official databases, such as the U.S. Bureau of Census or Statistics Canada;
  • Used their access to these large databases to pair VC-backed companies with companies that were comparable at the time of investment;

In so doing: (i) they eliminated the potential survey biases; (ii) they gained access to a much larger number of variables (employment, revenues, sales, assets, wages, R&D expenditures) collected with a consistent methodology for all years over the period; (iii) they were able to build a robust control group of comparable companies based on a large set of variables; (iv) they were able to conduct rigorous econometric studies, and; (v) they eliminated any temptation to massage the results (for strict confidentiality reasons, statistical institutes run the computation, but do not give any access to individual data).

The first and most comprehensive set of PE economic impact studies of this sort was developed for the World Economic Forum (2008, 2009 and 2010) by a team led by Harvard’s Josh Lerner. That study focused on U.S. buyout companies and used the Longitudinal Business Database of the U.S. Bureau of Census. Other studies on venture capital impact in the U.S. have followed.

The CVCA study published last week is the first of its kind to be developed in Canada. It was conducted by a team of economists from Industry Canada, linking Thomson Reuters‘ databases of VC-backed companies with firm-level administrative data from Statistics Canada. And Statistics Canada provided the above-mentioned guarantees of independence and methodological robustness. The technical study that supports it can be found here.

The good news is that the study confirms the results of the previous survey-based studies (including the 2009 study that I directed) – not only in terms of direction (VC-backed companies outperform), but also in terms of orders of magnitude (the impacts are large). See for yourself in a comparison of the 2013 CVCA study with other international studies completed over time – it’s posted on the CVCA’s website.

Dr. Gilles Duruflé is an independent consultant advising venture and private equity funds, institutional investors and governments. He is executive vice-president of the Québec City Conference and president of its Public Policy Forum on Venture Capital and Innovation.

Photo courtesy of Shutterstock.