A common instantiation of the carbon credits concept is to allow companies and individuals to mitigate the impact of the environmental damage they cause due to emitting greenhouse gases by paying for a “credit”. In this case the credit is sold on behalf of a project which has somehow reduced carbon usage elsewhere.
The idea is that whilst you still created your emissions you also paid for someone else to work on something that removes the same amount of greenhouse gases as you emitted. Thus in aggregate you didn’t increase the amount of emissions in the world. Buying these credits is one way companies claim to be carbon neutral.
But work by various scientists and journalists suggests that most of the supposed carbon credits generated by the world’s biggest provider of them, Verra, are most likely entirely fictional. That is to say that purchasing a carbon credit from them does not correspond to a substantial net reduction in carbon usage at all. Were this to be true, carbon credits are not only a waste of money and resources, but also potentially damaging to the planet if they cause people to incorrectly feel comfortable with the idea that whatever pollution they’re causing is being neutralised by someone else.
One common method of carbon reduction Verra claims to enable is the prevention of deforestation. Forests are stores of carbon. But when they’re damaged or destroyed that carbon ma be released back into the atmosphere as carbon dioxide, further contributing to climate change. So Verra funds projects that prevent areas of forest being destroyed that otherwise would have been, keeping the carbon safely in its store.
From the Guardian:
…if an organisation estimates its project will stop 100 hectares (247 acres) of deforestation, it can use a Verra-approved formula to convert that into 40,000 CO2e (carbon dioxide equivalent) of saved carbon emissions in a dense tropical forest if no deforestation takes place
Deforestation prevention makes up 40% of the credits Verra approves.
But according to several studies, there’s usually a big problem with the claim of “will stop 100 hectares of deforestation” in the above.
As is all too common when it comes to non-trivial data analysis, the complication is in defining what the result we observe, in this case the rate of deforestation following a Verra project, should be compared to.
Intuitively the answer to that is “compared to a world where someone didn’t buy that Verra carbon credit”, or “a world where Verra didn’t run that deforestation project”.
However these schemes aren’t run as randomised controlled experiments. Thus the all-important calculation of what deforestation benefit the Verra intervention causes relies on having an accurate estimation of how much deforestation would have occurred without their intervention. This is not something that is directly observable.
There are plenty of big and important problems we can’t or don’t want to run RCTs on. The famous example is smoking; no-one ever ran a big, long term study where humans were randomly allocated to smoke cigarettes vs not, and yet we have developed overwhelming evidence via other means such that almost everyone who knows anything about the subject understands that smoking tobacco is harmful to health.
So luckily we have methods to address questions that are observational, rather than experimental, in nature. Fields such as causal inference tackle exactly that sort of setup.
It’s rare, perhaps impossible, that one will be able to deliver a perfectly certain answer in these cases. We are after all trying to simulate what would have happened in a world that doesn’t exist: the counterfactual. But if done well we might produce a useful answer with an appropriately quantified level of uncertainty that can inform our decisions.
Importantly, this counterfactual world doesn’t mean a world where deforestation carries on at the same rate as it did before Verra started up. Many other people and organisations also recognise deforestation as a problem. There exist many campaigns, national policies, environemtal regulations and the like that aim to reduce the rate of deforestation irrespective of Verra, as well as external drivers such as the economy and market demand for the associated products. Buying a carbon credit is net pointless if the reduction in deforestation it’s associated with would have happened in any case, “for free”, due to actions from someone else.
Studies on the impact of carbon reduction forest preservation projects
The article I read cited three studies. The first two have passed peer-review and are published, the third is currently only available as a preprint.
- Overstated carbon emission reductions from voluntary REDD+ projects in the Brazilian Amazon by West et al.
- A global evaluation of the effectiveness of voluntary REDD+ projects at reducing deforestation and degradation in the moist tropics by Guizar-Coutiño et al.
- Action needed to make carbon offsets from tropical forest conservation work for climate change mitigation by West et al.
Overstated carbon emission reductions from voluntary REDD+ projects in the Brazilian Amazon
The first used a quasi-experimental synthetic control method. Simplistically, this means that they create a “control group” of real actual observable areas to compare areas with carbon reduction focussed anti-deforestation projects (which are known as REDD+ projects) to.
The synthetic control areas are matched to the actual project areas in terms of their accessibility and biophysical characteristics, such that they match the historical deforestation trends of the REDD+ project areas. If this is done well then it’s reasonable to expect that whatever we see in the control group areas with regards to deforestation rate would be a reasonable baseline for our expectations of what would have happened in the project areas if these projects hadn’t existed.
They looked at 12 such projects and unfortunately found “no significant evidence that voluntary REDD+ projects in the Brazilian Amazon have mitigated forest loss.” Deforestation in project areas is lower than the appropriate control group in just 4 projects, and only achieves significance in one case. It was sometimes the case that the project areas saw more deforestation than their controls, as can be seen in their figure 3.
In 11 out of 12 projects, the baseline assumptions behind the claims the projects made when extolling the amount of deforestation they’d save reflected dramatically higher default deforestation than what was seen in the control areas. This suggests they were extremely pessimistic about what would happen if they didn’t intervene, and thus gave themselves a lot of credit for something that would probably have happened anyway. Their figure 5 illustrates this.
Overall,
….the weight of the evidence suggests that these projects caused less reduction in deforestation than claimed and that few projects actually achieved emission reductions
A global evaluation of the effectiveness of voluntary REDD+ projects at reducing deforestation and degradation in the moist tropics
The second study used a similar technique, this time finding synthetic controls via matching project areas on “sociodemographic and biophysical characteristics that are typically associated with deforestation, elevation and slope, distance to the nearest urban center in 2015. and distance to forest edge.
This study analysed 40 projects and found that the projects were associated with some reduction in deforestation, with the rates in the project averaging 0.22% per year vs 0.36% in the matched controls. Some project sites appeared to have greater deforestation or degradation than their controls but more saw less.
However the general effect size was small, leading to the researchers concluding that they’ve evidence that these schemes do work in at least some places, but only address a very small percentage of carbon emissions.
Our analysis provides promising evidence that site-based REDD+ projects have helped reduce deforestation, particularly in areas of high deforestation threat. Yet, emissions reductions in the 40 REDD+ projects analyzed represent a tiny fraction of global emissions. In total, they amounted to about 0.01% of 2018 emissions, or 0.13% of emissions from tropical deforestation in 2013.
Digging into this paper’s results, a group of journalists concurred that the actual area of deforestation that had been prevented was small and concentrated. 75% of it came from just 4 out of the 40 projects.
They further went on to compare the results the researchers established with the claims that Verra had made with regards to the baseline, i.e. what would have happened if they hadn’t intervened. There was enough data for them to do this in 32 of the projects.
They found that Verra’s assumptions had overstated the amount of forest that would be lost without their assistance by 400% in comparison to what the researchers’ control groups imply. Once again, this suggests they were, accidentally or otherwise, seeking to take credit for a reduction in deforestation that a lot of likely would have happened in any case.
A lot of the positive effect actually came from just 3 projects; the baseline assumptions would have been overstated by not far off 1000% had it not been for them.
Action needed to make carbon offsets from tropical forest conservation work for climate change mitigation
This one shares many authors with the first one, and is currently in pre-print status. It looks at 31 sites from 27 projects across various tropical countries, again using the same kind of synthetic control method. Any sites where the controls didn’t pass their method of validating that the control sites did fairly represent the project site were excluded.
26 sites remained for the main analysis. Of these, just 6 showed some evidence of reduction in deforestation potentially attributable to the carbon credit project. Even these tended to be against forecast baselines that were much more pessimistic than the control sites would suggest was realistic, meaning that the amount of deforestation reduction was significantly less than the project claims.
The researchers attempt to consider the results in terms of the carbon offset credits that these projects have issued based on these sites. They had enough information to do this for 18 out of the 27 projects.
They calculate that if one uses the estimates claimed by the project owners then they should have generated up to 89 million carbon offsets. But 71% of these credits come from projects that this analysis suggests have not in reality significantly reduced deforestation or emissions at all compared to their control groups.
Worse yet, many of the rest are from projects that did likely prevent some deforestation but a lot less than the original estimates from the project team baseline imply. All in all, they calculate that only around 6% of the issued credits associate with real carbon emission reductions that wouldn’t likely have happened anyway. Or, if you prefer to think of it the other way around, 94% of the credits that were issued should never have been approved.
In reality, as of November 2021 these projects had issued 62 million credits, of which at least 24% have been “used” by individuals or companies looking to offset their own emissions. Based on the findings of this analysis, that means the credits have been already used to supposedly “offset” nearly 3x as much carbon as these projects actually saved in total…and there’s still 47 million extra carbon credits for sale in the market that correspond to no additive emission reduction.
All in all:
Only a minority of the projects significantly reduced deforestation compared to the ex-post counterfactuals, and even those did not reduce deforestation to the extent claimed.
In conclusion
The dynamic involved in reducing emissions via preventing deforestation are clearly very complex. In reality any deforestation efforts are surely subject to various pressures in all directions including those of design, implementation, politics and economics. It’s unrealistic to model every detail directly. So assumptions of some kind thus do have to be made given what data we have access to.
All these studies are of course open to debate in one way or another. The researchers in each case acknowledge various weaknesses or degrees of researcher freedom that mean that their particular approach is not the only possible one, their answer not the only plausible one.
The most obvious potential contention is whether or not the correct comparison groups have been selected. Perhaps there are some factors that cause changes in deforestation level that no-one yet identified but yet matter. It’s possible that one day the literature might change in favour of these types of credits.
Verra certainly are claiming that it will. They strongly disagree with conclusions of all these studies on the basis they “that do not account for project-specific factors that cause deforestation”.
Their general claim is that:
…they reach incorrect conclusions because they rely on synthetic controls that do not accurately represent the pre-project conditions in the project area…Synthetic controls compare a project to a control scenario based on a set of variables that impact deforestation, known as covariates, whereas Verra’s approach for REDD+ projects compares them to real areas.
However as far as I can tell, the method they used to do this hasn’t been fully published, in which case it’s not like anyone can check whether it makes sense or even if they did it in the first place. It’s hard to argue that Verra don’t have a vested interest in their projects looking effective. They might be. I hope they are! But they need validating by someone other than themselves.
I’m especially curious what the “real areas” they refer to are. After all, the synthetic controls are all coming from real places. And they can’t be comparing their project to the same area it’s located to see what deforestation would occur if the project had never happened because, well, the project did happen.
A point they make that I do think is fair is that Verra doesn’t pick these schemes at random. There may be something special about the places they select that no-one else understands and truly does make them different from areas that in all other respects look similar. But the only way we could determine whether this makes enough difference to make their credits actually appear to have the claimed benefit would be to know how the areas are selected. That information might let researchers select more apt control locations if it turns out their current ones aren’t great. But they didn’t mention how this is done in their press release so, again, it’s probably uncheckable.
As complex as it is, real answers to the question of the efficacy of this type of carbon credit scheme are clearly critical for schemes like Verra’s. After all, they are claiming to be one of the solutions to literally saving the planet. As far as I have understood, which may not be all that far, these papers are the best third-party attempt at quantifying the impact of these schemes so far, and unfortunately don’t present a very positive message.
They’re not entirely bad news. It seems like there’s evidence that these schemes can have some positive effect on reducing deforestation. But perhaps only in some areas, under specific conditions, and even then the impact is much less than has been assumed. They might still be good projects to do, but they should not be permitted to generate the amount of carbon credits they currently do if we don’t believe they’re having a commensurate effect.
Given the stakes, we must not delude ourselves into the belief that we’re mitigating our own damage to the planet by paying for a carbon credit if the best current evidence is in the direction of credits having little impact. The idea that we can just pay our way to having no net impact on emissions feels almost certain to lead towards us emitting more of these dangerous gases, in addition to diverting resources away from ideas that could fare better.
Dabbling with Data 