The data used in the analysis of sustainability-related factors has much room for improvement. Data quality affects our ability to understand exposures to different climate and environmental risk factors and how these change over time, the contributions to positive and negative externalities, and the vulnerabilities facing investments.
Though numerous studies have looked into the issue of data quality, exactly how bad the problem is hard to say precisely. It is, however, raised regularly as a major issue by financial firms and their supervisors, as well as by policymakers, civil society, and researchers. This data quality and integrity gap hampers the pricing of climate and environmental risks and externalities and the capital reallocation required to transition to a sustainable global economy.
Sustainability data reporting frameworks for the financial services sector require widespread availability of self-disclosed information from investee companies and clients. Since this information is typically provided on a voluntary basis, data quality and coverage can vary significantly. When reported data is inconsistent or not available at all, alternative datasets are used as proxies to fill gaps or replace disclosures. These can range from production data or revenue based emission factors to generalised datasets or high-level statistics.
These datasets can miss out on a lot of company-level detail and context, which ultimately limits their usefulness. Applying these for target setting, for instance, might limit a financial institution’s ability to monitor progress, as metrics end up tracking performance of a sector as a whole rather than the role and progress of a company within it.
In addition, the prioritisation of reported data, either verified or unverified, poses additional challenges as the quality of reported data varies widely by sector or company. This can be due to a mix of issues, such as firm-level application of GHG accounting rules which can allow for varying approaches, the underlying emissions measurement, or the completeness of datasets amongst others.
Efforts do exist that categorize and assess data quality to provide transparency into these issues and incentivise data improvements. One example is the Partnership for Carbon Accounting Financial (PCAF) Global GHG Accounting and Reporting Standard’s data quality scoring framework. This framework, built around different types of financial assets, introduces a data quality hierarchy with reported data ranked highest, followed by various inferred estimation methods based on emissions, physical or economic activity data that generalise to all companies in a sector. However, this approach overlooks sector specific data availability and nuances which risks incentivising the use of suboptimal datasets based on a universal and generalist view of data quality issues.
What can we do to try and improve the situation?
While the availability and quality of reported information is set to increase in the next couple of years (e.g. US Securities and Exchange Commission climate-related disclosures, UK Sustainability Disclosure Requirements), there are often good quality datasets available that already exist today for various high impact industries. While these might not be considered under existing frameworks and data quality rankings, they could offer more granular, company or geography specific insights.
One example is the power sector, where datasets from non-profit organisations (e.g. Global Energy Monitor, World Resources Institute, ClimateTrace) or from regulators (e.g. US Environment Protection Agency, European Environment Agency) can provide granular insights to fill reported data gaps. Another example is in the agri-food sector where numerous datasets and insights from non-profit organisations (e.g. Trase, MightyEarth) allow environmental damages from the production of specific commodities in specific regions, to be attributed to traders and buyers in commodity supply chains.
We believe the market would benefit from methodologies that evaluate data quality, particularly on a sector by sector basis, given that most data quality and integrity issues are sector specific.
The Oxford Sustainable Finance Group at the University of Oxford is therefore launching the Sectoral Data Quality and Integrity Project to conduct empirical research on the most relevant and complete available sectoral datasets today. The project will compare reported data to other data collection methods on a sector- and geography-specific basis with the aim of evolving data quality frameworks to map to sector-specific characteristics.
This will include analysis of the growing number of sector specific asset-level datasets from both open initiatives (e.g. Global Energy Monitor, Spatial Finance Initiative) and commercial providers (e.g. WoodMackenzie, S&P Global Platts). The expanded quality and availability of datasets will enable financial institutions to analyse numerous sustainability issues, from greenhouse gas emissions to physical climate risk or biodiversity impacts, with enhanced granularity at the company, portfolio or country level. We need to know which datasets actually have quality and integrity.
Data quality metrics and analysis from this project will be applicable to different use cases for financial institutions. From net zero target setting, to financed emission calculation, risk management or climate risk reporting. We believe that providing clear sector-specific guidance on data quality and characteristics, will help improve and align the datasets that underpin different reporting and analytical frameworks. This will enable financial institutions to better assess the quality and type of data they are working with, by understanding it within the sector specific context rather than scoring it against an idealised universal framework. It will also help financial institutions evaluate and communicate their data needs and contextualise their own reports for its ultimate users.
We will be working closely with financial institutions, alliances, standard setters and other stakeholders to ensure recommendations are practical and implementable within existing reporting frameworks.
By Ben Caldecott & Christophe Christiaen, Oxford Sustainable Finance Group, University of Oxford. To find out more, contact christophe.christiaen@cgfi.ac.uk
