What Gets Measured

The researchers of the S&O Inclusive Economy Center joined forces with S&P Global Ratings to take stock of the most influential ESG frameworks currently in use and ask: What social factors do these frameworks actually cover, and what factors are being left out? On which social factors do they agree and where is there disagreement?

This is a very timely report, as companies are under increasing regulatory and market pressures to step up their actions along the ‘S’ dimension and to be transparent about it.

The Scope

We highlight four main challenges that may limit effective social factor reporting and evaluation: 

1. Scope of coverage – The market lacks comprehensive agreement on what factors fall under the ‘S’ dimension and of common terminology and definitions to describe and categorize these factors.
2. Depth of coverage – Factors may not be treated with adequate level of depth required to make informed decisions related to what are complex issues. For instance, reporting criteria may require evidence of a policy but not actions to implement the policy. 
3. Performance metrics and data – Factors are often treated qualitatively or are measured using predominantly input variables that do not necessarily reflect performance, outcomes or impact, which can lead to more of a box-checking exercise.
4. Relationship to the “E” and “G” factors of ESG – An understanding of how the ‘S’ factors interact with the “E” and “G” factors remains limited – leading some frameworks to underweight social factors in the overall ESG performance assessments.

Combined, these challenges inhibit consistent, reliable, and comparable social performance analyses of companies used to inform stakeholder decision-making and drive improvements over time. In this report, we address the first challenge – scope of coverage – by establishing what social factors get measured and what is missed using ESG frameworks.

Methodology

We created a new database ourselves, and for this, we proceeded in two steps. First, we singled out all the ESG frameworks that were sector agnostic, generalist in approach (meaning not restricted to one specific ‘S’ dimension only), and with publicly available data on framework approach and topics included. Eighteen ESG frameworks met these selection criteria. Second, we consolidated all the social topics included by these frameworks into 33 social factors organized within five social dimensions. We chose these dimensions and labeled the social factors in the most recognizable, meaningful, and interpretable way.

We identified 33 individual social factors along the following 5 social dimensions:

1. Product responsibility
2. Labor and human rights at work
3. Community and business ethics
4. Employment practices
5. Employee health and safety. 

Developing such a categorization was not straightforward; undertaking this process highlighted a great deal of inconsistency in how social issues are labelled, described, and organized. 

Undertaking this process highlighted a great deal of inconsistency in how social issues are labelled, described, and organized.

The social factor coverage differs depending on ESG framework type used

We find that the strength of social factor coverage varies widely depending on the type of ESG framework and the social dimension one considers. For example, ESG standards and reporting frameworks tend to provide the strongest coverage of employment practices and employee health and safety, but the weakest coverage of social factors related to product responsibility. The opposite is true for ESG evaluation frameworks: They tend to provide the strongest coverage of product responsibility and the weakest coverage of employment practices and employee health and safety. ESG impact frameworks sit somewhere in between, they tend to privilege depth over breadth. 

All ESG framework types provide weak coverage of issues related to community and business ethics and disagree on which factors to include.

As for labor and human rights at work, ESG framework types differ in terms of their coverage: ESG evaluation frameworks tend to provide good coverage, ESG standards and reporting frameworks moderate coverage, and ESG impact frameworks weak coverage.

Finally, it is important to recognize that some social issues currently don’t get covered by any of the major ESG frameworks. For example, none of them require companies to consider their impacts on the refugee crisis.

We find that the firms’ ESG frameworks vary wildly in the social factors covered, resulting in very different pictures of a firm’s social performance.

Despite some topical overlap, social factor inconsistencies exist across all ESG frameworks

We made a list of the top 10 social factors that are most consistently covered by all ESG framework types. Minority inclusion, workplace accidents and injuries and social protection and freedom of association top this list. We did the same for top 10 social factors that are the least systematically covered by all ESG framework types. Sanctions, legal action and regulatory compliance; mental health and wellbeing, and social issue controversies are the three social factors that are the least systematically covered.

Conclusion 

We should not take it for granted that all important areas are systematically covered by a given ESG framework. And some important areas may not be covered at all.

Investors and stakeholders at large looking to understand a company’s social impact in a comprehensive way, are well advised to combine multiple ESG frameworks, and not rely on one ESG framework only. 

The authors

Marieke Huysentruyt, S&O HEC Paris
Leandro Nardi, S&O HEC Paris
Bruce Thomson, S&P Global Ratings
Bénédicte Faivre-Tavignot, S&O HEC Paris