# RepRisk interviews Dr. Julian Kölbel, Academic Advisory Board Member
1. RepRisk: One of the interesting findings from the Aggregate Confusion paper is that the difference in ESG scores cannot be solely attributed to differing methodologies. You found that there's substantial diversion between scores, even when they were based on similar methodologies. Can you please share the implications of this finding for investors?
Julian Kölbel: We find that the divergence in the raw data that underlies ESG ratings contributes about half of the overall divergence, and so the implication of that is that it's very difficult to understand why these ratings diverge.
Ideally, you'd want to drill down to the very bottom and see what is causing this difference. Is it that one rating emphasizes the environment more than the other? That would make sense. Or that one provider thinks that a particular company has its emissions under control while the other one doesn't? As an investor, that would be a point where you could do additional research and make up your mind about what is most important to you.
That would be the ideal case, but for that to work you need the aggregation methodology and the actual raw data. If you have neither of those, it's impossible to know why there are differences in the scores, and I think that leads to a lot of frustration.
2. RR: What are some of the solutions to the problem of aggregate confusion with ratings? What can data providers do? As an industry, what should we be doing?
JK: I think that transparency is one obvious response. Because that allows investors to trust that the numbers that they are seeing are based on something that makes sense to them and that they can independently ascertain. Transparency alleviates the concerns that there might be flaws in the data, because you can test for that.
And another recommendation I have is to have a consistent architecture of the ESG issues. From a practical point of view, it's tricky if you have 215 versus 78 versus 24 ESG issues. And obviously you can slice and dice ESG in many ways, but for the purpose of comparison and coherence, a standardized set of issues has a couple of advantages.
First, when comparing two ratings you could easily see not only at the aggregate level, but also at the issues level, where the differences lie in the methodology. So you immediately know where you might want to dig a little deeper. And the second advantage is that it could make ESG modular, so an asset manager could use one provider for most issues, but for one particular issue they could use a specialized provider.
So transparency and a coherent set of ESG issues would be my two recommendations for the ESG data space.
3. RR: As a researcher, you have been a proponent of RepRisk’s approach to ESG data and currently sit on RepRisk’s Academic Advisory Board. Can you share what is unique about the RepRisk dataset that makes it interesting to you?
JK: As a researcher, it’s very straightforward to understand what the RepRisk data does and what it doesn't do. It's very clear. And that makes it easy to apply for a specific use case, and to apply it in the correct way. As researchers, we are always trying to measure something, but because measuring is expensive and labor intensive, we look to providers such as RepRisk that have already done the work for us.
And we have to match what we are interested in with the data that we find. And in the case of RepRisk that's easy to do because it's very clear what the data is based on and how it is generated. Even to the point that it's easy to double-check whether all the relevant risk incidents are actually contained in the dataset. Which I've done by the way. For one paper that uses RepRisk data, we did a small robustness check and sampled some news events and checked whether they were contained in the dataset and indeed they were. We could see for ourselves that this is a high-quality dataset that we could rely on.
4. RR: What are some of the attributes of the RepRisk dataset that should be interesting from an investor’s perspective?
JK: At the end of the day, companies will make sure to tell you what they did great. RepRisk will tell you what they did wrong. That's why RepRisk is a really important complement to the corporate social responsibility report. If you consume both together, you will have a good impression of what the companies' issues are and what they're doing about them.
I think that we are far enough into the development of ESG that most people are not relying on a single score to tell them everything they need to know. There is specialization in the data and depending on if you want to look at risk or impact or values or compliance, and depending on whether you’re an active or a passive manager, there's very different demands. Simply put, the data and the product you buy should fit very neatly into the purpose and your strategy. And in order to match that, you need to know exactly what the data is doing.
Bio – Dr. Julian Kölbel
Dr. Kölbel is an economist and environmental scientist. His research covers the real-world impact of sustainable investing, analysis of ESG metrics, and the effects of climate change on risk and return. Dr. Kölbel is an Assistant Professor at the University of St. Gallen, where he is affiliated with the Center for Financial Services Innovation and teaches courses on impact investing and ESG metrics. He is also a research affiliate at MIT Sloan, where he co-founded the Aggregate Confusion Project. His research on the divergence of ESG ratings is well known in the industry and is widely cited in the academic literature. Dr. Kölbel has studied at ETH Zurich and the University of Oxford. Next to academic work, he serves on the investment committee of the Swiss pension fund, Abendrot.
