Interoperability: have we finally learned from the past?

There were many interesting and thought-provoking sessions at the recent Commodity Trading Week, but one that particularly caught our attention focussed on interoperability in commodity software.

As commodity management providers who have always advocated for end-to-end commodity management through technology ecosystems, we were pleased to see that software interoperability is back on the agenda, but saddened that after 20+ years it still needs to be advocated for.

What do we mean by interoperability?

Think about the different systems in use across your business; as a minimum most trading firms will have systems or processes for trade capture, risk management (CTRM), and an ERP, as well as ways to manage contracts and counterparties, and often a range of operational spreadsheets too. Now consider all the other systems that feed data into or out of your organisation. These may or may not be integrated into your core systems already; for example, how do you receive quality information from inspections, get information from counterparties and pricing feeds or use vessel tracking?

Processes at each stage of commodity trading can be complex and vary by commodity and by organisation as well as other factors. This makes it difficult to create end-to-end software that works for all use cases. Some large software providers have attempted to do so in the past by adding more modules and a degree of configurability to their software, but feedback within the industry is that these solutions often lack the flexibility to truly solve the business problem and that configurability, while useful, often adds needless complexity when done badly.

And these solutions are designed to be monolithic; if part of the system does not meet your needs, it can be very difficult, if not impossible, to then use a best-of-breed specific solution for those requirements and integrate that into the wider system, or to connect in other external systems.

Interoperability arises from a different software philosophy, shared by Gen10. We want our clients to create a complete end-to-end commodity management ecosystem and use the best software for their teams at each stage, whether it’s ours or not. Interoperability allows these different systems sourced from different providers to work together in one connected ecosystem, sharing data as if they were one platform.

Interoperability allows software vendors to be more agile and adaptive; they can focus on continuously improving their products, leaving other areas to other experts. It also means that commodity traders can choose the best solutions for each stage of their process and are not locked in to any one vendor, with the ability to change out or upgrade systems as their needs evolve.

The challenges of interoperability

Despite the many clear advantages of interoperability, and the fact that it is rapidly becoming a core business competency, the panel at Commodity Trading Week highlighted several challenges in creating a software ecosystem due to a lack of interoperability.

For example, almost every system now includes APIs to connect with other systems and data sources, but the panel found that it can still be difficult to connect them, citing multiple issues ranging from a lack of documentation to conflicting field types or character limits in some systems.

One development we have seen over recent years is more willingness to co-operate as an industry to overcome systemic challenges in commodity trading. This was reflected in the panel’s proposed solution of standardisation. And it was supported by the audience who, when polled, responded that “different data standards in software” was overwhelmingly their biggest challenge, even above “outdated systems”.

However, standardising interoperability comes with its own challenges. It can’t be driven by regulators as the challenge is international and spans jurisdictional boundaries, and with so many commodities, developing a standard that works for all parties would be difficult at best. Standards also need widespread industry adoption to work; there is little incentive for a company to invest in adopting a standard if it is only relevant for a small percentage of their business.

So, even taking each commodity in isolation via trade associations instead of considering the full vertical, there are many stakeholder organisations, and even more stakeholders within these organisations that would need to be involved – and still not solve the entire problem.

The good news from the discussion of these challenges is that they are being discussed, and that businesses seem ready to attempt to tackle challenges collaboratively.

We’ve been here before, so what’s changed?

Industry associations and regulators have certainly made a success of standard processes in the past, but when it comes to technology standardisation there has historically been more resistance, failed projects, and successful technology projects that failed to be widely adopted. So, will this latest push for collaborative technology finally be successful?

The Commodity Trading Week panel believed so. They argued that following the shift to technology-enabled remote working, technology is seen as more critical now than at any time in the past. The sense of urgency around technological improvement is also more pronounced now, with a major change in mindset amongst senior leaders.

Attitudes towards collaboration have also changed, and now follow a similar process to agile technology development. Whereas historical projects attempted to solve every challenge in one large-scale solution, nowadays there is more willingness to implement a smaller-scale improvement and allow it to evolve over time. For example, rather than attempting to create one association that speaks for all parties, there are more small-scale collaborative efforts being encouraged, that can spread and adapt to each other and allow a wholescale standard to evolve over time.

And of course, technology has also changed. Modern commodity management systems are far more flexible than a legacy CTRM and have interoperability in mind. It is now much easier to implement a system to solve a specific pain-point and integrate or further develop it down the line. Modern systems make it far easier for organisations to connect data flows within their organisation and with their partners. A mode of operating that industry analysts refer to as “Commodity Management as an Architecture”. Organisations can now store and use data anywhere – or in multiple places at once, with secure access, version control and verification.

It’s not standards that we need

It is both positive and disheartening to see a return to the discussion of interoperability. Positive to see that once again we as an industry are coming together to enact changes that will benefit all, but disheartening to see that a challenge we have been pushing to address for 20 years has still not been solved.

The ways of thinking we had in the past have clearly not been able to solve this problem, so it is good to see new thoughts on collaboration emerging. But the industry is also being held back by the technologies of the past. The push towards agreed standards across technologies is being driven by the fact that these systems do not integrate as well as they should. But instead of limiting business operations by creating a standard for all shared data, we should focus on ensuring our technologies are flexible enough to connect and interoperate without the need for these standards.

The problems caused by inflexible legacy technologies are so long-standing they have become embedded in the way we do business and in our thinking about the challenges we are looking to address. In the above-mentioned audience poll, a strong majority claimed that “different data standards in software” was a bigger problem than outdated systems. But it is the outdated systems that mean data standards are even needed in the first place!

With modern commodity management systems, each organisation has the flexibility to use the technology that works best for each of their operations, combine them into one completely joined-up commodity management ecosystem and integrate connections with external systems as desired. And these systems are easier to set up and to develop than their legacy equivalents.

More flexible commodity management allows every commodity market participant to operate in the manner they find most effective and retain the competitive advantage they have created through their own processes. And the flexibility means that systems can be truly interoperable, powering collaboration within the business and across complex and varied commodity supply chains.

The fact that we are having conversations about collaboration, interoperability and industry standards again is great news. But these conversations have been had before. The important thing is to go from discussing these changes to making them happen. Hopefully this time we as an industry will succeed where previous attempts have failed, but to do so, we need to learn from the past and do things differently this time. Mindsets have already changed, but if organisations remain subject to the technologies that have created the issues, they are going to struggle to solve them.

Modern technologies provide the collaboration and interoperability that organisations are looking for, with the flexibility to connect to other systems as needed, bypassing the challenge of creating standards, but allowing organisations to realise the benefits of collaboration. Set up a call with us today to talk about the many integrations and options Gen10 can offer, and how fast our past integrations have been.