Digital transformation has been identified as one way to help improve profitability in commodities trading, where profit margins have been squeezed since 2015. In fact, digital transformation has become a trend in the industry and at least part of the digitalisation effort often takes the form of big data projects, which analysts claim are the way “to arrest the relentless erosion of profit margins”. Indeed, some have even stated that the capacity for advanced predictive analytics will be such a competitive advantage that it will accelerate consolidation of the industry.
Big data explained
Big data involves collating and analysing data sets comprised of information from a wide range of sources and types, including structured and unstructured data. Definitions both of big data and of the benefits organisations can gain from it vary, but it has historically been seen as perhaps the preserve of organisations large enough to employ teams of data scientists equipped with the most powerful computing hardware. However, cloud commodity management apps now give all organisations the ability to gain more insights from their data, and show that big data analytics is no longer the preserve only of the very largest trading firms.
As the name suggests, big data often involves the analysis of large amounts of data, although it is not the file size that determines whether a data set is classed as big data. Big data is defined by its complexity and the number of sources it is drawn from.
When assessing whether your project involves big data, consider the 3 Vs:
Volume – the large number of sources as well as data points; for example, the data your organisation generates when processing a contract may be supplemented by external market pricing, a physical contract, emails and vessel tracking information.
Velocity – the data is constantly changing as new information is created in real-time, so big data also needs to be rapidly processed and analysed in real-time.
Variety – data can be in a range of formats, from structured databases to unstructured text documents and any format in-between.
The promise of big data in commodities
Simply having big data stored in your organisation is not a competitive advantage. The benefit comes from analysing the data to generate useful insights and in sharing these insights with the people who need them. These analytics can enable organisations to address challenges they have been unable to before and make better decisions based on more complete data.
There are many other potential benefits of this analysis, not least that it is one of the few remaining ways for trading companies to generate a competitive information advantage in an era of unprecedented market transparency. Faster access to the insights created from data analytics means faster decisions based on better data, and can therefore confer an edge in trading.
As well as better decision-making, analysing big data can reduce costs and time in the real world, such as allowing operators to make better use of shipping capabilities or accelerating contract progression. Big data can also lead to predictive analytics, where some future events can be identified before they occur. For example, when Gen10 implemented a real-time data analytics and visualisation tool for the USDA, they gained the ability to identify whether laboratory equipment was going to develop a fault before the problem occurred.
Read the full case study
The challenge of using big data in commodities
Big data analysis is not easy, which is why it can be such an advantage to the organisations that do it well. Simply organising data so that it can be analysed involves understanding its sources, the systems that generate it and who needs access to the data, as well as security and legal issues.
Because the data is often collected in a range of formats and structures, creating a useable format that works across platforms and draws on all relevant sources can be difficult. The sheer amount of information that is used can also create challenges such as “noise” or irrelevant signals that distract from relevant insights.
Data also needs to be securely stored whilst being accessible to all who need it. And accessibility can be more complicated than simply providing login details. Traders in particular need a range of information wherever and whenever they are agreeing deals, which may mean they need this data on a mobile or tablet, outside of your offices and outside of business hours.
Breaking down the CTRM monolith
CTRM systems are vast repositories of data that could power faster, better decision making in many organisations – if that information were readily available to the people who need it. As the pace of digitalisation increases and competitors bring even more advanced analytics capabilities online, it is more important than ever for organisations to be able to unlock value from the data held within their CTRM.
For those organisations using a large CTRM that does not provide the insights they need, there are ways to access and mine these vast stores of data, even if replacing the CTRM is not viable. Commodity management apps allow you to break down the CTRM monolith by adding areas of additional functionality where you need them – whether that’s capturing pricing in a mobile app, analysing risk and forward curves or monitoring sustainability data across your supply chain.
Apps also allow you to securely share this information with all users in the format they need; for example, some Gen10 apps are designed with mobile use in mind. The apps can also provide secure business insights to individuals in refreshable external reports. Gen10 recently created an Excel Daily Business Report specifically for CEOs and senior managers of large groups. This report uses a Gen10 API to display information from the CTRM to these leaders in an instantly-accessible document without the need for them to navigate a CTRM platform that they otherwise wouldn’t use.
With dashboards, visualisation tools and true real-time reports, all app users are empowered to use data insights – even users without any background in data science. And outside of reports, Gen10 apps provide users with the information they need to make better decisions as part of their everyday flow of work. External data sources such as market data and vessel tracking are incorporated for risk management and smoother operations, and data generated as part of a deal’s progress is readily available for all users. Any data that is input at deal capture automatically populates documents such as invoices and shipping instructions down the line, and unstructured data is also available to users, including documents associated with the deal. Breaking down the CTRM monolith puts your organisation’s data in the hands of its users and gives them the insights they need to create more value.
Whilst a large data science team focussed exclusively on big data analytics may not be feasible for most commodity traders, there are steps organisations can take on their digital transformation journey. One of the key steps is to break down the CTRM monolith by adding new capabilities that allow users to access the data they need in an easy-to-use format and improve the data capabilities of all teams without a big investment in big data.
Want to read more?
Subscribe now for monthly updates
By submitting your details you agree that we can store your data and communicate with you. You can opt out of these communications at any time. Read all in our Privacy Policy.
