In this blog, we will discuss 6 Generations of Demand Forecasting Software, how the logic and algorithms are developed over time. This article is written in collaboration with LOKAD where they have talked about,,
“How companies went from using technologies that were fundamentally based on mathematics, which had not changed that much since the 18th century, to Big Data-oriented technology powered by Machine Learning and Deep Learning”
It is known that demand forecasting is the process of using historical data and other information to estimate future customer demand within a defined period.
Correct demand forecasting can provide companies with valuable information about their potential in the current market and other markets so that managers can make informed decisions about prices, business growth strategies, and market potential.
Without demand forecasts, companies may make wrong decisions about their products and target markets, and wrong decisions may have a profound negative impact on inventory holding costs, customer satisfaction, supply chain management, and profitability.
There’s a way to do it better – find it – Thomas A. Edison
How to Run a S&OP Process – Benefits & Barriers
Lokad is an enterprise software provider, founded 12 years ago by Joannes Vermorel, and it is doing its best to provide the most accurate predictions that technology can produce. Its technology is constantly evolving to reflect the latest discoveries in mathematics and computer science.
You can watch the conversation with Joannes Vermorel – CEO & Founder at Lokad in The Supply Chain Show below.
In the past decade, data-related technologies have developed at an alarming rate. A number of companies started to develop from the use of mathematics-based technology, which has not changed much since the 18th century but turned to big data-oriented technology powered by machine learning and deep learning. Lokad has always been committed to maintaining a leading position and bringing the best scientific knowledge to supply chain optimization.
Lokad has started its journey while passing through 6 interesting forecasting generations :
“Whenever you need to serve a client, you need to anticipate his needs” – Joannes Vermorel
What are the 6 Generations of Demand Forecasting Software
-
The First Generation of 2008 : Classic Forecast (The Classical Time Series Forecasting Techniques)
This generation is about switching from a manually adjusted mathematical model to a fully automated benchmark test of the entire model library.
The problem was about winning benchmarks accuracy, it was getting worse although the forecast was more accurate.
Knowing that supply chain costs are concentrated on the extremes (the high demand) not the average and that the low demand is the one that causes the dead inventory.
One of the problems of this first generation is that it focuses only on what happens on the middle.
2. The Second Generation of 2012: Quantile Forecast
Through this generation, the aim is to forecast the extremes and it’s about having massive buyers on purpose in order to have more profit.
It’s about changing from average forecasts to biased forecasts that reflect business-specific asymmetries.
3. The Third Generation of 2015: Quantile Grids
Looking at the entire probability distribution of demand and injecting supply chain constraints. Instead of having one extreme scenario, we tend to have more quantiles.
The discovery was that it wasn’t only the demand that needed to be forecasted, but also lead time and plenty of other areas where there is uncertainty.
4. The Fourth Generation of 2016 : Probabilistic Forecasting
First, there were two areas where Lokad was departing from the quantile grid :
- Looking at all the possible futures with probabilities not just a series of quantiles steps. It’s mathematically quite different in terms of the way of organizing algorithms, etc.
- It’s about starting to embrace the idea of looking for other types of forecasts.
It’s about embracing uncertainty with the help of machine learning and high-dimensional statistics.
5. The Fifth Generation of 2018: Deep Learning
Probabilistic forecasting and predictions powered by robotics through artificial intelligence (AI) and GPU grids.
There was a problem with expressiveness. Before deep learning, machine learning and libraries were set of models, and so far, until 2014, machine learning was a vast collection of models and Lokad was a collection of models as well. Lokad realized that there was a sort of accidental complexity at the software level.
Thus, deep learning is essentially part of the whole machine learning as a body of knowledge. Its emphasis is on building extraordinary complex functions in order to capture extraordinary complex phenomena.
6. The Sixth Generation of 2019 : Differentiable Programming
The fusion of two algorithm fields: machine learning and numerical optimization.
It may be considered as a subset of deep learning. It is game-changing since it lets you embrace supply chain through high dimensional methods the structure of the supply chain problems.
The Data’s quality & frequency
Although there are many data problems, in the western world (developed countries), and during the last two decades, most supply chains have been digitalized and the quality of their Data, objectively speaking, is considered excellent in the sense that companies know exactly what they buy, what they produce, what they have in stock and what they sell.
Nevertheless, people still have serious problems :
- First, when most Lokad competitors are having problems, it’s because they have prepackaged enterprise software that make very strong assumptions about the way the Data should be and that don’t match with the reality of the situation.
- When people say that the Data is too infrequent, they should have tools that, statistically speaking, are able to manage the data and embrace its efficiency.
The Right Ingredients & The secret of success
Lokad’s technology is not about using one or even several magical statistical models, it is a combination of multiple ingredients that together create the right alchemy. Therefore, Lokad quickly realized how big the gap between pure mathematical modeling and supply chain reality is.
According to the theory, the effective method is to apply it to the actual business inefficient: the data is not clean, the depth is not enough, and it is too sparse. The large number of references or entries in the sales history of certain businesses makes the entire model category extremely difficult to use. Then, due to the constraints of the supply chain itself, improving the classic accuracy indicators of forecasts will actually reduce the performance of the enterprise. Consequently, Lokad must suggests appropriate technical solutions to all these problems and completely change its perspective on forecasting and supply chain optimization.
Correlations
with Deep Learning
When viewing one product at a time, there is simply not enough data to generate accurate statistical forecasts. In fact, in most consumer markets, the product life cycle is less than 4 years, which means that, on average, most products don’t even have a 2-year history-that is, the minimum required to perform reliable seasonal analysis Depth when viewing a single time series. Lokad solves it through statistical correlation: the information obtained from one product helps to improve the prediction of another product. For example, even if the product has only been sold for 3 months, Lokad will automatically detect the applicable season of the product. Although seasonality cannot be observed with only 3 months of data, if there are older, longer-lived products in the historical record, seasonality can be extracted and applied to new products.
Computing Power
through Cloud Computing and GPUs
While using correlations in historical data to greatly improve accuracy, it also increases the amount of calculations to be performed. For example, to view all possible pairs of 1,000 products, the combination is less than 1,000,000. To make matters worse, many companies have more than 1,000 products. By using cloud computing and graphics processing units (GPUs), when customers push data to us, Lokad allocates machines when needed. Then, in less than 60 minutes, it will return the results while reallocating the computers accordingly. Since the cloud (Microsoft Azure) Lokad uses is billed by the minute, it only consumes the capacity it really needs. Since no company needs to make multiple predictions per day, this strategy can reduce hardware costs by more than 24 times compared with traditional methods.
Probabilities
to embrace business constraints
The traditional forecast is the median forecast, that is, there is a 50% chance that the value will be higher or lower than future demand. Unfortunately, this classic vision does not solve the core problem of the supply chain: avoid stockouts and reduce inventory. In 2016, Lokad introduced the concept of supply chain probability forecasting, in which the probability of each future demand level can be estimated. Lokad does not predict the value of each product, but rather the entire probability distribution. For slow-movers, unstable sales and surging demand, probabilistic forecasts are much better than classical forecasts.
From a mathematical library to and end to end solution
Lokad has a huge library of statistical models. It includes famous classics such as Box-Jenkins, exponential smoothing, autoregressive and all its variants. In addition, because the classic model cannot make good use of correlation, Lokad has developed a better model that can take advantage of all available data. From the beginning, Lokad has been monitoring the quality of delivered forecasts and running simulations to carefully evaluate the remaining weaknesses of its technology. Lokad continuously improves its models and provides new models and new paradigms for its library. Therefore, its customers will benefit from continuously improving technology. However, Lokad’s team realized long ago that this was not enough. They need to have a deeper understanding of the reality of the supply chain and the constraints and particularities of each business. Therefore, they not only do not require any statistical skills from the customer, but they can manage the entire process to provide a fully usable solution, which includes precise purchase orders, scheduling or pricing recommendations, and dashboards of key performance indicators to assess its accuracy.
Lokad’s supply chain scientists can help any company include all business insights in a tailor-made implementation. This can be achieved by using its supply chain-oriented programming language. Its flexibility allows them to fine-tune the script so that it can fully reflect the peculiarities of their business, thus providing a perfect complement to their forecasting technology.
Before conclusion, we would recommend visiting Meade Willis, who helps companies integrate systems and automate their business processes with customized and out-of-the-box cloud solutions.
Conclusion
There are many reasons why demand forecasting is an important process for companies:
Sales forecasts help with business planning, budgeting and goal setting. Once you have a good understanding of future sales, you can start to develop a wise purchasing strategy to ensure that your supply meets customer needs.
It enables companies to optimize inventory more effectively, increase inventory turnover and reduce holding costs.
It provides insight into upcoming cash flows, which means that companies can more accurately budget to pay suppliers and other operating costs, and invest in the development of the company.
With sales forecasts, you can also identify and correct any entanglements in the sales channel in advance to ensure that your business performance remains strong throughout the period.
If you think we have missed something important in describing the journey of 6 Generations of Demand Forecasting Software, please let me know in the comments or in Linkedin.
Very Useful Guides and Courses from SCMDOJO
An Introduction to Demand Planning & Forecasting
S&OP Course- Why S&OP, 5 Step Process, Benefits & Barriers
The Complete Guide to Plan-For-Every-Part (PFEP)
About the Author- Dr Muddassir Ahmed
Dr MuddassirAhmed is the Founder & CEO of SCMDOJO. He is a global speaker, vlogger and supply chain industry expert with 17 years of experience in the Manufacturing Industry in the UK, Europe, the Middle East and South East Asia in various Supply Chain leadership roles. Dr. Muddassir has received a PhD in Management Science from Lancaster University Management School. Muddassir is a Six Sigma black belt and founded the leading supply chain platform SCMDOJO to enable supply chain professionals and teams to thrive by providing best-in-class knowledge content, tools and access to experts.
You can follow him on LinkedIn, Facebook, Twitter or Instagram
