I would first like to thank you for taking the time to read the article and express your thoughts.
What you stated is completely true! It is extremely difficult to predict outcomes for prices of stocks, oil e.t.c. The reason for which this appears to be the case is the element of the unknown. In the stock market, for example, insider information is king. Hedge funds constantly manipulate the markets and leave mathematical and machine learning models completely helpless.
Oil, in particular, is a prime example where the ‘unknown’ factor is huge. This is due to the nature of this commodity and its price’s vulnerability to geopolitical games (e.x. Russia vs U.S.A. e.t.c.).
This article is meant to explore how one would go when attempting to predict the prices of a commodity using time series analysis. Nevertheless, as you saw the accuracy is quite high, but once more the ‘unknown’ can always be the reason for which everything gets messed up.
If you are trying to predict the prices of a commodity such as oil, I think that in a real-world application there are some tweaks one can make to his/her model in order to be able to be more prepared for such cases.
This entails the following:
- The model should possess the ability to be interactive (human interference can be used as data inputs for the model).
- In addition to time-series analysis, sentiment analysis on news articles related to the commodity should be done in 3-hour intervals.
- Twitter sentiment analysis should be conducted daily.
- The model should also take into consideration, data imported from google trends.
Although the above will not be able to have 100% accuracy when the ‘unknown’ comes into play, situations will be surely better handled.
I personally believe that a model following the guidelines presented above could have partly predicted the recent oil-crash (I am currently working on creating such a model). This could be done in the following way:
- Newsfeed analysis would show that there is an increased supply of oil currently in the market.
- Using google trends, the model would then see that the demand for oil has suddenly decreased.
- Twitter could also be used as a reference point but it would not really help in this case.
The model would then come to the conclusion that there is an excess supply of oil in the market and thus the prices will probably go down, indicating that the best move would be shorting the market.
I find this topic quite interesting, and I would be more than glad to discuss the matter further (My Linkedin can be found in the bio section of my profile).