After publishing the deep Q-learning test results, I have received several responses and questions. I found most of them useful and based on them I prepared a new test configuration. In this article, I’m going to share new results on the subject deep reinforcement learning for algorithmic trading. First, I want to list some of the questions that the followers asked:
- Do training and testing data sets have non-overlapping time ranges?
- Do you think that the training data length is enough?
- Do you think that the input space size is large enough?
In the previous experiment, I trained the deep Q-network using the Litecoin data. Then, I tested it with Bitcoin data. Some of the readers concerned that although the training and testing datasets are different, they might be correlated if their time ranges overlap. Because I used the data recorded in the same time slice, it is highly likely that the network can memorize the correlation information between different coins. Therefore, I decided to change the training and testing configurations. In the new configuration, the training and testing coins as well as the time ranges are different. Using this configuration, I conducted several experiments and I observed that the profits are very low as compared the previous configuration. Then, I thought about the second question and decided to use longer training data which has 4 months length instead of 1 month. The new experiments showed that using longer training data increases the profit because the DQN agent exploits ‘experience learning’. Regarding the third question, it is all about the feature engineering. I extended the feature extraction class by adding extra features. The new version of this class could be found in the following post:
In summary, the new configuration has the following properties:
- The testing and training data sets have different time ranges and coins
- The training data length is 4 months instead of 1 month
- The input space has extra features such as choppiness index
Now, let’s have a look at the close price history of the traning data. It’s the Litecoin data recorded between the dates 01 April, 2019 and 01 Aug, 2019. The data frequency is 1 hour.
The testing data is the Bitcoin data recorded between the dates 01 Aug, 2019 and 01 Sep, 2019. In the following figure, you see the testing data, buy/sell instances, and the resulting profit made within 1 month period.
The results indicate that we made about 31 percent profit using 54 trades (buy/sell). Cryptocurrency market Binance applies 0.1% percent trading fee. In our example, if you had invested 1000$, after one month you would have obtained 1310 – 54 = 1256$ which is quite satisfactory. If you want to obtain the same results, please follow the steps given below:
- Training steps: Deep Q-learning for algorithmic trading
- Testing steps: Deep Q-learning test results for algorithmic trading
And make sure that you use the new version of feature extraction class given below:
In the next post, I’m going to share the results with other coins such as, XRP (ripple), BNB (binance coin), and LTC (lite coin). Enjoy your trading!