Deep Reinforcement Learning for algorithmic trading (Test results 2)

After publishing the deep Q-learning test results, I have received several responses and questions. I found most of them useful and based on them I prepared a new test configuration. In this article, I’m going to share new results on the subject deep reinforcement learning for algorithmic trading. First, I want to list some of the questions that the followers asked:

  • Do training and testing data sets have non-overlapping time ranges?
  • Do you think that the training data length is enough?
  • Do you think that the input space size is large enough?

In the previous experiment, I trained the deep Q-network using the Litecoin data. Then, I tested it with Bitcoin data. Some of the readers concerned that although the training and testing datasets are different, they might be correlated if their time ranges overlap. Because I used the data recorded in the same time slice, it is highly likely that the network can memorize the correlation information between different coins. Therefore, I decided to change the training and testing configurations. In the new configuration, the training and testing coins as well as the time ranges are different. Using this configuration, I conducted several experiments and I observed that the profits are very low as compared the previous configuration. Then, I thought about the second question and decided to use longer training data which has 4 months length instead of 1 month. The new experiments showed that using longer training data increases the profit because the DQN agent exploits ‘experience learning’. Regarding the third question, it is all about the feature engineering. I extended the feature extraction class by adding extra features. The new version of this class could be found in the following post:

In summary, the new configuration has the following properties:

  • The testing and training data sets have different time ranges and coins
  • The training data length is 4 months instead of 1 month
  • The input space has extra features such as choppiness index

Now, let’s have a look at the close price history of the traning data. It’s the Litecoin data recorded between the dates 01 April, 2019 and 01 Aug, 2019. The data frequency is 1 hour.

Litecoin price history (01 April, 2019 – 01 August, 2019)

The testing data is the Bitcoin data recorded between the dates 01 Aug, 2019 and 01 Sep, 2019. In the following figure, you see the testing data, buy/sell instances, and the resulting profit made within 1 month period.

Bitcoin test data (01 August, 2019 – 01 September, 2019), trade instances, and profit

The results indicate that we made about 31 percent profit using 54 trades (buy/sell). Cryptocurrency market Binance applies 0.1% percent trading fee. In our example, if you had invested 1000$, after one month you would have obtained 1310 – 54 = 1256$ which is quite satisfactory. If you want to obtain the same results, please follow the steps given below:

And make sure that you use the new version of feature extraction class given below:

In the next post, I’m going to share the results with other coins such as, XRP (ripple), BNB (binance coin), and LTC (lite coin). Enjoy your trading!

Sharing is caring!

Leave a Reply

Your email address will not be published. Required fields are marked *