Deep Reinforcement Learning test results for Litecoin, Ripple, and Binance Coin

In the previous post, I shared the trading results for Bitcoin data using deep reinforcement learning approach. I have also tested the model with some other coins using the same trained deep network. In this post, I’m going to share the results for Litecoin, Ripple, and Binance Coin. Remember that the training data was recorded between the dates 01 April, 2019 and 01 Aug, 2019 and its frequency was 1 hour.

In the first test, I used the Litecoin (LTC) data which is recorded between the dates 01 Aug, 2019 and 01 Sep, 2019. In the following figure, you see the testing data, buy/sell instances, and the resulting profit made within 1 month period.

Litecoin data, trades (buy/sell) instances, and profit

Notice that the data has downtrending characteristics. Its price almost linearly decreases from about 100$ to 70$ within 1 month. We also observe lower peaks during that period. On the other hand, when we have a look at the profit made by the deep Q-network (DQN) agent we observe nearly linear increase. Within the same period it reaches to 25.54 % using only 96 trades (buy/sell) which is quite satisfactory.

In the second test, I used the Ripple (XRP) data which is also recorded recorded between the dates 01 Aug, 2019 and 01 Sep, 2019. The testing data, buy/sell instances, and the resulting profit made within 1 month period are shown in the figure below.

Ripple data, trades (buy/sell) instances, and profit

Notice that for this case, the testing data has also downtrending characteristics. In addition, we observe very sharp fall between time indices 300 and 400. This time the profit made by the DQN agent reaches to 20.02% by using only 38 trades (buy/sell).

In the last experiment, I used Binance Coin (BNB) data. The results are shown in the following figure.

Binance coin data, trades (buy/sell) instances, and profit

In this case, we also have similar results. This time we made 11.76% profit using 70 trades. It is extremely important to test the model using several different testing data. The results in the previous post and this post show that the model we trained works quite well and it might be ready for real trading. If you also want to obtain the same results, please follow the steps given below:

That’s all for this post. Enjoy your trading!

Sharing is caring!

5 thoughts on “Deep Reinforcement Learning test results for Litecoin, Ripple, and Binance Coin

  1. Hi,

    first thanks for sharing,
    i have been working a lot also on dqn for trading.
    i would have a question:
    from your code, it seems that you only allowed the dqn to be long only, not both sort and long.
    it makes sens in order to feel the Q table when your are in position = 0 and action = 0 (sit), because you say if the market goes up, i fill a negative reward, because, i woul miss the opportunity to make a better buy now, that buy for the next step.
    given the above explanation, do you think you can transform the dqn to be short or long instead of only initiate position as long?

    feel free to contact.

  2. Hi matokoz,

    hi have some difficulties to understand the fellowing, if you could explain:

    when you take a position given pos[0] = self.pos[0] + self.actions[action_id], you take that possition at the end of the previous state, so if you are in self.t, you buy at a price in self.t – 1.

    hence, price = self.df.iloc[self.current_index][‘close’] in self.total = self.total + self.account / price, seems false. it should be price = self.df.iloc[self.current_index – 1][‘close’].

    mooreover, self.pos[0] belong to self.current_index, so how can you combine
    next_state_line = self.df[self.data_features].iloc[next_index] with self.pos[0]
    it should be next_state_line = self.df[self.data_features].iloc[self.current_index] with self.pos[0]
    state(current_index) = self.df[self.data_features].iloc[self.current_index], self.pos[0]

    price = self.df.iloc[self.current_index][‘close’]
    price_next = self.df.iloc[next_index][‘close’]

    given reward(next_index) = price_next – price, it makes sens, that step function return
    state(current_index), reward(next_index)

  3. Hi,
    Thank you for running various tests thanks to your great code.
    However, in my case, even if I did the same process under the same conditions, I am not getting the same results as you.
    (Same coin(LTC for learning, and the others for testing) and period(1.4.2019~1.8.2019 for learning and 1.8.2019~1.9.2019 for testing) as yours)
    So, could you share information about the hyperparameters you used for learning?
    (such as trade cost, maximum epochs, drop out prob, etc.)
    Thanks,

Leave a Reply

Your email address will not be published. Required fields are marked *