In the previous post, we trained our deep Q-network using Litecoin historical data. In this post, we are going to learn how to test the network using a test data. We will also see whether we can make money or not using AI. Before starting, I highly recommend you to read the previous post, because you will need a trained network in order to test the trading data. The link of the previous post is:
Now, let’s have a look at the close price history of the traning data. It’s the Litecoin data recorded between the dates 10 July, 2019 and 10 Aug, 2019. The data frequency is 1 hour.
I assumed that you trained your network and you have the weights file named ‘ltcusdt-1hour-weights.h5f’. Now, it is time to test the network. For testing, it is extremely important that you use a different data set. This is because we need to make sure that the performance of the network on the training data is not due to overfitting. Therefore, we are going to use Bitcoin data which is also recorded between the dates dates 10 July, 2019 and 10 Aug, 2019. You may also use some other coin data or Litecoin data in a different time interval. For this you may have a look at the following posts:
I assumed that you have already obtained the file ‘btcusdt-1hour.csv’ for testing. Let’s get started. In your working directory, create a file named ‘binanceTest.py’. Using your favorite editor, copy and paste the following code block:
from binanceFeatures import FeatureExtractor from binanceCreateModel import create_model from keras.optimizers import Adam import matplotlib.pyplot as plt import matplotlib.gridspec as gridspec from rl.agents.dqn import DQNAgent from rl.memory import SequentialMemory import gym from gym.envs.registration import register, make ENV_NAME = 'binanceEnvironment-v0' if ENV_NAME in gym.envs.registry.env_specs: del gym.envs.registry.env_specs[ENV_NAME] register( id=ENV_NAME, entry_point='binanceLib:binanceEnvironment', max_episode_steps=10000, ) env = make(ENV_NAME) input_file = 'btcusdt-1hour.csv' output_file = 'btcusdt-1hour-out.csv' w_file_name = 'ltcusdt-1hour-weights.h5f' feature_extractor = FeatureExtractor(input_file, output_file) feature_extractor.extract() feature_list = feature_extractor.get_feature_names() trade_cost = 0.5 env.init_file(output_file, feature_list, trade_cost, False) model = create_model(env) memory = SequentialMemory(limit=5000, window_length=1) dqn = DQNAgent(model=model, nb_actions=env.action_size, memory=memory, nb_steps_warmup=50, target_model_update=1e-2, policy=None) dqn.compile(Adam(lr=1e-3), metrics=['mse']) dqn.load_weights(w_file_name) dqn.test(env, nb_episodes=1, action_repetition=1, callbacks=None, visualize=True, nb_max_episode_steps=None, nb_max_start_steps=0, start_step_policy=None, verbose=1) fig = plt.figure() gs = gridspec.GridSpec(2, 1, figure=fig) ax1 = fig.add_subplot(gs[0, 0]) ax1.plot(env.df['close'],'-b',linewidth = 0.5) ax1.plot(env.df['close'][env.df['action']>0],'og', label='Buy',markersize = 1) ax1.plot(env.df['close'][env.df['action']<0],'or', label ='Sell',markersize = 1) ax1.set_xlabel('Time') ax1.set_ylabel('Price') ax1.legend() ax2 = fig.add_subplot(gs[1, 0]) ax2.plot(env.df['profit'][:-1],'-r',linewidth = 0.5) ax2.set_xlabel('Time') ax2.set_ylabel('Profit (%)') plt.tight_layout() plt.show()
Now, run the script ‘binanceTest.py’ and you will see a result like the following:
Testing for 1 episodes ... --------------v0---------------- episode: 1, time: 0, gain: 0.00, trades: 0 episode: 1, time: 100, gain: 1.25, trades: 21 episode: 1, time: 200, gain: 21.75, trades: 51 episode: 1, time: 300, gain: 33.86, trades: 68 episode: 1, time: 400, gain: 42.28, trades: 89 episode: 1, time: 500, gain: 41.13, trades: 108 episode: 1, time: 600, gain: 54.61, trades: 123 episode: 1, time: 700, gain: 64.38, trades: 147
In this result, ‘gain’ indicates the percent profit and ‘trades’ indicates the number of trades. You will also see a plot results like the following:
In the first figure, you see the price history as well as the buy/sell instances. In the second figure, you see how your profit increases with time. Linear increase in the figure shows that although falls and fluctuations take place in the test data, the algorithm continues to make profit which you should expect from a good predictor. Now, we can say that our DQN agent performs quite well and at the end of one month period, we make about 64 percent profit using 147 trades (buy/sell). Cryptocurrency market Binance applies 0.1% percent trading fee. In our example, if you had invested 1000$, after one month you would have obtained 1643 – 147 = 1496$ which is quite satisfactory result. In the next post, we are going to discuss how we can construct more stable networks and how we can increase our profit. That’s all for today. Enjoy your trading!