Recurrent neural networks
RNNs for sequence prediction
In our models, the sequence of vectors is produced by either a bidirectional RNN (Schuster and Paliwal, 1997) or a convolutional encoder (Rush et al., 2015).
3 Actor-Critic for Sequence Prediction
We note that this way of re-writing the gradient of the expected reward is known in RL under the names policy gradient theorem (Sutton et al., 1999) and stochastic actor-critic (Sutton, 1984).
我们注意到，重写预期回报的梯度这样的RL是已知的名字政策梯度定理下（萨顿等，1999）和随机演员评论家（萨顿，1984）。
Training the critic
Applying deep RL techniques
Attempts to remove the target network by propagating the gradient through qt resulted in a lower square error (Qˆ(ˆyt ; Yˆ 1...T ) − qt) 2 , but the resulting Qˆ values proved very unreliable as training signals for the actor
采样 5page
To compensate for this, we sample predictions from a delayed actor, whose weights are slowly updated to follow the actor that is actually trained. This is inspired by (Lillicrap et al., 2015), where a delayer actor is used for a similar purpose。
有关target critic network解释
CONTINUOUS CONTROL WITH DEEP REINFORCEMENT 1509.02971.pdf
