rlworkgroup / garage / 2890

65%

master: 64%

Pull #329

travis-ci-com

Add miscellaneous changes

 - Make the replay buffer base class garage.replay_buffer.ReplayBuffer
abstract. Rename the regular replay buffer to SimpleReplayBuffer.
 - Delete enums and pass ReplayBuffer class into algorithm. Now use can
construct a replay buffer by their own.
 - Move `configure_dims` and `dims_to_shapes` into replay buffer
constructor. Delete `env.reset()` in configure_dims. Notice the reason
that we need to use `env.step()` because it's the only way to get
env_info in gym.GoalEnv. And the env_info is used to compute reward in
`env.compute_reward`. But the source code in gym shows that the env_info
is not actually used when computing reward. So we can not store info in
transitions.
 - Set default value for optimizer params in DDPG.
 - Remove commented code.
 - Redesign OffPolicyBatchPolopt. The BatchPolopt is more compatible for
on policy algorithms. In last commit, OffPolicyBatchPolopt inherits from
it. But actually off policy algorithms usually samples from experience
replay instead of batch samples optimization. So it sounds like an
unusual name. So rename it to OffPolicyAlgorithm and inherits from
garage.algos.RLAlgorithm. Also, the training process in
OffPolicyBatchPolopt is more specific for DDPG. Before design a sound
training process function, I move the training function into DDPG.

435 of 435 new or added lines in 18 files covered. (100.0%)

9489 of 15004 relevant lines covered (63.24%)

0.63 hits per line