rlworkgroup / garage
65%
master: 64%

LAST BUILD BRANCH: release-2018.10
DEFAULT BRANCH: master
Repo Added
Files 239
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH fix_replay_buffer
branch: fix_replay_buffer
CHANGE BRANCH
x

3053

Pull #356

travis-ci-com

web-flow 
Change replay buffer initialization

This commit changes replay buffer initialization in a more proper way.
Formerly, replay buffer initialized the shape of each key in the
transition by getting the flat_dim of env_spec.*_space. This works for
Box and Dict spaces. But it has two flaws:
1. It only considers when observation is Dict space. But doesn't deal
with the situation when action or other key has Dict space.
2. If action_space is Discrete(3), the ReplayBuffer._buffer["action"]
will have a shape of (size, time_horizon, 3). However, the actual value
of action is of shape (1). So that causes a serious error.

Based on the above disadvantages, a better idea would be initialize the
buffer shape by sampling. Like when adding a real transition to the
buffer. Or use env.sample(). I used the former one. Because when trying
the latter option, the Box.sample() causes an invalid range error. To
solve this we may need to creat an Box space instance and then use
sample(). It is not proper to do this in a construcing process in replay
buffer. Thus, I initialize the shape of each key in buffer when adding a
real transition. This has an advantage is that we may add info into the
buffer for GoalEnv.compute_reward().

Also, the update plotter line in ddpg were in wrong position. Fixed it.
Pull Request #356: Change replay buffer initialization

14 of 14 new or added lines in 2 files covered. (100.0%)

9761 of 14995 relevant lines covered (65.1%)

0.65 hits per line

Relevant lines Covered
14995 RELEVANT LINES 9761 COVERED LINES
0.65 HITS PER LINE
Source Files on fix_replay_buffer
Detailed source file information is not available for this build.

Recent builds

Builds Branch Commit Type Ran Committer Via Coverage
3053 fix_replay_buffer Change replay buffer initialization This commit changes replay buffer initialization in a more proper way. Formerly, replay buffer initialized the shape of each key in the transition by getting the flat_dim of env_spec.*_space. This works for Box... Pull #356 web-flow travis-ci-com pending completion  
3052 fix_replay_buffer Change replay buffer initialization This commit changes replay buffer initialization in a more proper way. Formerly, replay buffer initialized the shape of each key in the transition by getting the flat_dim of env_spec.*_space. This works for Box... push CatherineSue travis-ci-com pending completion  
3040 fix_replay_buffer Change replay buffer initialization This commit changes replay buffer initialization in a more proper way. Formerly, replay buffer initialized the shape of each key in the transition by getting the flat_dim of env_spec.*_space. This works for Box... Pull #356 web-flow travis-ci-com pending completion  
3039 fix_replay_buffer Change replay buffer initialization This commit changes replay buffer initialization in a more proper way. Formerly, replay buffer initialized the shape of each key in the transition by getting the flat_dim of env_spec.*_space. This works for Box... push CatherineSue travis-ci-com pending completion  
3028 fix_replay_buffer Change replay buffer initialization This commit changes replay buffer initialization in a more proper way. Formerly, replay buffer initialized the shape of each key in the transition by getting the flat_dim of env_spec.*_space. This works for Box... push CatherineSue travis-ci-com pending completion  
3027 fix_replay_buffer Change replay buffer initialization This commit changes replay buffer initialization in a more proper way. Formerly, replay buffer initialized the shape of each key in the transition by getting the flat_dim of env_spec.*_space. This works for Box... push CatherineSue travis-ci-com pending completion  
See All Builds (1528)