rlworkgroup / garage

65%

master: 64%

LAST BUILD ON BRANCH fix_vpg_ddpg

branch: fix_vpg_ddpg

CHANGE BRANCH

x

Reset

• fix_vpg_ddpg
• 2018.10
• Automated_Benchmarking_normalized
• GautamDev
• IndexedSlices_warning
• add_baseline
• add_changelog
• add_cnn
• add_cnn_progress
• add_dict_support
• add_erwr
• add_init
• add_param
• add_tnpg
• add_vanilla_loss
• add_wrappers
• add_wrappers_progress
• automated-benchmarking
• backport-415
• backport-418
• backport-426
• backport-434
• backport-434-446
• backport-544
• backport-pr-418
• backport_622_201810
• backup_exp_runner
• bc
• behavioral_cloning
• behavioral_cloning_ag
• benchmark
• box2d-install
• box2d_install_linux
• bugs_ryan
• bump_gym
• changelog_2018101
• check_modules
• clean_mujoco_conv_import
• codecov
• copy_actions
• dead_code
• decouple_sampler_from_algo
• deterministicMLP
• dm_comtrol-mujoco_1.5
• dqn
• dqn_2
• dqn_benchmark
• dqn_debug
• dqn_testing
• dqn_tf
• dyn_rand_nested
• epilson_greedy
• epsilon_greedy
• eric-torch
• example_shebang
• exp_runner
• fix-install
• fix-logging
• fix-scripts
• fix_copy_mjkey
• fix_dm_control
• fix_docker_ci
• fix_failed_tests
• fix_flake8_exclude
• fix_gcc_osx
• fix_lasagne_import
• fix_lbfgs
• fix_mujoco_import
• fix_off_policy_sampler
• fix_pickled_openai
• fix_pol_ent
• fix_pol_re
• fix_pol_std
• fix_precommit
• fix_replay_buffer
• fix_sampler
• fix_setup_osx
• fix_sleeping_proc
• fix_test_quirks
• fix_travis_tests
• fix_vvpg_v
• flake8-refactor
• flaky_tests
• formater
• garage_docker
• garage_experiment
• garage_model
• ignore_egg_info
• ignore_flake8_D107
• kill_stub
• latest_reps
• logger_decouple_tensorboard
• make_test
• maml
• master
• meta_learning
• mjpro200
• model_benchmark
• model_fix_pickle
• models
• models_embed
• models_gautam
• models_new
• models_progress
• models_ryan
• move_dm_tests
• move_nb_utils
• multitask_env
• new-logger
• new_dm_control
• newer-logger
• no_docker_opengl
• not_scripts
• ppo-test
• pr420
• pre_commit-flake8
• product_to_tuple
• randomized_env_test
• refactor_imports
• release-2018.10
• remove-most-envs
• remove_gym_dupes
• remove_horizon
• remove_layer_ddpg_test
• remove_layers
• remove_layers_lstm
• remove_mjkey_check
• remove_sawyer
• remove_theano
• rename_macos
• reps
• revert-430-backport-418
• setup_py
• tb_images
• td3
• test_bug
• test_bug_wip
• test_keras
• test_keras_local
• tf112
• theano-gpu
• torch
• torch-rlkit
• tsan
• uint8_support
• uint8_test
• unused-deps
• update-contributing
• update-docs
• update_gitignore
• v2018.10
• v2018.10.0
• v2018.10.1
• viskit_flake8

Build # 2906

Build Type

Pull #329

travis-ci-com

Committed by web-flow

Commit Message

Add miscellaneous changes

 - Make the replay buffer base class garage.replay_buffer.ReplayBuffer
abstract. Rename the regular replay buffer to SimpleReplayBuffer.
 - Delete enums and pass ReplayBuffer class into algorithm. Now use can
construct a replay buffer by their own.
 - Move `configure_dims` and `dims_to_shapes` into replay buffer
constructor. Delete `env.reset()` in configure_dims. Notice the reason
that we need to use `env.step()` because it's the only way to get
env_info in gym.GoalEnv. And the env_info is used to compute reward in
`env.compute_reward`. But the source code in gym shows that the env_info
is not actually used when computing reward. So we can not store info in
transitions.
 - Set default value for optimizer params in DDPG.
 - Remove commented code.
 - Redesign OffPolicyBatchPolopt. The BatchPolopt is more compatible for
on policy algorithms. In last commit, OffPolicyBatchPolopt inherits from
it. But actually off policy algorithms usually samples from experience
replay instead of batch samples optimization. So it sounds like an
unusual name. So rename it to OffPolicyAlgorithm and inherits from
garage.algos.RLAlgorithm. Also, the training process in
OffPolicyBatchPolopt is more specific for DDPG. Before design a sound
training process function, I move the training function into DDPG.

Pull Request Pull Request #329: Fix VPG and DDPG to use VectorizedSampler

Run Details

436 of 436 new or added lines in 17 files covered. (100.0%)

9787 of 15004 relevant lines covered (65.23%)

0.65 hits per line