feature(cy): add BDQ algorithm

[Cloud-Pku]

Description

Add the bdq algorithm and config for hopper and halfcheetah env.

Related Issue

TODO

n-step for BDQ
gradient scaling for backbone network
unify bdq_nstep_td_error and q_nstep_td_error function
unify action_map function
unit Tests writing

Check List

• merge the latest version source branch/repo, and resolve all the conflicts
• pass style check
• pass all the tests

[codecov]

Codecov Report

Merging #558 (880ed8b) into main (0a25e46) will decrease coverage by 0.14%.
The diff coverage is 64.28%.

❗ Current head 880ed8b differs from pull request most recent head 7138bc4. Consider uploading reports for the commit 7138bc4 to get more accurate results

@@            Coverage Diff             @@
##             main     #558      +/-   ##
==========================================
- Coverage   84.59%   84.44%   -0.15%     
==========================================
  Files         555      556       +1     
  Lines       45197    45405     +208     
==========================================
+ Hits        38233    38344     +111     
- Misses       6964     7061      +97     

Flag	Coverage Δ
unittests	84.44% <64.28%> (-0.15%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
ding/model/common/__init__.py	100.00% <ø> (ø)
ding/rl_utils/__init__.py	100.00% <ø> (ø)
ding/policy/bdq.py	26.00% <26.00%> (ø)
ding/model/template/q_learning.py	85.82% <94.73%> (+0.69%)	⬆️
ding/model/common/head.py	98.56% <100.00%> (+0.13%)	⬆️
ding/model/template/__init__.py	100.00% <100.00%> (ø)
ding/model/template/tests/test_q_learning.py	99.18% <100.00%> (+0.05%)	⬆️
ding/policy/__init__.py	100.00% <100.00%> (ø)
ding/policy/command_mode_policy_instance.py	93.38% <100.00%> (+0.11%)	⬆️
ding/rl_utils/td.py	92.96% <100.00%> (+0.25%)	⬆️
... and 5 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.