Skip to content
This repository was archived by the owner on May 27, 2026. It is now read-only.

Fix to issue #648#739

Merged
ChrisCummins merged 3 commits into
facebookresearch:developmentfrom
nluu175:development
Aug 1, 2022
Merged

Fix to issue #648#739
ChrisCummins merged 3 commits into
facebookresearch:developmentfrom
nluu175:development

Conversation

@nluu175

@nluu175 nluu175 commented Jul 26, 2022

Copy link
Copy Markdown
Contributor

Fixes #648

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 26, 2022

@ChrisCummins ChrisCummins left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @nluu175! Thanks for the PR. I left some comments inline 🙂

Cheers,
Chris

Comment thread compiler_gym/wrappers/time_limit.py Outdated
Comment on lines +35 to +44
# def step(self, action: ActionType, **kwargs):
# assert (
# self._elapsed_steps is not None
# ), "Cannot call env.step() before calling reset()"
# observation, reward, done, info = self.env.step(action, **kwargs)
# self._elapsed_steps += 1
# if self._elapsed_steps >= self._max_episode_steps:
# info["TimeLimit.truncated"] = not done
# done = True
# return observation, reward, done, info

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can delete these lines. Don't submit commented-out code

# done = True
# return observation, reward, done, info

def multistep(self, actions: Iterable[ActionType], **kwargs):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def multistep(self, actions: Iterable[ActionType], **kwargs):
def multistep(self, actions: Iterable[ActionType], **kwargs):
actions = list(actions)

I would suggest forcing the conversion of actions to a list here, since there could be a subtle bug if actions is an iterator, as it would break the len(actions) below.

fkd.close()


# @pytest.mark.xfail(strict=True, reason="https://github.com/facebookresearch/CompilerGym/issues/648")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't commit commented out code

# @pytest.mark.xfail(strict=True, reason="https://github.com/facebookresearch/CompilerGym/issues/648")
def test_time_limit(env: LlvmEnv):
"""Check CycleOverBenchmarks does not break TimeLimit"""
env = TimeLimit(env, max_episode_steps=1)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest testing with max_episode_steps=3, so that you can test that done is False for the first two calls to step().

env.reset()
_, _, done, _ = env.step(0)

assert done

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
assert done
assert done
assert info["TimeLimit.truncated"], info

Best to also test the reason why done=True.

@nluu175

nluu175 commented Jul 26, 2022

Copy link
Copy Markdown
Contributor Author

Hi @ChrisCummins . Thanks for the feedbacks. I'll resolve those and as soon as possible!

@codecov-commenter

codecov-commenter commented Jul 26, 2022

Copy link
Copy Markdown

Codecov Report

Merging #739 (1d91115) into development (4a9f10d) will decrease coverage by 0.02%.
The diff coverage is 100.00%.

Impacted file tree graph

@@               Coverage Diff               @@
##           development     #739      +/-   ##
===============================================
- Coverage        88.79%   88.77%   -0.03%     
===============================================
  Files              129      129              
  Lines             7854     7855       +1     
===============================================
- Hits              6974     6973       -1     
- Misses             880      882       +2     
Impacted Files Coverage Δ
compiler_gym/wrappers/time_limit.py 96.55% <100.00%> (+0.12%) ⬆️
...loop_tool/service/loop_tool_compilation_session.py 87.83% <0.00%> (-2.71%) ⬇️
compiler_gym/service/service_cache.py 89.47% <0.00%> (-2.64%) ⬇️
...ompiler_gym/service/client_service_compiler_env.py 90.23% <0.00%> (-0.22%) ⬇️
compiler_gym/service/connection.py 78.92% <0.00%> (+0.33%) ⬆️
compiler_gym/views/observation_space_spec.py 85.71% <0.00%> (+2.85%) ⬆️
compiler_gym/views/observation.py 100.00% <0.00%> (+3.12%) ⬆️
compiler_gym/util/truncate.py 96.29% <0.00%> (+3.70%) ⬆️

@ChrisCummins

Copy link
Copy Markdown
Contributor

Changes LGTM, thanks! I will merge once CI tests pass.

Cheers,
Chris

@ChrisCummins ChrisCummins merged commit b58e83b into facebookresearch:development Aug 1, 2022
This was referenced Nov 1, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Logical Failure when combing TimeLimit Wrapper with IterateOverBenchmarks

4 participants