-
Notifications
You must be signed in to change notification settings - Fork 281
This PR adds the Momentum strategy #1469
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 9 commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
47d8a06
Add Momentum strategy
dongwonmoon 6c20f54
Format code with black and isort
dongwonmoon 9e8092c
Remove unnecessary DataFrame import
dongwonmoon a6733f4
Add type annotation for frequency table variable
dongwonmoon bb00034
add Momentum strategy
dongwonmoon 37a5271
changed annotation
dongwonmoon e236312
add description
dongwonmoon ccf81ca
add description
dongwonmoon 5348663
run black
dongwonmoon bcdf817
add name
dongwonmoon 020dba4
add name
dongwonmoon a349ee0
run black
dongwonmoon File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| from axelrod.action import Action | ||
| from axelrod.player import Player | ||
|
|
||
| C, D = Action.C, Action.D | ||
|
|
||
|
|
||
| class Momentum(Player): | ||
| """ | ||
| This strategy is inspired by the concept of Gradual and the mathematical foundation of | ||
| the Momentum optimizer used in deep learning. | ||
|
|
||
| The idea is that trust (or cooperation) evolves dynamically. A shift in trust can | ||
| create significant and rapid changes in the player's behavior, much like how momentum | ||
| responds to gradients in optimization. | ||
|
|
||
| Parameters: | ||
| - alpha: Momentum decay factor that determines the rate of trust reduction. A higher value leads to slower decay, and the opponent's Defect acts as a trigger. (Optimized by Genetic Algorithm) | ||
| - threshold: The minimum momentum required to continue cooperation. If momentum falls below this value, the strategy switches to Defect as punishment. (Optimized by Genetic Algorithm) | ||
| - momentum: Represents the inertia of trust, dynamically changing based on past cooperation. | ||
|
|
||
| Names: | ||
| - Momentum: Original name by Dong Won Moon | ||
|
|
||
| """ | ||
|
|
||
| name = "Momentum" | ||
| classifier = { | ||
| "memory_depth": float("inf"), | ||
| "stochastic": False, | ||
| "long_run_time": False, | ||
| "inspects_source": False, | ||
| "manipulates_source": False, | ||
| "manipulates_state": False, | ||
| } | ||
|
|
||
| def __init__( | ||
| self, | ||
| alpha=0.9914655399877477, | ||
| threshold=0.9676595613724907, | ||
| ) -> None: | ||
| super().__init__() | ||
| self.alpha = alpha | ||
| self.threshold = threshold | ||
| self.momentum = 1.0 | ||
|
|
||
| def __repr__(self): | ||
| return f"Momentum: {self.alpha}, {self.threshold}" | ||
|
|
||
| def update_momentum(self, opponent_action): | ||
| # If the opponent defects, the momentum decreases, reflecting a loss of trust. | ||
| action_value = 1 if opponent_action == C else 0 | ||
| self.momentum = ( | ||
| self.alpha * self.momentum + (1 - self.alpha) * action_value | ||
| ) | ||
|
|
||
| def strategy(self, opponent: Player) -> Action: | ||
| if len(self.history) == 0: | ||
| self.momentum = 1.0 | ||
| return C | ||
|
|
||
| else: | ||
| self.update_momentum(opponent.history[-1]) | ||
| return C if self.momentum >= self.threshold else D | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,88 @@ | ||
| import axelrod as axl | ||
| from axelrod import Action | ||
| from axelrod.strategies.momentum import Momentum | ||
| from axelrod.tests.strategies.test_player import TestPlayer | ||
|
|
||
| C, D = Action.C, Action.D | ||
|
|
||
|
|
||
| class TestMomentum(TestPlayer): | ||
| name = "Momentum" | ||
| player = Momentum | ||
| expected_classifier = { | ||
| "memory_depth": float("inf"), | ||
| "stochastic": False, | ||
| "long_run_time": False, | ||
| "inspects_source": False, | ||
| "manipulates_source": False, | ||
| "manipulates_state": False, | ||
| } | ||
|
|
||
| def test_initialisation(self): | ||
| player = self.player(alpha=0.9, threshold=0.8) | ||
| self.assertEqual(player.alpha, 0.9) | ||
| self.assertEqual(player.threshold, 0.8) | ||
| self.assertEqual(player.momentum, 1.0) | ||
|
|
||
| def test_repr(self): | ||
| player = self.player(alpha=0.9, threshold=0.8) | ||
| self.assertEqual(repr(player), "Momentum: 0.9, 0.8") | ||
|
|
||
| def test_strategy(self): | ||
| actions = [(C, C)] | ||
| self.versus_test( | ||
| axl.MockPlayer(actions=[C]), | ||
| expected_actions=actions, | ||
| init_kwargs={"alpha": 0.5, "threshold": 0.5}, | ||
| attrs={"momentum": 1.0}, | ||
| ) | ||
|
|
||
| actions = [(C, D), (C, D), (D, D)] | ||
| self.versus_test( | ||
| axl.MockPlayer(actions=[D]), | ||
| expected_actions=actions, | ||
| init_kwargs={"alpha": 0.5, "threshold": 0.5}, | ||
| attrs={"momentum": 0.25}, | ||
| ) | ||
|
|
||
| def test_vs_alternator(self): | ||
| actions = [(C, C), (C, D), (C, C), (C, D), (D, C)] | ||
| self.versus_test( | ||
| axl.Alternator(), | ||
| expected_actions=actions, | ||
| init_kwargs={"alpha": 0.5, "threshold": 0.5}, | ||
| ) | ||
|
|
||
| def test_vs_cooperator(self): | ||
| actions = [(C, C), (C, C), (C, C), (C, C), (C, C)] | ||
| self.versus_test( | ||
| axl.Cooperator(), | ||
| expected_actions=actions, | ||
| init_kwargs={"alpha": 0.5, "threshold": 0.5}, | ||
| ) | ||
|
|
||
| def test_vs_defector(self): | ||
| actions = [(C, D), (C, D), (D, D), (D, D), (D, D)] | ||
| self.versus_test( | ||
| axl.Defector(), | ||
| expected_actions=actions, | ||
| init_kwargs={"alpha": 0.5, "threshold": 0.5}, | ||
| ) | ||
|
|
||
| def test_vs_random(self): | ||
| actions = [(C, D), (C, C), (C, C), (C, D), (D, D)] | ||
| self.versus_test( | ||
| axl.Random(), | ||
| expected_actions=actions, | ||
| seed=17, | ||
| init_kwargs={"alpha": 0.5, "threshold": 0.5}, | ||
| ) | ||
|
|
||
| def test_vs_random2(self): | ||
| actions = [(C, C), (C, C), (C, C), (C, C)] | ||
| self.versus_test( | ||
| axl.Random(), | ||
| expected_actions=actions, | ||
| seed=3, | ||
| init_kwargs={"alpha": 0.5, "threshold": 0.5}, | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Include
{self.momentum}in the name as well for completeness.