Full Implementation of the Captum-Optim Module #1545
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Merge master branch into optim-wip
* Removed test version checks for versions below 1.6.0. * `AssertArrayAlmostEqual` -> `AssertTensorAlmostEqual` * General linting changes / fixes.
Optim-wip: Merge master branch into optim-wip
…lename (pytorch#822) * Fix the Channel Attr notebook image download & metadata filename * Fix ChannelAttr notebook
* Miscellaneous Changes & Fixes * Add missing docs * `get_model_layers`, `collect_activations`, `Conv2dSame`, & `get_neuron_pos` were all missing documentation. * Fix `utils/image` tests and add missing `_dot_cossim` tests * Fixed `image_cov` and the dataset tests. * Renamed `utils/image/dataset.py` to `utils/image/test_dataset.py` as the lack of a `test_` prefix was causing the tests not to be run. * Renamed `utils/image/common.py` to `utils/image/test_common.py` as the lack of a `test_` prefix was causing the tests not to be run. * Added missing `_dot_cossim` tests. * Fix `nchannels_to_rgb` & `Direction` assert * Moved the `hue_to_rgb` function outside of `nchannels_to_rgb` for JIT support. * Fixed `nchannels_to_rgb` and `hue_to_rgb` functions. * Fixed `Direction` loss objective assert. * Fix `image_cov` & related tests * Fix conflicts for common -> test_common.py * Merge updates from optim-wip branch * Fix test error * Fix cossim test
…orch#831) * Add explanations to losses * Add argument documentation for losses * Lint fix
…edImage (pytorch#833) * Add new StackImage parameterization & JIT support for SharedImage * Added `SimpleTensorParameterization` as a workaround for JIT not supporting `nn.ParameterList`. It also helps `StackImage` support tensor inputs. * Added JIT support for `SharedImage`. * Added new parameterization called `StackImage`, that stacks multiple parameterizations (that are can be on different devices) along the batch dimension. * Fix test version checks * More tests & new AugmentedImageParameterization base class * Added `AugmentedImageParameterization` class to use a base for `SharedImage` and `StackImage`. * Removed `PixelImage`'s 3 channel assert, as there was no reason for limitation. * Added tests for `InputParameterization`, `ImageParameterization`, & `AugmentedImageParameterization`. * Add JIT support for SharedImage._interpolate_tensor * Added JIT support for SharedImage's interpolation operations. * Unfortunately, JIT support required me to separate SharedImage's bilinear and trilinear resizing into separate functions as Union's of tuples are currently broken. Union support was also a newer addition, so now SharedImage can support older PyTorch versions as well. * Add dim variable to StackImage * Added the `dim` variable to `StackImage` so that users can choose what dimension to stack the image parameterizations across. * Fix test version * AugmentedImageParameterization -> ImageParameterization * Remove unused code
* Improve ModuleOutputsHook, testing coverage, & fix bug * Added the `_remove_all_forward_hooks` function for easy cleanup and removal of hooks without requiring their handles. * Changed `ModuleOutputHook`'s forward hook function name from `forward_hook` to `module_outputs_forward_hook` to allow for easy removal of only hooks using that hook function. * `ModuleOutputHook`'s initialization function now runs the `_remove_all_forward_hooks` function on targets, and only removes the hooks created by `ModuleOutputHook` to avoid breaking PyTorch. * Added the `_count_forward_hooks` function for easy testing of hook creation & removal functionality. * Added tests for verifying that the 'ghost hook' bug has been fixed, and that the new function is working correctly. * Added tests for `ModuleOutputsHook`. Previously we had no tests for this module. * Make hook fix optional * Remove hacky hook fix * Lint: Fix import order
…o ImageTensor (pytorch#839) * Add better colorspace support, image grids, & user agent to ImageTensor * Added color space support to `save_tensor_as_image` & `ImageTensor.export`. * Added image grid creation support to `ImageTensor.export` , `ImageTensor.show` , `show` & `save_tensor_as_image` via a new `make_grid_image` function. * Added user agent to `ImageTensor.open` as sites like Wikipedia require user agents. * Add description to make_grid_image tests * `nrow` -> `images_per_row` * Remove test description It's no longer required now that the images_per_row variable was renamed. * Add missing tests * Fix test
* Ensure testing coverage is as high as possible. * Simplified code with new `rmodule_op` function. * Removed the NumPy import from loss testing.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
So this project got left on the back burner for a while but I've brought it back. The original plan was to either merge it into Captum or spin it out as a separate library under the PyTorch umbrella. The project was almost ready for release before it was paused, and I have since pushed it closer to full release quality.
The ReadMe for the module can be found here: https://github.com/ProGamerGov/captum/tree/master-optim/captum/optim
This PR contains all the following PR merged together:
#983
#968
#967
#966
#965
#961
#960
#957
#955
#955
#951
#949
#946
#945
#943
#927
#828
The module was developed based on this initial starting PR: #412