Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FedProx implementation #837

Draft
wants to merge 4 commits into
base: develop
Choose a base branch
from
Draft

FedProx implementation #837

wants to merge 4 commits into from

Conversation

JulienVig
Copy link
Collaborator

Closes #802

@JulienVig JulienVig changed the base branch from develop to gdhf November 28, 2024 16:00
@JulienVig JulienVig force-pushed the gdhf branch 4 times, most recently from 66af668 to 75a67ec Compare December 3, 2024 15:08
Base automatically changed from gdhf to develop December 3, 2024 15:17
@JulienVig
Copy link
Collaborator Author

@tomasoignons I rebased the branch onto the latest develop commit. Here is a breakdown of the current commits:

  1. tfjs allows training via multiple functions: fit, fitDataset, trainOnBatch. We were using fitDataset on each batch so I switched to using trainOnBatch to be consistent and simplify things
  2. We used to compute logits twice for each update, once for the loss and once for the accuracy. This is very inefficient but not super practical to fix. I implemented one but feel free to suggest a better refactoring
  3. A temporary commit, I override the training loss by implementing it by hand to then implement the fedprox loss. This commit saves the codebase state where the custom training loss behaves the same as the default loss (make sure this is the case)
  4. Fedprox loss building on the previous commit, I didn't have the time to test it in depth so this is up to you

As we were discussing, the fedprox sketch has been hard-coded and doesn't allow users to choose which algorithm they want between fedavg and fedprox, and potential future algorithms like Mime.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement FedProx to improve robustness to heterogeneity
1 participant