HiTL Dashboard - Best Model Loss & Accuracy Visualization Backend

[mialsy]

Description

• Added backend implementation to send best model's loss and accuracy via socket
  • get_best_model_loss_acc_by_id
    • get loss and accuracy from a model log for a specific run's best model, the loss and accuracy infomation is reterived by crawling the best model's log file
    • input:
      • batch id of a specific run
    • output:
      • a dictionary containing:
        • epoch loss and accuracy
        • text_span loss and accuracy
      • or an error code indicating the best model log cannot be find
• Added utils for reading model log to extract loss and accuracy data
• Added a frontend block for viewing loss and accuracy (right now only showing data in the alert, will create a new PR for visualizing the accuracy and loss in a graph)

Type of change

Please check the options that are relevant.

• Bug fix (non-breaking change that fixes an issue)
• Proposes a change (non-breaking change that isn't necessarily a bug)
• Refactor
• New feature (non-breaking change that adds a new functionality)
• Breaking change (fix or feature that would break some existing functionality downstream)
• This is a unit test
• Documentation only change
• Datasets Release
• Models Release

Type of requested review

• I want a thorough review of the implementation.
• I want a high level review.
• I want a deep design review.

Before and After

• Demo:
  • 

Testing

Manually tested

Checklist:

• I have performed manual end-to-end testing of the feature in my environment.
• I have added Docstrings and comments to the code.
• I have made changes to existing documentation where needed.
• I have added tests that show that the PR is functional.
• New and existing unit tests pass locally with my changes.
• I have added relevant collaborators to review the PR before merge.
• [Polymetis only] I ran on hardware (1) all scripts in tests/scripts, (2) asv benchmarks.

[ethan-carlson]

LGTM, thanks!

[facebook-github-bot]

Hi @mialsy!

Thank you for your pull request.

We require contributors to sign our Contributor License Agreement, and yours needs attention.

You currently have a record in our system, but the CLA is no longer valid, and will need to be resubmitted.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at [email protected]. Thanks!