Skip to content
This repository was archived by the owner on Mar 20, 2026. It is now read-only.

UnitY implementation#4670

Merged
hirofumi0810 merged 44 commits into
mainfrom
ust_unity
Oct 7, 2022
Merged

UnitY implementation#4670
hirofumi0810 merged 44 commits into
mainfrom
ust_unity

Conversation

@hirofumi0810

@hirofumi0810 hirofumi0810 commented Aug 26, 2022

Copy link
Copy Markdown

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
  • Did you read the contributor guideline?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?

What does this PR do?

This PR supports new speech-to-speech translation models based on two-pass decoders.

  • UnitY (text->unit)
  • Translatotron2 (text->spectrogram)

These are related to this PR

In addition to supporting new models, I also made several updates that may affect other tasks,

  • Support dual cross-attention for Transformer decoder

Each of them is necessary to obtain the best result with UnitY.

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

Loading
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants