Codename-RVC-Fork-V3.0.3

Release of the version: 3.0.3

Notes:

• New logging mechanism for losses: Average loss per epoch logged as the standard loss,
  and rolling average loss over 5 epochs to evaluate general trends and the model's performance over time.
  Both work for each metric individually.

• Features a different optimizer: RAdam ( Rectified Adam )
  ( More init. stability and compared to AdamW, doesn't require using / configuring warmup. )
  Most likely better convergence / generalization on average, compared to plain AdamW without a warmup.

• Few tweaks for the ui, some formatting changes and generally, updated in-line with Applio main repo.

PS. Do not turn on warmup. RAdam handles all of that on it's own. It's kept in for the sake of future AdamW's optional usage.