SDAG (Synchronous Accumulated Gradient Normalization) A hybrid of SSGD and AGN. This method averages gradients over the communication window but apply updates to the ps variables synchronously.