Setting tolerance when using Adaptive weights

[Abhishekc95]

1. If adaptive weights are used, wouldnt the total loss get affected? Say we set a tolerance of 1e-5, will the algorithm reach this if the weights keep changing ?