LSTMs provide us with a large range of parameters such as learning rates, and input and output biases . Hence, no need for fine adjustments. The complexity to update each weight is reduced to O(1) with LSTMs, similar to that of Back Propagation Through Time (BPTT), which is an advantage.