The gain-loss asymmetry, observed in the inverse statistics of stock indices is present for logarithmic return levels that are over $2\%$, and it is the result of the non-Pearson-type autocorrelations in the index. These non-Pearson-type correlations can be viewed also as functionally dependent daily volatilities, extending for a finite time interval. A generalized time-window shuffling method is used to show the existence of such autocorrelations. Their characteristic time scale proves to be smaller (less than $25$ trading days) than what was previously believed. It is also found that this characteristic time scale has decreased with the appearance of program trading in the stock market transactions. Connections with the leverage effect are also established.