It's worse if you open your turn then forget you did.  
 
I almost completely missed mine, cause I began it, but then walked off and got distracted. 
Thankfully I remembered ten minutes before the time was up.   
 
There's a price to pay, but it's worth it if you hate it when people hold.