It's worse if you open your turn then forget you did.
I almost completely missed mine, cause I began it, but then walked off and got distracted.
Thankfully I remembered ten minutes before the time was up.
There's a price to pay, but it's worth it if you hate it when people hold.