Pretty much exactly what I do. I don't like having it on. I was able to pull a second out of the quiver last year after blowing a chip shot over the back of a double split brow (never got that second shot, but the second buck the next day wasn't so fortunate :D ).
I have had to make follow up shots numerous times where I actually had to sit down to get another arrow. However, I still take mine off, but I do try to keep it positioned high enough for a follow up shot. Once I had a big ol doe on the last day of the season give me a shot behind my tree. I had her dead to rights except my dang quiver was hitting the tree and I couldn't shoot. That is the only time I ever had an issue.
