Alex, I really do understand- I generally don't care for this sort of genre either. I got sucked into the Potter vortex courtesy of a co-worker, who asked me way back when the first movie came out if I was going to see it. I scornfully responded no, at which point she shreiked "You haven't read the books!" I was a bit proud of the fact that I hadn't succumbed to to Pottermania, and I said as much. When I arrived at work the next day, I found she had deposited the first three books on my station. That weekend I started reading book one, and by Monday I'd read all three and was jonesing for the fourth.
As far as 'annotating', you are not far off the mark. The way the books are laid out there really is no other way to do it. If you can read the books and then figure out how to do otherwise, you may want to contact the producers. There are so many intricate plot twists and turns- some that are for entertainment purposes, some for red-herring duty and others that are essential to the ending, which has already been written. It really is fantastically involved and it is the writers somewhat thankless task to incorporate esential elements and to weed out the unnecessary, thereby facing the wrath of zillions of fans. (I'm still convinced that having Harry give the Weasely twins his Tri-Cup winnings to start their gag shop is a better ending, as it gives them a platform to continue on as the comedic relief after their departure from the school and foreshadows the coming darkness).