Does anyone remember the episodes of I Love Lucy where she'd trick Ricky into thinking it was her birthday so many times that she forgot it herself? Well, not to sound manipulative or anything, but you could do that. I mean, say, "Thanks for the gift honey." And then the next day, remind him that it's your birthday. Then when he says he's already given you a gift, say you thought that was a simple "I love you" type thing.
Interestingly enough, I think I'd be the person to forget what day is someone's birthday. I have a habit of staunchly reminding myself days before the event, and then when it actually comes, I totally forget.