Oh man, I was so hoping this was not going to be the end of the story for Charlie. I can't offer any advice to you both, but whatever you decide, you have my support. I'm here, looking at my two sweet kitties and so thankful they are healthy and happy.
I know you will do what is best for you and Charlie, who looks to be a real affectionate fella. If, ultimately, the decision is to be the sad ending for him, you can take heart in knowing that you have provided him with love and care and made his time with you to be likely the happiest he has known. Whomever his former owners are, shame on them and may the karma they deserve come back to strike them.
I'm so sad now, going to go and snuggle with my little ones and think of you and sweet Charlie.
Donna