There's the train and the bus. What you choose will depend on how early you need to be at the stadium. Do you have general admission, or assigned seats? If assigned seats, you can probably plan to take the train. I'm not that familiar with where it picks up, so you should explore the options
here.
The cheaper hotels will be in NJ, not NY. Do a search on Seacaucus, and search for hotels close to Giants Stadium. Keep in mind that just because the hotel is close doesn't mean you'll be able to walk from there to the stadium. There are a lot of busy roads and highways around there.