Well, if that bothers you keep in mind that there are already a century's worth of flourescent light bulbs in landfills, generally of types that have way more mercury than household CFLs. Mercury-using flourescent bulbs are not new, just the smaller home uses.
Also, if you use a new model CFL until it burns out there will be about
0.2 milligrams (0.0002 grams) of Mercury released into the soil or air when it gets broken at a landfill. According to that link, if every single CFL sold was used to lifespan and then broken it would amount to a 0.1% increase in human mercury emissions. As it also notes, while a CFL puts 0.2 milligrams of mercury into the environment when broken at the end of its life, it saves 4.5 milligrams of mercury from going into the environment due to reduced power use.
So, I think you can safely say that in terms of environmental mercury, CFLs are very much the obvious choice.
Then on top of that, it is technically (though I doubt their is much recognition of it) illegal to put CFLs into the California landfills. But you can do your part to keep even that 0.2mg out of landfills by recycling your bulb
at a designated e-waste recycling center.