Not everyone can afford to stay at home, even if they want to. Needing to bring in an income for your baby is nothing to be ashamed about, and it is a pretty darn good reason to go to work. If you are one of those who would love to stay at home, but you do need an income, you could look into working at home, though.
Being at home with a small baby, a toddler, or an older kid all day long can do funny stuff to your brain. Many parents feel that taking time out, and spending a good portion of the day in a productive environment with other adults, makes them a better parent and a happier person.
Staying at home, even for just a few years, impacts your career in the long term. You might make less in the long run, miss out on promotions, and lose touch with your industry. If you are committed to having a professional career, being a stay at home parent may not be an option for you even for a few years. Between financial benefits, a good work-life balance, and the long term picture, there are some excellent reasons to have a job outside of the home. And remember, whether you are a stay at home parent or a work out of the home parent, there is no reason at all to justify your choice to anyone!