Sadly enough, unless you have really small breasts, they will droop in time. It is natural and it cannot be evaded. Why does this happen?
Breasts do not have muscles. Chest muscles are something else, the both men and women have it, but they are not a part of breasts. Breasts are fat, ligaments, glands, skin and connective tissue. As they have a certain weight and form a protruding part of your body, gravity will pull on them. Breasts hang from skin and ligaments, and constant pull of gravity will gradually stretch both skin and ligaments. Speed of this process depends on your genetics (how tough your skin and ligaments are), size (weight) of your breasts and on your age, as skin and ligaments loose elasticity with age regardless of strain. Generally, you can expect perky breasts only while you are a teenager.
The empty sack effect
Another process can make breast seem saggy and empty. Tissue inside the breast can shrink. This can include either mammal (milk) glands, which shrink after weaning, in menopause, or after pregnancy if there is no breastfeeding at all, or fat, which melts away if you loose weight. Ligaments and skin frequently don't retract as readily, which makes breasts look deflated, pumped out. Note that breastfeeding by itself has nothing to do with sagging breasts.
Being active, as in sports, makes your breast bounce and adds to stretching. Of course, wearing a good sports bra, which holds the breasts firmly in place, reduces the impact of this. Wearing a bra also slows breast sag as some of the breast weight is supported by shoulders via bra straps, but it does not prevent it entirely.
What do others think?
Breast size and shape in women and penis size in men - has there ever been a more important question in the whole history of mankind? When you look at the magazines, TV, commercials, what do you see? Women with what is deemed to be a "standard issue" breast. Truth is, you see women whose breasts have, in vast majority of cases, been enhanced by plastic surgery and carefully re-touched in an image editing program, in order to create a perfect or near perfect image. Well, what's in the can never looks so good as depicted on the can. Your breast looks different than those nowhere-near-reality boobs on the supermodel or actress? That pair of breasts on a girl that you saw on the street the other day looks so good, just in the right shape and position? Well, maybe she found the right bra?
Anyway, most men do not care. They look at you as a whole, not as a set of parts, and your breast might be less-than-perfect, but will serve their purpose anyway.
Of course, you can go and get a boob job, lift them, insert implants and whatnot. But you can end up with ugly and obvious scars, oddly (worse) positioned breasts than before, and results are not permanent, breasts will sag again. It is, after all, what you decide to do.