The exact cause of thrush is not known. The candida albicans fungus is a normal part of everyone's digestive system, but thrush is an overgrowth of this. Thrush is not dangerous, but can be very uncomfortable indeed. Of course, you don't want your baby to suffer needlessly. So, if you suspect that your baby has oral thrush, because you noticed some of the symptoms described above (and in more detail in the linked article), seeing your baby's doctor as soon as possible is your best bet.
If necessary, your baby's pediatrician will take a cell sample from the baby's mouth and have it analyzed in a lab. Sometimes, doctors will recommend expectant management of thrush. Many mothers know from experience that thrush will usually go away by itself when not treated. But, thrush can be transferred back and forth between a mother and baby during breastfeeding, so it is certainly recommended that you keep this in mind and see your doctor, rather than just waiting for the fungus to disappear off its own accord.
If you have thrush on your nipples as well, a topical cream to treat thrush is needed for you as well. Symptoms of thrush on the nipples include cracked nipples and visible white patches. Thrush can be treated with these antifungal drugs (Nystatin is one example), or by internal drugs that are designed to treat the infection more thoroughly. The latter may be recommended after topical creams turned out not to work.