You either have a fluid leak in your brake system or your master cylinder is defective. Either way, your brakes need immediate attention.
If the brake warning light is on, you most likely have a fluid leak. Your vehicle may not be safe to drive in this condition! You should have the brakes inspected as soon as possible to determine where the fluid is leaking (usually a hose, brake line, brake caliper or wheel cylinder) so the necessary repairs can be made.
If the brake warning light is not on, it does not necessarily mean you do not have a leak. The warning light only comes on when there's been enough fluid loss to create a pressure differential between the two sides of the hydraulic system that actually apply the brakes.
The brake system is divided into two hydraulic circuits. On most rear-wheel drive vehicles, it is divided so one circuit applies the front brakes and the other applies the rear brakes. On front-wheel drive cars and minivans, the system is usually split diagonally. One circuit works the right front and left rear brake, and the other works the left front and right rear brake. This is done for safety purposes so if one circuit loses all its brake fluid and fails, the vehicle will still have one remaining circuit to apply two wheel brakes.
A quick way to check for leaks in either circuit is to simply check the fluid level in the master cylinder reservoir. The reservoir is divided into two chambers (one for each brake circuit). If one chamber is unusually low or empty, there's a leak somewhere in that circuit. The brakes should then be inspected to check for fluid leaks. Wet spots around hose or line connections, or fluid leaking from a disc brake caliper or drum wheel cylinder would indicate a serious problem that needs immediate attention.
If the brake warning light is not on and there are no apparent leaks, then the master cylinder may be worn or leaking internally allowing the pedal to slowly sink when pressure is applied to it. This type of condition will be most noticeable when holding constant pressure against the brake pedal at a stop light. If the pedal sinks or requires pumping to keep the car from creeping ahead, the master cylinder needs to be replaced.
On some vehicles with rear-wheel antilock brake systems (ABS), it's also possible that a leak in the ABS unit may cause a similar sinking pedal condition.