Just because he says "fathers," he does not exclude mothers. Paul simply addresses the party with the overall responsibility.
Even though it is not directly stated, we must remember that God consistently teaches that the strong are responsible to care for the weak. In this context, the parents are strong, the children are weak. However, parents must not depend upon their size and strength to demand respect, but should strive to earn it through strength of character, wisdom, and clearly expressed love.
The Greek word translated "bring them up" at first meant merely providing bodily nourishment. Through time its usage extended to include education in its entirety since bringing up children obviously is more than just feeding a child food. "Training" is more correct than the weak "nurture" used in the KJV. The Greek word means "to train or discipline by repeated and narrow exercises in a matter." It implies action more than intellectual thought and corresponds to the word "train" in Proverbs 22:6, which means "to hedge" or "narrow in." Thus God expects parents to train their children to walk the straight and narrow way rather than allowing them to wander aimlessly about on the broad way.
Paul adds in Colossians 3:21, "Fathers, do not provoke your children lest they become discouraged." To some degree, all children resist their parents and what they represent and teach. How parents overcome it is Paul's concern. These verses testify that many parents strive to elicit their children's obedience and respect in the wrong manner.
The wrong way provokes embittered, fretful, defensive, listless, resentful, moody, angry, or sullen children. Paul counsels not to challenge the child's resistance with an unreasonable exercise of authority. Correction is necessary, but a parent must administer it in the right spirit, counterbalanced by lavish affection and acceptance. A twig should be bent with caution.
Firmness does not need to be harsh nor cruel. Punishment should never be revenge nor dispensed just because the parent is irritated. Severity only hardens the child and makes him more desperate. If a parent does not use his authority justly, he cannot expect a child to be respectful. It does not happen automatically.