I don't think any age is too young, but here is the one thing I would worry about if I was taking a young child and it is this: What would happen if DH and I got sea sick or became ill. I would think it would be very difficult to care for a young child in a stateroom, at sea, if we were very, very ill throwing up , etc. It wouldn't be a picnic to be in a hotel room sick at DW either, but at least you could take the sea sick equation out of it. I am someone who always thinks of every worst case scenario and that one would be enough for me to wait a while.
Having said all of that, though, the chances of you going and everything being 100% magical are quite high! I say trust your gut and go with that. Have you ever cruised before? If you have an know what to expect, then I would think that would make your decision easier.
Good luck!
When we went in January the youngest was just 12 weeks old. That cruise was booked before they changed to the new 6 months/1 year minimum age policy. It was my wife and I, 4 adult children, 2 husbands, 1 long time boyfriend and 6 grand kids (12-months to 6 years). There was some sea sickness involved, but that was the 4 and 5 year olds. Multiple adults around to help out. We had a great time.