I didn't get told I shouldn't, just a few "you're still cycling?" comments. I didn't have a balance problem when my bump got quite large, and what actually stopped me cycling was my pelvis going nuts (Symphisis Pubis Dysfunction) which made any movement involving leg separation painful and ill-advised.

I found I cycled more defensively, as in leaving more space between me and hazards, and the scariest moment was someone opening a car door into my path on the fastest bit of Castle Hill downhill: but because I was cycling defensively I was able to avoid them rather than have a nasty fall.