The common symptoms of fracture are pain, swelling and discoloration. It depends on the precise location and severity of the injury, also broken toe may get infected if there is a broken skin. So depending on these factors decide whether to go to emergency. But for any possible fracture you should see doctor as soon as possible.