The ferry terminal is right in the town and that area is not somewhere locals tend to go after about 10pm - too many drunks spilling out from bars, clubs etc. Would recomend you stay out of town and then drive down for your crossing. If you decide to do this, watch out for trafic hold-ups. On a sunny weekend the road over the ridgeway from Dorchester gets horribly snarled up from about 9am - also the big new link road they're putting in doesn't help things.