We had to get up & move quick last year. We were parked in a town in Spain - looked a decent area, but we had the bikes on the back. 1 am, & we heard quite a lot of noise - local teenagers just having a laugh, dossing as they do, but then we got a couple of bangs on the side of the van - ignored it, then sneeked a peek & realised there were a lot of them, & looking our way. Another bang on the side of the van - probably to see if there was anyone in before they broke in. Their faces were a picture when we pulled down the screens from the inside & drove off.
Another time we parked up in the early hours & a bit later a car full of "people" pulled up at the side of us - we didn't like the look of them, so again took down the inside screens & moved on.
Personally I feel much happier if I can move away quickly. OH thinks I am too cautious, but to me its far better safe than sorry.