Similarly we've had geroff-my-lan' complaints when cycle camping with the children.
If we hadn't pitched, we moved on, and where we'd pitched (common land, not even owned by the complainant) we moved on the next day, after inviting the complaint to call the police (they didn't, or the police declined to attend). Indeed, on that occasion we took our tourist pound completely off the island, which was the opposite of the complainant's desire (they wanted us in a B&B, paying into the local economy, which ignored the fact that, unlike car tourism, we had to shop local to feed ourselves!).
People be people, even with against the context of access legislation.
Robert