I mean, you're not required to do anything in the Wilderness, though. Going there is completely optional.
I completely understand that having someone camp you in the Wilderness and preventing you from doing anything there is frustrating, but if that happens, there's a plethora of other content in the game that doesn't involve any PvP.