I would go and have a look around, because usually they would be close by and ask them. I gly drones for 10+ years and if it’s a sports drone, it’s annoying and they should be pretty close. If it’s a dji (slow moving around) they might be just nosy assholes.
I would give them the benefit of the doubt that they just fly there because like you said it’s in the middle of nowhere. But flying over a house or property on purpose is not only a dick move, it’s also dangerous annoying for all sides.
Finally a friend