Login or register (free and only takes a few minutes) to participate in this question.
You will also have access to many other tools and opportunities designed for those who have language-related jobs (or are passionate about them). Participation is free and the site has a strict confidentiality policy.
Explanation: That's right, it's an expression (see reference for example of use). But I think you interpreted it right. He's using the expression to mean even if the town is "theirs" (i.e. theirs for the picking), he doesn't want it.