Link buying is like being a politician. You never admit the real reason for doing it. To touch on relevancy - of course relevancy should and does play a role, but even natural inbound links, come in from non-relevant sites. I might have a blog about nothing but design, yet link to a local plummer because I just used him and thought he was great.
I think the main thing to ask yourself when link buying, is would you ever naturally receive these types of links.
1. Buy links from pages that are relevant, but don't always rule out non-relevant sites. If it's not relevant, it will be ok, just won't have as big of an impact (of course other factors play into this as well -
PR, # of pages, etc.).
2. Don't always use the same anchor text. I don't think any website would ever receive natural link backs with the anchor text always being the same.
3. Tell everyone that your intentions are true, and you simply hope to receive more traffic and exposure by having these links (what is SEO...?)