Freedom of association works both ways
Suppose you want to deal with me but I don’t want to deal with you. Should I be forced to?
You might ask, “What are you talking about?”
Here’s a short list. Suppose you want to marry me, but I don’t want to marry you....