This is a serious problem. Your project exists to serve somebody, and your success is directly proportional to understanding what they want, so that you can build it. You need as much communication bandwidth between the programmers and those people as possible.
If you have one customer and they can’t come to you, consider going to them. Can you co-locate on their premises? Can a fragment of your development team visit the customer for a few days of intensive collaboration on requirements? The quality of information you get from a customer will probably be better if the developers go to where the customer is, due to “knowledge accidents”. The customer may be distracted in their own office by their day to day work, but to the extent that the developers can witness and understand the customer’s daily life, this can actually be a good thing.
However, many software companies have more than one customer, or are trying to build something for multiple prospective customers. They may not consider your product important enough to dedicate someone to be your onsite customer, nor to let you colocate your team on their site.
One option is to choose a single customer and treat them as though their requirements are typical of others. There is a risk that their requirements are atypical, but in exchange you avoid the risk of building something that doesn’t satisfy anyone’s requirements enough for them to buy it.
In any case, when you’re hoping to sell to a large number of customers, you still have to focus on a small number of customers (a particular type of company, a vertical market, etc.) in order to be able to prioritize your efforts.
In the case where no customer is willing to co-locate with you or allow you to co-locate with them, the key is to reduce the number of layers of communication between the customer and development team. The usual substitute for an actual customer is an internal employee acting as a Customer Proxy, who talks to the real customer very often, preferrably in person, preferrably while demoing the current software.
The thing to avoid is an arrangement where multiple layers of opinion and guesswork sit between the customer and the developers. Perhaps sales and marketing folks have gathered wishes from prospective customers, and customer account reps have gathered feedback and wishes from existing customers, and they filter and reword and interpret everything before giving this information to the product manager. Then the product manager synthesizes this into a requirements document and hands this down to the development team, and acts as the Customer, answering requirements questions without consulting a real customer. This is broken. The fix is to have the product manager act as a true Customer Proxy rather than a Mock Customer: the synthesized requirements and their prioritization must be validated with multiple customers to ensure their correctness, and questions about requirements must be relayed to real customers in order to obtain a real answer. And if you can get a customer to come in and visit once in a while (or if you can visit them), by all means do so.