By the time you ask for references, you should be very close to a decision. The key points about a reference check are;
- The company should be a similar size and in a similar industry
- They should be running the same modules as you are considering
- They should have been live for at least six months
- The person you are speaking to should have been involved in the implementation
- The person you are speaking to should have some hands on experience of the system
The last two points may mean you meet with two or more people.
The Vendor should not attend the meeting as it may stifle the conversation. Ask that if the Vendor is to attend, they make the introductions and leave. Sometimes the only way to do a reference check is by phone. Make sure you have a clear block of time as does the other person. You do not want to have to rush off to a meeting or deal with other phone calls while you are speaking.
Take notes from the meetings, and if necessary update the Solution Impacts document. Things may come to light during the discussion that are solution impacts.