In a survey of 2,500 members of Springboard UK, the market research experts, on behalf of Vision Critical, half of respondents (51%) reported being fairly or very uncomfortable at the prospect of mobile payments.
The survey's findings will come as a blow to companies such as O2 and Orange, which are launching mobile payment systems later this year.
Women were particularly sceptical, with just 21% saying they would be "comfortable" making mobile payments. In men, this increased to 33%, but jumped to 41% in men aged 18 to 34.
Mike Stevens, head of research at Vision Critical in London said: "A lot of people are clearly uncomfortable. There is a perceived security risk that will need to be addressed as services are rolled out. Banks could play a strong role here – they are generally more trusted by their customers than other players in the mobile payments game."
Respondents also had views about how money would be debited from their handsets.
A total of 41% favoured funds being debited directly from a bank account, while 20% wanted a dedicated "mobile wallet".
The research suggests that mobile payments would drive down the use of cash more than the use of credit or debit cards.
Some 33% said they would spend a lot or a little less with cash and debit cards, while 22% said they would spend less on their cards.
The most encouraging news for those planning mobile payments is that users of smartphones are the most enthusiastic about making mobile payments.
IPhone users lead the way, with 48% "very" or "fairly" comfortable with making mobile payments, with figures for Blackberry and HTC users a little lower, but still much higher then those who do not use smartphones.
O2 and Orange will launch mobile payments systems later this year, with Apple believed to be considering using iTunes for mobile payments.
YouGov research published last month also suggests consumer reluctance, with 37% of those surveyed found to be wary of making mobile payments, but incentives such as loyalty driving use.