The retailer was hit by an 8.7% dip in sales at its flagship Oxford Street store and a 10.3% drop at its Peter Jones store in Sloane Square for the week ending on the first day of Olympic competition, 28 July.
Andy Street, managing director of the official Olympic department store, said that though the chain's sales rose overall growth had slowed as demand was being "sapped" by London 2012.
The figures follow central London attractions and theatres reporting a slowdown in business in the lead-up to and beginning of the Olympics.
Bernard Donoghue, director at the Association of Leading Visitor Attractions, said: "We’ve seen about a 30%-35% reduction in the number of overseas visitors to central London attractions and that’s partly because the kind of visitor you get in an Olympic year is different to any normal year and it’s also partly because people are thinking it’s going to be crowded and hectic here in London."
In response, London & Partners, the official promotional organisation for London, has partnered with the West End Marketing Alliance to launch a six-figure campaign today (3 August) to boost trade in the capital.
The campaign is targeting Londoners and Home Counties residents with press ads in the Metro and Evening Standard and comes after New West End Company chief executive Richard Dickinson, one of the partners in the alliance, called for a "new approach to marketing".
He said: "I think transport is one of the issues they did have to face and I think the authorities have done a very good job of managing demand but I think now is the time for a more sophisticated approach on the marketing front because we’ve got one and a half million people working from home and a lot of people taking summer holidays early.
"We just need it to be a bit more measured and do more marketing around the West End being open for business."
Nevertheless, Transport for London (TfL) and Culture Secretary Jeremy Hunt have dismissed claims that central London has become a "ghost town" due to the Olympics and high-profile 'Get ahead of the Games' marketing campaign.
A Transport for London (TfL) spokesman said: "People are not staying away from London. In fact, Tube passengers are already up 10% on what we would normally see at this time of the year and journeys to the key West End stations are up 12%."
However, data from Experian Footfall reveals that footfall in West London was down 4.53% on Monday and Tuesday.
Experian also reports that visits to retail websites increased by 15% year-on-year on Tuesday, suggesting consumers are opting to shop online rather than brave the transport network.
Prime Minister David Cameron is calling on shoppers to return to the capital now it has been proven the transport system can hold up amid the influx of Olympics visitors.
Cameron told Sky News: "Clearly there is a challenge now though to say to Londoners [that] London's working well, it's open for business, come back into the capital, come and shop, come and eat in London's restaurants and let's make sure that all of London's economy benefits from this."
A TfL spokesman said there were no plans to drop its 'Get Ahead of the Games' marketing to lure shoppers back to central London because the message is to stagger journeys and avoid hotspots rather than avoid travel altogether.
In comments likely to anger retailers, the culture secretary said the Olympics had turned the West End into a ghost town were "absolute nonsense" during an interview with the Evening Standard.
He added that "some West End businesses have done extremely well because they’ve marketed on the back of the Olympics".
His comments come after numerous reports of businesses being prevented from marketing around the Olympics because of the stringent regulations put in place by Locog to protect the interests of the official London 2012 sponsors.
Stories include a newsagent in Stoke Newington being forced to take down balloons and bunting they had put up to celebrate the Olympic Torch passing through the area.Follow @mattchapmanuk