Pirc considered Sports Direct's response to the report as "insufficient".
House of Fraser went bust with debts of almost £1 billion and was bought out of administration by Mr Ashley with promises to create "the Harrods of the high street".
Sports Direct owner Mike Ashley bought House of Fraser out of administration for £90million last month.
Sports Direct owns just under 30 per cent of Debenhams, close to the threshold at which it must launch an official takeover bid. "We might have our hands full", he said.
Shares in Debenhams fell 7% on Thursday after Sports Direct issued a statement to the market saying it had no intention of making an offer.
"If there's opportunities in the future, then we'll be in a position to take advantage of them".
Mr Bentley later said: "I was asked a general question about whether or not we discuss our strategic investments and, in particular, Debenhams to which I replied in the positive".
Outgoing director Simon Bentley said at Sports Direct's annual general meeting in London that the board had looked at the option of merging the two chains.
Mr Hellawell said: "Having overseen significant improvements in the working practices and corporate governance of the company, which includes a refresh of the board, now is the right time for me to step aside".
ISS said investors should not support the re-election of both Mr Ashley and Mr Hellawell due to the firm's "apparent unwillingness to fully address the concerns of independent shareholders".
The new chair of Sports Direct is David Daly, who joined the board past year after a long career at Nike.
Investors have been warned about Mr Ashley's power over the company because he is the Sports Direct's majority shareholder, and owns 61 per cent of the group. Almost 37% of independent shareholders who voted, wanted to block his reappointment.
Glass Lewis, the investor advisory group, accused the board of "poor governance" and urged investors to vote against the re-election of Hellawell, Bentley and Ashley.
Mr Murray has been charged with leading the overhaul of Sports Direct's property portfolio, part of Mr Ashley's plan to make the company the "Selfridges of sport".
The chair of the Local Authority Pension Fund Forum welcomed the exit of Hellawell and Bentley and called on Daly to commission an independent review of Sports Direct's employment standards and corporate governance.