Froome lost nearly a minute to his team-mate on the opening stage of the Tour when he crashed in a field five kilometres from the finish, and gave up another five seconds when he was gapped slightly on the Mur-de-Bretagne on Thursday.
Spanish veteran Alejandro Valverde (Movistar) led a select chasing pack over the line for third place in the 181km stage from Brest, three seconds back, with BMC's Van Avermaet finishing safely in the mix to retain his lead in the general classification. "It was good to finish second back then but this time I was coming for victory", said Martin.
Riders twice went up the Mur de Bretagne, or "Wall of Brittany", which the local cycling-crazed Bretons affectionately refer to as their smaller version of the famous Alpe d'Huez.
"I'd like to thank Greg (Van Avermaet) because he did me a favour when he went for it, I don't know if he did it on objective but he dropped a lot of people", said Sagan, who has tightened his grip on the points classification's green jersey.
"I was following quite good wheels but it was choppy".
Cavendish, who is chasing a 31st stage win to close in on the Tour record of 34 held by Belgian Eddy Merckx, said a late collision hampered his finish but conceded other teams seem in better form.
"It was a boring stage, no wind, no stress just boring, I was talking to everybody", he said.
Bardet would lose 31 seconds on the stage in a blow to the Frenchman's general classification hopes, while 2017 Giro d'Italia victor Dumoulin lost 53 seconds. "Third is okay, if I win what do I get except an extra point or two?" The legs were just there all the time, I don't know if it was adrenaline or not.
Van Avermaet, wearing the yellow jersey, then pulled to the front in a bid for the stage win as the finish line approached but peaked too early.
There was further fallout too from Tom Dumoulin's time penalty, imposed after Thursday's finish to Mûr-de-Bretagne, when Froome's rival slipstreamed his team vehicle after suffering a broken wheel five kilometres from the finish.
"I tried to win the stage but it was pretty complicated", Van Avermaet said.
On the same long straight uphill three years ago, Martin finished second and he has been hinting for the past few days that he had fireworks planned this year.
This was the first summit finish of this Tour.
A long day - it's the longest stage of the race - flat day with even the one solitary climb of the Côte de Buisson de Perseigne (Category 4, 1.5km at 3.9%) not really counting as Toms Skujiņš can't be dislodged from the polka dot jersey.