The Lions went ahead in the second half from the penalty spot after Rovers defender Grant Hanley had fouled Steve Morison and was then sent off.
Michael Keane nodded Blackburn level from a corner before bringing Simeon Jackson for a second penalty two minutes from time.
The Canadian converted from 12 yards, but two minutes into stoppage time substitute Josh King prodded the equaliser following a long throw from Tommy Spurr.
And having been pegged back twice from set pieces, Holloway was left fuming as his side remained in the Championship relegation zone.
He said: “We’ve let two set pieces in against 10 men, one of which was a free header, and I’m absolutely fuming.
“The group needs to be more organised and now is the time to step up to the plate.
“If somebody makes a mistake then somebody else should be there to head things away.
“We ended up just trying to be safe instead of just dealing with it (the long throw).
“No-one organised us and we needed someone like Danny Shittu in there who bellows at people.”
Holloway added: “We had five or six people go for the same ball, and it dropped for them and they scored.
“It is only organisation. “I don’t know whether its nerves because of the way the season has gone, but we have got to be braver.
“We’ve got to get out of this vision that everything is going to go wrong because I feel that the boys have got that a little bit.
“We’ve got to win five out of seven now but we should have won this one. I’m fuming.”
The Lions’ opener arrived in controversial circumstances when Hanley was harshly judged to have pulled back Morison inside the area.
After the match Blackburn manager Gary Bowyer bashed his head on the desk he was sat at when quizzed about the incident, with even Holloway having some sympathy with his opposite number.
The Millwall boss said: “Truthfully, I thought the first penalty was nowhere near as bad as two or three things I have witnessed recently.
“If it was a penalty then that’s a yellow card. I don’t know why the referee sent him off.
“It was a man to a man and Grant Hanley is a big, strong boy.
“I didn’t understand it.
“We can’t blame the referee for Saturday because he’s given us two penalties and we scored them both, which is nice and makes a change.”
Follow @NewsShopperSprt on Twitter for live updates from Nottingham Forest v Millwall on Saturday