Millwall’s meetings with Leeds are always heavily policed due to several clashes in the past between both sets of fans and Holloway thinks it is a shame the two teams still carry such a reputation.
He said: “Everybody has got rivalry with Leeds because they had a massive crew who would go around fighting people and so did we.
“I’m not proud of that and nobody should be, but we were good at it and so were they.
“That was when it was fashionable and it wasn’t just us. That is all we are remembered for and I think it is a shame.”
Millwall’s community work, such as dedicating Saturday’s game against Charlton to Jimmy Mizen, is often overlooked and Holloway wants to help change the image of the club.
The Lions boss said: “I’m really proud of everything about the place - what a fantastic club.
“We are honest, hard-working people and Millwall have always been that.
“They stand up for what they believe in, they won’t back down, they’re brave and they were strong in a time when football had a terrible, shameful behaviour about it.”
Holloway added: “I want us to be remembered for the way we play and the way we go about it and the passion that we all have. I have felt it immensely here.
“I don’t want people using football as a vehicle.
“We are going to go to Leeds and try and score and hopefully we can all talk about the performance that we put in.”
Follow @NewsShopperSprt for live updates from Leeds v Millwall on Saturday