The Dutchman, who is now a coach at Eredivisie club FC Twente, said fans directed racist chants and monkey noises at him regularly during his career.
"Wherever I played I had it, as a player, it was not big but in every part of the world you have some people who are active in racism. In England as well, yes for sure," Kluivert said.
Capped 79 times by The Netherlands, Kluivert said his prerogative was often to ignore the abuse but said in light of recent events, including former England captain John Terry's racial abuse of QPR's Anton Ferdinand, more needed to be done.
"As a player as long as you don't react and give them your attention then it will probably be less effective," Kluivert said.
"If you give them attention they think they are important and that can be a negative towards them.
"If you had the ball they were making monkey sounds or something like that but I was a player and it didn't really affect me but it is not nice to hear it.
"Racism is a difficult issue, I hope everybody can stick together and get rid of racism or the people who are active in it.
"It will be very difficult because that small part who are involved are a strong part who really stand for their feelings and it is very difficult to get every individual thinking about racism and how it affects the ones who are involved.
"We have to get rid of it, I think it is people who are really stupid and short minded and don't have a heart."