FK Rudar Prijedor was first established in May 1928 as FK Rudar Ljubija. Originally located in Ljubija, a small mining town, the team have endured a disjointed history, having ceased operations on several occasions, including during the Second World War and during the Bosnian war. In 1995, after the end of the war and following the break-up of Yugoslavia, the team reformed and united with OFK Prijedor. It was decided that the new combined team should be based out of the city of Prijedor.
The club's time in the old Yugoslavia was mixed, with Rudar regularly competing in the Yugoslav Second League and closely missing out on promotion to the country's top tier, the Yugoslav First League, on several occasions. Their best performance in the old Yugoslav Cup came in 1990, with the team eventually reaching the semi final stage of the competition.
Since the dissolution of Yugoslavia and the merger with OFK Prijedor, the team have risen to new heights. In 1995, they reached the final of the Republika Srpska Cup and finished as the tournament's runners up. Having spent several seasons in the First League of Republika Srpska, the team were crowned as league winners at the conclusion of the 2008-09 season and gained promotion to the Premier League of Bosnia and Herzegovina for the first time. Since reaching the division, the team have established themselves as a mid-table team. Their best league finish was 9th place in the 2010-11 campaign.
Mirza Džafić, Haris Handžić and Nebojša Pejić are amongst the best-known players to play for the club during its time in the Premier League.
Rudar Prijedor play their home matches at the Gradski Stadion, Prijedor. Since entering the top flight the stadium has been renovated, with help from the Prijedor municipality. As a result, its capacity has been increased from 2,000 to 6,000.