The Welshman's 114th-minute header secured a 2-1 win and a lucrative home tie against Premier League side Everton in the next round to be played on January 5-6.
The visitors went ahead on the stroke of half-time through Shaun Harrad's controversial penalty before Hereford's Sam Clucas equalised in the 74th.
Bury defeated Southend 3-2 on penalties after their tie ended 1-1 after extra time.
Zac Thompson opened the scoring for Bury shortly before the hour mark but Gavin Tomlin squared it for the home side just four minutes later.
The Lancashire club meet Brentford in the third round.
Barrow's tie with Macclesfield and Notts County's clash with Rotherham were both called off due to frozen pitches.