Burgess saw red on 26 minutes for his second yellow card and it made the going extra tough for Robbie Blake's men against the high-flying Urchins.
Liam Nash had already given the home side the lead on 22 minutes and a minute after Burgess left the pitch Hornchurch were 2-0 up thanks to a goal from Ryan Scott.
Bognor had gone in to the game missing the hugely influential Craig Robson but Kieran Douglas being available again was a boost as was the inclusion of both Calvin Davies and Harvey Whyte, both injury doubts going in to the match.
See pictures from Hornchurch v Rocks – taken by Lou Chamberlain – aka @gingegooseberry on Twitter – on this page and the ones linked.
Blake had the chance to reinvigorate his troops at the break and seize an opportunity to lay out a plan for how to cope with playing with one man missing against such a powerful opponent, who have lost only one league game so far this season.
Alfie Rutherford replaced Jasper Mather at the break and it was hoped the striker, on loan from Dorking Wanderers, could add some spark to the visitors’ frontline, potentially teaming up with fellow Meadowbank loanee Jimmy Muitt to good effect.
But a lack of concentration proved to be costly within just two minutes of the restart as Bognor found themselves 3-0 down when the ever-lively Nash grabbed his second goal of the game.
Skipper Davies, who picked up a yellow card along with Whyte, Douglas and Dan Smith, hit home a free-kick to give the visitors a goal on 53 minutes, providing faint hope that perhaps a comeback was on the cards.
But in reality, there was little chance of that and the home outfit’s Femi Akinwande made it 4-1 on 81 minutes and in so doing condemned the Rocks to their third away day defeat of the campaign that leaves them 13th in the league standings.
Blake's charges get the chances to go again on Tuesday when they travel to Whitehawk for a Sussex derby, kick-off 7.45pm.
Bognor: 1 Ryan Hall, 2 Harvey Whyte, 3 Reuben Livesey-Austin (Ben Anderson 65'), 4 Calvin Davies, 5 Kieran Douglas, 6 Danny Howick, 7 Lucas Pattenden, 8 Matt Burgess, 9 Dan Smith (Tommy-Lee Higgs 70'), 10 Jasper Mather (Alfie Rutherford 45'), 11 Jimmy Muitt. Subs: 12 Tommy-Lee Higgs, 14 Alfie Rutherford, 15 Ben Anderson, 16 Isaac Olaniyan, 17 Joe Briffa