A Roque Santa Cruz double cancelled out first-half strikes by Abou Diaby and Eduardo, before the Croatian striker slotted home the winner in the 104th minute of a fascinating game of football.
The two sides arrived at Ewood Park on the back of very different Premier League results.
Mark Hughes made two changes to the Rovers side that were hammered 5-3 at Wigan Athletic, with Matt Derbyshire preferred to Benni McCarthy in attack and David Dunn replacing the suspended Brett Emerton.
Meanwhile, Arsene Wenger named an entirely new starting eleven following the 1-0 victory at home to title rivals Chelsea.
In a bright opening, Rovers registered the first effort of the game after just two minutes. Stephen Warnock sprayed a raking pass across field to David Bentley, his touch over the head of Armand Traore was sublime, but his finish found the Arsenal fans behind the goal.
And it was the travelling army of supporters who were in full voice just four minutes later. After some neat interplay on the right flank, Nicklas Bendtner picked out Diaby at the far post and the France international expertly volleyed the ball over Brad Friedel and into the opposite corner of the net.
Three minutes later and the provider almost turned scorer, when Bendtner raced clean through on the left, but his lofted effort bounced back off the bar.
Rovers opted for a change of formation and it almost paid an immediate reward. On 28 minutes, Morten Gamst Pedersen's cross dropped kindly for David Dunn, but his left-foot strike produced a stunning save from Lukasz Fabianski.
And a minute later, the young Gunners doubled their lead. Denilson fed the ball through from midfield, Chris Samba failed to cut it out and Eduardo coolly slotted past Friedel.
Rovers again responded well, with Bentley's free-kick forcing a flamboyant save from Fabianski, before the man with the appropriate name for the time of year again delivered the goods.
On 42 minutes, Bentley slipped a pass through to Derbyshire, the young Englishman fired the ball across goal and Santa Cruz slid in to connect with his fourth goal in his last two games.
The home side continued where they left off after the break, with the Paraguayan striker almost getting his side back on level terms.
On 48 minutes, Robbie Savage played a ball over the top, Santa Cruz just got there ahead of Fabianski, but Alex Song got back to clear. From the resulting throw-in, Ryan Nelsen just failed to connect with Bentley's deflected cross.
A succession of Arsenal fouls followed and from one subsequent Bentley set-piece, Bendtner headed onto the roof of the net and from the resulting corner, Nelsen again failed to get the decisive touch at the far post.
Youngster Mark Randall eventually picked up the visitors first booking for a challenge on Warnock and, from another Bentley free-kick, Santa Cruz headed home the equaliser.
The game ebbed and flowed as both sides pushed for the winner. And Rovers almost got it eight minutes from time, when good pressure from Santa Cruz saw the ball break to Derbyshire in the box, but his goal-bound shot deflected wide off Philippe Senderos.
As the 90-minute mark approached, Lassana Diarra was booked for a foul on Bentley, Santa Cruz miskicked in front of goal when presented with the opportunity for his hat-trick and Denilson, who had already been booked, was shown a straight red card for a nasty tackle on Dunn.
With the man advantage, Rovers looked the more threatening in the opening period of extra-time.
Gunners substitute Nacer Barazite was stretchered off following an innocuous challenge on the edge of his own box, before the visitors again regained the lead against the run of play when Song played the ball through to Eduardo and the Croatian slotted home his second of the evening.
In the final period of extra-time, Pedersen curled a free-kick into the side-netting, substitute McCarthy volleyed narrowly wide with a clear sight of goal, Pedersen's close-range shot from point-blank range was deflected over the bar and then Samba headed onto the post from the Norwegian's cross.