It was tough luck on the Tigers who had led from the 13th minute when Nicky Barmby's deflected shot beat Lukasz Fabianski, deputising for Manuel Almunia, in the Arsenal goal.
Gunners boss Arsene Wenger made six changes to the side that beat Blackburn 4-0 on Saturday and it almost cost them dear when Barmby got on the end of Andy Dawson's pass and his effort deflected off Johan Djourou and over Fabianski.
Phil Brown's side also went close to adding to their lead as they looked to replicate their shock win at the Emirates Stadium in the league in the autumn.
Geovanni's free-kick was tipped over the bar by Fabianski while Barmby had an effort ruled out for offside.
Kamil Zayatte then headed over from Peter Halmosi's cross before the Gunners finally moved through the gears with Sam Ricketts forced to block Andrey Arshavin's effort.
After the break Abou Diaby headed wide before Arshavin was again denied by Ricketts. Dawson then managed to get Robin van Persie's header off the line before Alex Song fired the rebound wide.
In the 74th minute Arsenal finally grabbed an equaliser after the effervescent van Persie had seen a free-kick saved by Hull keeper Boaz Myhill and struck the bar with a header. Arshavin then set up the Dutchman to fire home from close range,
Hull almost regained the lead immediately, Geovanni shooting just wide, but it was Arsenal who grabbed the winner with six minutes left when Myhill failed to collect Samir Nasri's lofted free-kick into the box and William Gallas headed home from three yards.