After securing just five points in 2013, the Gunners were set for a second successive home draw until 12 minutes from time.
Substitute Lukas Podolski got the goal, although it was Geoff Cameron who did the damage by deflecting the German's free-kick past his own goalkeeper.
Arsenal remain sixth in the standings after the 1-0 win, but closed the gap on fourth-place Tottenham to a single point - much to Wenger's relief.
"I said already before the West Ham and Liverpool games, that we are in a position where we cannot drop points," the Gunners boss said. "We dropped two points against Liverpool but we couldn't do it again today.
"What is more important is the quality of performances is there. The spirit is there. The target to be in the top four and to focus on the two cups we are still in is very important now as we have strengthened our belief and our confidence."
Michael Owen aimed a punch at Mikel Arteta after reacting badly to a challenge from the Spaniard towards the end, which came after a strong Ryan Shawcross challenge on Laurent Koscielny.
As the former England striker was not booked by referee Chris Foy, he could now face retrospective action from the Football Association - something Stoke manager Tony Pulis was not keen to dwell on.
"I think Arteta's challenge was a poor challenge, but Michael should not do that," he said. "That should be left to the referee and I think if you look at Glenn Whelan's challenge on Santi Cazorla, it is a really good challenge and plays the ball."
Pulis cut a frustrated figure in the post-match press conference after seeing a point slip out of his side's grasp late on. "Obviously we're disappointed," the Welshman said.
Source: PA
Source: PA