A goal from Giroud and an own goal from Delaney gave the lads three points, but this was no easy win.
We missed plenty of chances, and Palace could easily have taken one or even all three points.
So we're off the mark, but one wonders how much more this team has to give - will Wenger strengthen?
Defensively we don't look too solid, and some would say that we need a top centre forward to turn us from also-rans into title contenders.
Report:
Olivier Giroud's stunning volley helped Arsenal battle to a 2-1 Barclays Premier League win at Crystal Palace.
The Gunners produced the required response from their opening day defeat at home by West Ham with a determined display at Selhurst Park, where German playmaker Mesut Ozil delivered a stand-out performance.
Ozil's floated cross was smashed home by Giroud on 16 minutes, and although defender Joel Ward drilled in a 25-yard equaliser before the break, a deflected header from Alexis Sanchez in off Damien Delaney proved enough for a hard-earned victory.
Manager Arsene Wenger had opted to start Sanchez, the Chile forward only just back from an extended summer break following the Copa America.
It was Sanchez who should have put the Gunners ahead on seven minutes following a lightning break.
Francis Coquelin fed Ozil down the left, and he in turn released Sanchez into the Palace penalty area.
The Chile forward's first touch was heavy, pushing him wide, with Ward recovering to block his shot before James McArthur was back on the line to boot away the goal-bound follow-up.
Arsenal did take the lead on 16 minutes with a superb goal from Giroud.
Ozil dispossessed Ward on the left, looked up and floated a ball across to the penalty area.
Arsenal remained on the offence, with Sanchez's low, angled drive saved at full stretch by Alex McCarthy before being hacked away in the six-yard box.
Palace, though, were level on 28 minutes with another fine goal.
Patient build-up saw the ball worked across to the left of the Arsenal box, where Yannick Bolasie chested the ball to McArthur who laid it off to Ward, some 25 yards out.
The defender then drilled a low shot into the far corner, just out of Petr Cech's reach.
Arsenal looked reassert their dominance before half-time, with Aaron Ramsey's near-post flick forcing McCarthy into a smart save.
It was Arsenal who eventually went 2-1 ahead on 55 minutes.
Hector Bellerin's deep cross from the Palace right was headed back down goalwards by a huge leap from Sanchez, and the ball cannoned into the net off Eagles captain Delaney.
The Palace fans were in uproar when Francis Coquelin, who had been booked in the first half, clattered into McArthur right in front of the referee, but Lee Mason decided against issuing a second yellow card.
Wenger was taking no further risks with the French midfielder, as he was replaced by Alex Oxlade-Chamberlain on 64 minutes.
Aaron Ramsey could have made the closing stages more comfortable, but he drilled over after being teed up by another superb chipped pass from Ozil.
In stoppage time, Oxlade-Chamberlain was denied by a smart save from McCarthy after a breakaway down the right.