The Gunners went into the game having won their opening two games against Hamburg and Porto, with Arsene Wenger recalling Robin van Persie and leaving Theo Walcott on the bench.
CSKA made most of the early running, before 14 minutes in Tomas Rosicky and Thierry Henry combined to send Cesc Fabregas through, but he was forced wide after a heavy first touch and the chance was lost.
Midway through the first half the home side made the breakthrough, after Dudu had won a free-kick right on the edge of the Gunners penalty area.
The resulting set-piece was taken by Daniel Carvalho, who bulleted a left-foot shot straight down the middle of the goal, with Jen Lehmann beaten for sheer pace.
Soon after, Lehmann had to tip over a superb dipping 30-yarder from goalscorer Carvalho, before Arsenal forged their first opening.
Tomas Rosicky found space on the right and his far-post cross was headed goalwards by Henry, but Sergey Ignashevich was there to clear the danger.
Vagner Love had an effort ruled out for offside, before van Persie wasted two openings, first failing to play in William Gallas, before losing the ball to home keeper Igor Akinfeev when well placed.
After the restart Arsenal started brightly, without threatening Akinfeev's goal.
Love then tried to get around Kolo Toure, who made a superb last-ditch tackle to avert the danger, before Carvalho tested Lehmann with a shot towards the near post.
Just past the hour CSKA won a series of corners as they went looking for a killer second goal, with the Gunners forced deep on the defensive.
Wenger brought on Emmanuel Adebayor and Walcott late on as the Gunners tried to rescue something from the game and four minutes from time they thought they'd got an equaliser.
Henry stroked the ball home from close range, but was penalised for handball and cautioned for his troubles, although replays suggested he had been hard done by.
That proved to be a decisive moment as CSKA held on to record a crucial win and dent Arsenal's hope of progressing from Group G.