Arsene Wenger's side were below their best for long periods at the Emirates Stadium, but Germany forward Podolski made the crucial contribution as he put Arsenal ahead after Gervinho's opener was cancelled out by Olympiakos striker Kostas Mitroglou.
Aaron Ramsey's late strike ensured Arsenal, who defeated Montpellier in their opening Group B fixture, remain firmly on course to qualify for the last 16.
But Wenger will know there is still plenty of room for improvement, especially in a defence which struggled to cope with Olympiakos's relatively limited forward line.
Wenger admitted this week that his players were still feeling down after suffering their first defeat of the season against Chelsea on Saturday and he had challenged them to show they are mentally strong enough to cope with negative results.
In the circumstances, Olympiakos, who lost at Schalke in their opening group match, should have been welcome opponents for Arsenal, who had beaten the Greek side at home in the Champions League twice in the last three seasons.
But Wenger, watching from the stands as he served a European touchline ban, must have been concerned in the early stages.
The Gunners looked short of confidence and almost self-destructed when Vito Mannone and Thomas Vermaelen make a hash of a short kick out, only for Olympiakos to squander the opportunity.
Santi Cazorla's inswinging free-kick forced Balazs Megyeri into action for the first time. But a succession of misplaced passes from Alex Oxlade-Chamberlain and Mikela Arteta brought a chorus of disapproval from the frustrated home fans.
That anxiety was creeping into Arsenal's play as well and Laurent Koscielny was fortunate to escape with a booking after launching into a needless two-footed lunge on Paulo Machado.
Olympiakos were starting to scent blood and Mitroglou's first-time shot had Mannone scrambling to save.
Leonardo Jardim's side went even closer when Arsenal's statuesque back-four failed to deal with a Giannis Maniatis cross, but from point-blank range Machado somehow managed to scoop his miscued shot well wide.
Out of nowhere Arsenal sparked into life and took the lead through Gervinho's fifth goal of the season in the 42nd minute.
Podolski directed a cross towards the edge of the Olympiakos penalty area and Arteta disrupted the visitors' attempts to clear, allowing Gervinho to pounce on the loose ball before driving in a low shot that beat the slow to react Megyeri.
But Arsenal's lead was short-lived as Olympiakos levelled on the stroke of half-time.
Leandro Greco curled over a teasing cross and Mitroglou, granted too much space between Koscileny and Vermaelen, rose to glance a superb header over Mannone and into the far corner.
Cazorla should have put the hosts back in front in the opening moments of the second half, but the Spanish midfielder side-footed wide from Podolski's cross.
There would be no need for any recriminations however as Podolski showed his team-mate how to do it in the 56th minute.
Gervinho's initial cross was blocked by Kostas Manolas, but the rebound fell into the Ivorian's path and he flicked a pass towards Podolski, who turned sharply before planting a powerful low strike under Megyeri.
That blow seemed to shatter Olympiakos's spirit and Wales midfielder Ramsey came off the bench to seal the win with a cool chipped finish from fellow substitute Olivier Giroud's flick in stoppage-time.
Source: AFP
Source: AFP