A Kieran Gibbs own goal and a late effort from skipper Wayne Rooney handed United their first victory on the road in over seven months, with Arsenal only able to manage a consolation through substitute Olivier Giroud.
United enter the top four of the Barclays Premier League after van Gaal's maiden three-point haul away from Old Trafford but only after they rode their luck following a slow start.
Former Red Devil Danny Welbeck missed a couple of decent chances to make van Gaal eat his words after the Dutchman sold the Manchester-born forward to the Gunners over the summer.
Jack Wilshere should have scored before he was lucky to not see a red card having shoved his head into Marouane Fellaini's chest - but United held on until Gibbs turned Antonio Valencia's low cross past his own goalkeeper.
Rooney doubled United's lead as he and Angel Di Maria hit Arsenal on the counter-attack and, although Di Maria was guilty of missing a great chance to add a third, the returning Giroud's well-taken strike could not spark a late revival for the hosts.
"I was not pleased with the way we have played in the first half," van Gaal said when asked if this was his best win as United boss to date.
"I don't think so
But it was an away game against a top team and we have won
Okay, then I can say yes
It's also the right moment to win this game.
"I have to say the first 35 minutes we gave the ball away so easily that it's not possible for a top team
Because of that, Arsenal created a lot of chances.
"Fortunately we have a very good goalkeeper and after the first 35 minutes you saw that we are coming back in the game and created a few possibilities."
"I told them at half-time that when we keep the ball, when we show confidence, then we shall create many chances.
"We created at least three big chances
Di Maria could finish the game also.
"But also two times the pass was easy to give to Di Maria and Wayne Rooney and the pass was not right.
"That was not a difficult pass in my opinion so we have to improve that
I am very happy, of course."
Luke Shaw was forced off after just 15 minutes as van Gaal saw yet another player added to his ever-lengthening injury list but the former Barcelona and Bayern Munich boss was pleased with the efforts of his inexperienced defence as Chris Smalling, Tyler Blackett and Paddy McNair grew stronger as the game progressed.
"First I hope the players are coming back from injuries because it's unbelievably difficult to create the consistency in the team - because I have to change my line-up every week," he added.
"Maybe you don't know but I am a coach who always has the same line-up, maybe I change one or two
But not constantly so many players.
"I have also changed the formation
It is not always good to create optimism in the team.
"Shaw has to play
Then he is coming back from the international break with the English national team and he is not fit enough
We make him fit.
"Then he is injured with another injury
Ankle
You have seen that
Then I have to give Young (a game) but Young is not fit enough to play 90 minutes.
"I think the players have done fantastic
Blackett, McNair and Luke Shaw - how old are they? You play against a top team like Arsenal and they are always changing position, so you need a lot of communication
I think that is pretty good
But I was not pleased."
Arsenal have endured their worst start to a campaign in 32 years and, as United entered the Champions League positions, Arsene Wenger saw his side slip to eighth and the Frenchman was left frustrated after seeing his side start the match so well.
"It's a game that we dominated for 80 per cent of the time and we haven't dominated a game like that against Manchester United for a long time," he said.
"At the end of the day we were not efficient enough in the final third, defensively and we made a mistake at the back which they took advantage of
That's the history of the game
We had plenty of chances
Their keeper is man of the match
That tells you the story of the game.
"Once we were 1-0 down we had of possession in the final third that they had to defend
We missed a few opportunities in the second half, even in the final 20 minutes
At the moment defensively we are a bit naive."
Wenger confirmed an injury to Wojciech Szczesny which forced the goalkeeper off after Gibbs' own goal is not serious but Wilshere could be out for some time after being replaced following a heavy challenge from McNair.
Source : PA
Source: PA