United pounded Arsenal goalie Manuel Almunia early in the match and scored in the 17th minute when O'Shea, free in the penalty area, beat Almunia with a half-volley from eight yards after Michael Carrick had turned Anderson's corner kick back across goal.
The closest United came after that was when Cristiano Ronaldo hit the crossbar with a dipping shot.
Despite the narrow score, United manager Sir Alex Ferguson was happy with the outcome.
"The most pleasing aspect is that before the game we said try and win and not lose a goal and we have done that," he said. "And the performance level was very good, so we have achieved that. and I think we have given ourselves a good chance overall."
Still, Ferguson acknowledged that United did not achieve a knockout blow.
"We always do things the hard way," he said, "and this is another night when we have got our fans frustrated where we maybe should be into the final but we are not -- but we have done that many times. We did it in the last round, and we will have to go through the process of taking the game to the wire probably."
The Premier League race is also going to the wire, so Ferguson can't rest his first team on Saturday.
"That's Arsenal's big advantage now, they could play [assistant manager] Pat Rice at right back and [Manager] Arsene [Wenger] at center forward if they like, it doesn't matter to them at Portsmouth," he said. "We have to play a team to win at Middlesbrough. It's a lunch-time kickoff and I don't think that's right but we will have to get on with it."
April 29 in Manchester
Manchester United 1 Arsenal 0. Goal: John O'Shea 17
Manchester United -- van der Sar, O'Shea, Ferdinand (Evans, 88), Vidic, Evra, Fletcher, Carrick, Anderson (Giggs, 67), Cristiano Ronaldo, Tevez (Berbatov, 67), Rooney.
Arsenal -- Almunia, Sagna, Toure, Silvestre, Gibbs, Diaby, Song, Walcott (Bendtner, 71), Fabregas, Nasri, Adebayor (Eduardo, 83).
Referee: Claus Bo Larsen (Denmark).