Swansea took the lead two minutes after halftime on a goal by Trinidadian Jason Scotland, but Dempsey equalized with his fifth goal of the English season for Fulham when he redirected Simon Davies' corner kick into the goal.
Dempsey almost added a second goal before Bobby Zamora gave Fulham the win four minutes later with a second goal set up by Davies.
Manager Roy Hodgson called Fulham's performance one of its best of the season.
Arsenal vs. Burnley
Fifth Round, Replays
Middlesbrough vs. West Ham United
Hull City vs. Sheffield United
Coventry City vs. Chelsea
Fulham vs. Manchester United
Everton vs. West Ham United/Middlesbrough
Arsenal/Burnley vs. Sheffield United/Hull City