Mitchell, whose contract was to run through 2010, was ousted following another humiliating World Cup qualifying campaign. Canada failed to win any of its six games and finished last in its semifinal qualifying group, behind Mexico, Honduras and Jamaica.
Mitchell, a member of Canada's Soccer Hall of Fame, was on the Canada team that went to the 1986 World Cup in Mexico -- the only time it has advanced as far as the finals.
"We have made this decision to move the program in a new direction," Canadian Soccer Association president Dominic Maestracci said in a statement.
No replacement has been named. An interim coach is needed for the 2009 Gold Cup. Among the candidates are technical director Stephen Hart and staff coach Tony Fonseca.
Canada had a 3-7-5 record under Mitchell, who had previously served as head coach of Canada's under-20 national team, which failed to score a goal at the 2007 Under-20 World Cup in Canada.
With Hart in charge as interim coach for the Gold Cup, Canada lost to the host United States in the semifinals.
Mitchell's ouster is only the latest in a series of setbacks for the CSA.
In 2008, 20-year-old Jonathan De Guzman, considered one of international soccer's hottest prospects, decided to play for the Netherlands instead of Canada, for which his older brother, Julian, has been a key player.
Previously, Owen Hargreaves of Calgary joined England's national team.
Chief Operating Officer Kevan Pipe left in November 2006, and the CSA ended up having to settle a costly wrongful dismissal lawsuit with his replacement, Fred Nykamp, when the CSA's board of directors did not approve the deal Nykamp had with the CSA.
The same thing had happened to Brazilian Rene Simoes, who was supposed to lead the men's national team program. Only after the CSA board refused to approve Simoes' deal was Mitchell hired.
The shining moment for Canadian soccer in recent years was its organization of the 2007 Under-20 World Cup. While the tournament went off to spectacular reviews, it was a financial bath, resulting in losses in excess of $1 million.
CSA President Colin Linford fed up with the lack of support quit in 2007 after a little more than one year on the job.
A month later, Canada played a rare home game when the national team debuted at Toronto's BMO Field, but the match featured protesters, a group of Canadian fans who blamed the CSA for the poor state of the Canadian game.