Rijkaard led Barca to back-to-back league titles in 2005 and 2006 and the 2006 UEFA Champions League crown, but the last two seasons have been disappointing.
The loss at Madrid, combined with Villarreal's victory over Recreativo Huelva, means Barca cannot finish any higher than third in La Liga.
"Frank has been the leader of the team that led us to glory in Paris, but the cycle is finished because the results haven't gone our way in the last two years," Barcelona president Joan Laporta said Thursday. "The situation was very disappointing last season. We haven't been able to correct the errors that were committed then and we have decided that Frank should be relieved of his post at the end of the season."
Rijkaard's contract was due to run to 2009.
The 37-year-old Guardiola, a former Barca star, joined the Catalan club's youth development program staff last summer and has coached the "B" team.
Click here for Soccer America Executive Editor Mike Woitalla's story on Guardiola and Barca's youth program.