Toulouse won the Top 14 last season but they have been beaten in each of their first two matches at the start of the 2019-20 campaign.
Racing 92 bounced back on Sunday from their opening-round defeat in the Top 14, but reigning champions Toulouse suffered a second successive loss.
A surprise reverse against promoted Bayonne represented a poor start to Racing's campaign, one the Parisian side appeared eager to atone for as they started strongly at home to Castres.
Fabien Sanconnie and Antoine Gibert crossed for tries inside the first 11 minutes and Racing remained in front thereafter, triumphing 23-14.
Their win would have been more emphatic had Geoffrey Palis not gone over for Castres in the 79th minute.
After losing to Bordeaux-Begles last weekend, Toulouse went down 22-12 to Lyon in their latest match.
Noa Nakaitaci scored the only try of the contest after seven minutes, with four Jonathan Wisniewski penalties and one from Charlie Ngatai enough to secure success for the hosts despite the dismissal of Loann Goujon on the hour.