فمن رغب عن سنتي فليس مني

"Whoever turns away from my Sunnah, then he is not of me." (Al-Bukhari no. 5063 and Muslim no. 1401)

This means whoever turns away from my way that I follow, then he is not of me. This is because they have chosen another way than the path of the Prophet (sallAllaahu `alayhi wa sallam). The optional and voluntary prayers are not related to this Hadeeth, not performing them is not done with the objective of turning away from the Sunnah of the Messenger (sallAllaahu `alayhi wa sallam). Rather, since these acts are optional, and not compulsory, people are permitted by Allaah Almighty not to perform them. So there is a difference between someone who abandons the guidance of the Prophet (sallAllaahu `alayhi wa sallam), turning away from it, and one who does not follow it out of laziness or because Allaah Almighty has permitted him not to do so.