The Immemorial Mass: "Just a fashion"?

Rorate Caeli reports that the Holy Father recently stated that he cannot understand why young people turn to the Traditional Latin Mass and that he sees the TLM as  "...a kind of fashion. And if it is a fashion, therefore it is a matter that does not need that much attention. It is just necessary to show some patience and kindness to people who are addicted to a certain fashion."  When I first read this statement I thought to myself:  "Once again this man scares the hell out of me.  What condescension!  What arrogance towards his own people!  Is it just a matter of time before he rescinds Summorum Pontificum?"  Once I had settled down a bit I thought that maybe his lack of attention towards us traditionalists is a good thing.  If he ignores us, maybe he'll just leave us alone to worship as we choose.  That way Summorum Pontificum will remain untouched and the law of the Church.  Fr. Z says not to necessarily believe this recent report as this information is third hand, but if the Holy Father really said these things he ignores stalwart, faithful Catholics at his own peril while ministering to nominal, lukewarm Catholics.  Fr. Z also said that he wouldn't be surprised if the Holy Father abdicates when he turns 80, and even expects that he will do so.  While I think this would cause disruption and a continued bad precedent if he does so, it would mean only three more years of him as pope.  That thought doesn't sadden me.