Thinking back, wasn't Jiraiya irresponsible regarding Naruto?

The whole world knew how Naruto was treated in the village, yet instead of helping his disciple's son, he just stood by and watched... T_T
With someone at Jiraiya's level, he definitely could have helped..