You should leave atleast by 5 am from pettah & also you should defa go in an intercity bus machan, I don't know about the leaving time of the earliest intercity bus of the day to kandy from pettah, anyway machan just be there at pettah atleast by 4.45 am & you can go in a Kandy or a Matale intersity bus.
Most of the normal buses & semi luxury buses stops on the way for tea, breakfast, etc in the morning.