If you go by the early morning or midnight, try to catch bus like supper line. If the weather is good, it will go to Badulla within around 6 hours. No other way unless by your own vehicle. Without breaking road laws you can go around 5 hrs. Day time, it sucks.