You can hire a bicycle from the Caves Visitor center.
Prices are:
Standard bicycle: 10,000 Kip / tour or 20,000 Kip / Day
Mountain bike: 15,000 Kip / tour or 30,000 Kip / Day