Authorities raised the volcano's alert status to the highest level yesterday following a "tremendous increase" in seismic activity. It last erupted in 1963, killing 1,100 people.
Villager Made Suda said he left overnight with 25 family members to stay in the Klungkung sports centre.
"I feel grief and fear, feel sad about leaving the village and leaving four cows because it's empty. Everyone has evacuated," he said today.
Waskita Sutadewa, spokesman for the disaster mitigation agency in Karangasem district around Mount Agung, has said nearly 11,300 villagers have been officially evacuated.
More From This Section
He said the real number of displaced might be two or three times that, since many have voluntarily fled their homes.
Officials have said there is no current danger to people in other parts of Bali, a popular tourist island famous for its surfing, beaches and elegant Hindu culture.
"I was very worried last night," he said. "At 11.30 p.m. we said 'we have to leave' and many other people in our area were also leaving."
In its last eruption in 1963, the 3,031-metre (9,944- feet) Agung hurled ash as high as 10-km and remained active for about a year.
The mountain, 72-km to the northeast of the tourist hotspot of Kuta, is among more than 120 active volcanoes in Indonesia.
The country of thousands of islands is prone to seismic upheaval due to its location on the Pacific "Ring of Fire," an arc of volcanoes and fault lines encircling the Pacific Basin.