Armando Escobar equalized in the 28th minute, and Alan Aguilar and Erick Acevedo added goals within a minute of each other midway through the second period to pace "Los Chapines" to a victory that left it in second place in Group F. Russia led the group after routing the Solomon Islands 16-0 in a match later Saturday.
It was the second straight time Guatemala has begun the World Cup with a victory, having beaten Egypt 1-0 four years ago in Brazil.
Colombia countered to go ahead in the seventh minute with a goal by Alejandro Serna. Jose Gonzalez, who led Guatemala with three goals at the CONCACAF Championship in July, equalized in the 12th minute but the South Americans restored their lead within seven minutes.
Escobar equalized a second time in the 29th minute, and Aguilar put Guatemala ahead for the first time by converting a penalty in the 31st. Acevedo added insurance 55 seconds later and Walter Enriquez completed the scoring with 38 seconds remaining in the match.
Guatemala will have its biggest test of the first round next on Tuesday when it faces Russia.