Why are some areas of the world so woefully less advanced?

Especially given their head start of a few thousand years...

Government and the desire to try to live in areas that are basically deserts.

Religion.

More government = less advancement. You can even see it happening now in the western world. We advanced so quickly because of the government we created when we first formed. But as things continue to become more regulated, advances in things like medicine slow down. This happened to the eastern part of the world much sooner, and they've been under strict control by people who want to keep advancement to a minimum to keep power

Less cultural heritage and emphasis on private property and commercial success.

Guns, germs, steel.

lack of capitalism

On a serious note, perhaps it has something to do with the need to improve. Necessity is the mother of all invention after all. If you look at civilizations in the tropics, there was little need to improve since the weather was conducive and resources were plentiful. The farther north you go, the more advanced you have to become in order to thrive. The same is true as you head south.

I may be completely off base, but by and large countries and civilizations in the tropics - between the Tropic of Capricorn and the Tropic of Cancer - are the least developed in the present day.

Tribal culture. Much of the Middle east and Africa is full of different ethnic, cultural, and religious groups. Most of these groups hate each other and are usually fighting or campaigning against each other. Hence the constant conflict in these area. Corrupt dictatorships that often enflame these tensions don't help much either.