Rome went from having 1,000,000 citizens to under 50,000. The powers that be were more worried and concerned with staying alive versus trying to rebuild infrastructure they didn't understand. Remember we lost a lot of knowledge when Rome fell, so the capability of the people was not as great as Rome.