This is due to the linear implementation of Activity Streams on that page.
Factors affecting performance are:

• number of stars the user has
• number of repos the user can view
• which tab is selected: All | Commits | Reviews | Issues
• status of the 'Show My Activity' checkbox

If you have no stars, and uncheck the 'Show My Activity' checkbox, then the Dashboard page should load in under 1 second.
However, the page load time will perform fairly linearly with the factors listed above.