Our customised workflow ist quite straight forward and looks like this:

Submitted > Open > Resolved | Reopened > Closed.

It seems that there is no possibility to predefine the sort of the state in order to reflect the workflow. In a Filter Portlet that displays the relevant issues ordered by "Status" the sequence is Open > Reopened > Resolved > Closed > Submitted. For us Submitted should be first. It looks like all custom workflow states are always added after the states provided by atlassian.