I can assure you that layer creation is neither chaotic nor arbitrary. New layers are appended to the end of an internal array of layers, which resides on the active document. The display order - the order the layers appear in the Layer panel, has nothing to do with a layer’s layer index.
That said, when Rhino saves the document, it does save layers in their display order. This is done because the display order is not saved in the 3dm file. Thus when you reopen a file, the layers appear how you arranged them before saving. So one inconvenient work around it to save the document, reopen the document, and then export to .AI.
I’ve added a YouTrack item to modify the .AI exporter to write layers in display order. You can track the item here: