Você pode usar SUBSTR e CAST AS UNSIGNED/SIGNED em ORDER BY:
SELECT * FROM table_name ORDER BY
SUBSTR(col_name FROM 1 FOR 1),
CAST(SUBSTR(col_name FROM 2) AS UNSIGNED)
SELECT * FROM table_name ORDER BY
SUBSTR(col_name FROM 1 FOR 1),
CAST(SUBSTR(col_name FROM 2) AS UNSIGNED)