Baseie suas junções adicionais nos aliases que você criou:
Distributor.query\
.outerjoin(alias_commercial_address, Distributor.commercial_address)\
.outerjoin(alias_commercial_address_country, alias_commercial_address.country)\
.outerjoin(alias_invoice_address, Distributor.invoice_address)\
.outerjoin(alias_invoice_address_country, alias_invoice_address.country)\
.options(contains_eager(Distributor.commercial_address, alias=alias_commercial_address)
.contains_eager(alias_commercial_address.country, alias=alias_commercial_address_country),
contains_eager(Distributor.invoice_address, alias=alias_invoice_address)
.contains_eager(alias_invoice_address.country, alias=alias_invoice_address_country))\
.all()
