Assumindo integridade referencial e Postgres 9.4:
SELECT *, total - wins - ties AS losses
FROM (
SELECT count(*) AS total
, count(*) FILTER (WHERE m.winner = um.team) AS wins
, count(*) FILTER (WHERE m.winner = 3) AS ties
FROM users_matches um
JOIN matches m ON m.id = um.match_id
WHERE um.user_id = 123; -- for one given user
) sub;
Sobre o agregado
FILTER
cláusula (introduzida com o Postgres 9.4):