Eu acho que a junção expressa na cláusula SELECT está sendo executada mesmo para as 100.000 linhas que você não está incluindo no conjunto de dados final.
Que tal agora:
SELECT s2.user_id,
(SELECT address_id FROM address a WHERE a.user_id = s2.user_id ORDER BY address_id OFFSET 0 LIMIT 1) AS a_id
FROM (select *
from subscribers s
ORDER BY s.user_id
OFFSET 100000 LIMIT 200) s2
Caso contrário, tente uma expressão de tabela comum:
With s2 as (
select *
from subscribers s
ORDER BY s.user_id
OFFSET 100000 LIMIT 200)
SELECT s2.user_id,
(SELECT address_id FROM address a WHERE a.user_id = s2.user_id ORDER BY address_id OFFSET 0 LIMIT 1) AS a_id
FROM s2
