Sim. A única maneira seria descartar a restrição com uma tabela Alter e recriá-la.
ALTER TABLE <Table_Name>
DROP CONSTRAINT <constraint_name>
ALTER TABLE <Table_Name>
ADD CONSTRAINT <constraint_name> PRIMARY KEY (<Column1>,<Column2>)
ALTER TABLE <Table_Name>
DROP CONSTRAINT <constraint_name>
ALTER TABLE <Table_Name>
ADD CONSTRAINT <constraint_name> PRIMARY KEY (<Column1>,<Column2>)