Judge correctly found refugee pact violates charter, lawyer tells appeal court