Critics see Victoria tie to refugee crackdown