Working 'on a refugee movement is like riding in a cavalry charge'