Harder times for refugees: agency chief