Three ministers 'meet anti-Brexit group over second referendum'