For some years I had problems when updating or installing a site with composer from my vagrant box.

It's a one liner in your vagrant file:

config.ssh.forward_agent = true

If it does not work out of the box, do the following in the guest: