To auto-start the ssh-agent when using zsh add this snippet to your ~/.zshrc:

if [ $(ps ax | grep "[s]sh-agent" | wc -l) -eq 0 ] ; then
    eval $(ssh-agent -s) > /dev/null
    if [ "$(ssh-add -l)" = "The agent has no identities." ] ; then
        # Auto-add ssh keys to your ssh agent
        # Example:
        # ssh-add ~/.ssh/id_rsa > /dev/null 2>&1
    fi
fi

This works great with zsh running on WSL 2, but I’m sure it should work just fine on a regular Ubuntu running ZSH.