Yes. When someone has “add users” permission for a document, they can give access to their associates. However, you can monitor who has been added and if you consider it necessary at any time, prevent these people from accessing the document, for which you will continue to have all the permissions.
You can see how to monitor added users in this article.
You can see how to add users to a document in this article.