While this could be set up by doing something like creating a single login server where each user's ssh public keys are used that then uses it's own key to connect to the various servers, doing so would mean a lot of administrative tasks that would need to be done manually if your team or the servers they need to access is subject to changes. Some other centralized system like KeyBox might allow for a more easy to manage option.