I am afraid you were using the wrong serve connection type.
Azure Repos/Team Foundation Server service connection is used in the repositories resources section in the yaml pipeline, which you can refer to the repositories in other organizations using a service connection. See this document for more informaiton.
resources:
repositories:
- repository: otherrepo
name: ProjectName/RepoName
endpoint: newTFSServiceConnection
steps:
- checkout: self
- checkout: otherrepo
If you want to set up a pipeline for a repos in another TFS server. You need to create a new Service Connection of type Other Git
And enter the tfs repo url and users name / password in the edit page.
Then you can select this Other git type Service connection when creating a pipeline.
If you cannot connect to your tfs repo via PAT. It might be because the IIS Basic Authentication is enabled on your windows machine, it prevents you from using personal access tokens (PATs) as an authentication mechanism. See here, You can try using basic anthentication method (username and password) instead
Please be noted that you need to run this pipeline on your self-hosted agent. As you are trying to connect to the repo on your tfs intranet server. Cloud Microsoft hosted agent cannot connect to your intranet tfs server. Unless it is can be accessed in the public network.
CLICK HERE to find out more related problems solutions.