A client is never told your endpoints by a human. It walks a chain:
- POSTs to your endpoint, gets a 401 carrying a pointer to your protected-resource metadata
- fetches that document, which names the authorization server
- fetches THAT server's metadata for authorize / token / register
Every hop must agree with the one before it. The identifier your metadata publishes has to be exactly the address the client connected to. If they differ, a strict client rejects the token — and nothing about that failure is visible from a status code, because every individual request looks fine.
The check that actually matters is the cross-check: fetch the 401 challenge and the metadata document separately, and compare them to each other. Testing either one alone tells you nothing about whether they agree.