A client is never told your endpoints by a human. It walks a chain:

1. POSTs to your endpoint, gets a 401 carrying a pointer to your protected-resource metadata
2. fetches that document, which names the authorization server
3. fetches THAT server's metadata for authorize / token / register

Every hop must agree with the one before it. The identifier your metadata publishes has to be exactly the address the client connected to. If they differ, a strict client rejects the token — and nothing about that failure is visible from a status code, because every individual request looks fine.

The check that actually matters is the cross-check: fetch the 401 challenge and the metadata document separately, and compare them to each other. Testing either one alone tells you nothing about whether they agree.