You'll need to find the IPv4 address within your network. If you open up your command line and enter ipconfig, it should provide that to you. Then, make sure that you have your application setup to run on that IP by adjusting your launchSettings.json file within your Server project (ProjectName > Properties > launchSettings.json). Your IPv4 address will be something like 192.168.1.102, so your launchSettings.json will need adjusted like so:
...
"YourProjectNameHere": {
...
"applicationUrl": "https://localhost:5001;http://192.168.1.102:5000",
...
}
...
Then once your site is running locally, you should be able to visit your IPv4 address with the 5000 port on your phone. My guess is your launchSettings.json is missing this, so the server doesn't know to listen on that address.