I didn't figure it out but I found this:

https://stackoverflow.com/questions/58718589/adding-a-razor-page-in-blazor-project-creates-a-cshtml-file

With this answer:

I'm using Microsoft Visual Studio Professional 2019, Version 16.4.3.

Instead of creating a new "Blazor Page", go to the dialog to add a "New Item". From ASP.NET Core, Select "Blazor Component". This will create a new file called YourComponent.razor.

Bonus Fact: To create a code-behind file for the component, add a new class and specify the name YourComponent.razor.cs. Declare the class partial (because the .razor file itself makes a partial class declaration).

I am also new to Blazor and this has been a source of confusion for me. I hope that they make it more straight-forward in the next version of Visual Studio.

Only I don't see Blazor component, just Razor component.

And I have to add @page "/xxx" to it can appear in the NavMenu

I would upgrade your version of Visual Studio and see if the issue goes away. I believe the most current at this moment is 16.7.3

Also, the "Razor Component" is what you're looking for. Read more at: https://docs.microsoft.com/en-us/aspnet/core/blazor/?view=aspnetcore-3.1#:~:text=Components%20in%20Blazor%20are%20formally%20referred%20to%20as,support.%20Razor%20Pages%20and%20MVC%20also%20use%20Razor.

The page directive is necessary to setup your page in routing, so it'll appear at that URL. It won't appear in your NavMenu unless you manually go add a link into your NavMenu component.

I realized that yes, it was the Razor component I was looking for. The name is not great.

I'm at 16.7.3. There is 16.7.5 available.