Advise needed from old school masons

Tldr;

I'm wondering what kind of mix y'all would use for mortar/grout on a stone or block/veneer wall.

The long version:

I made a sidewalk to my kid's playhouse. I poured two stemwalls (5"W x7"H) and inlayed some 8"x16'"X2" (staggered)red pavers that I've had for forever.

In-between the stemwalls, I laid down a vapor barrier, +/-3" of type 2, 1" of sand, then 1" of basically a grout/stucco mix of 1 part Portland cement, 1 part type S lime, and 4 parts stucco sand/type 2/ lime chat mix and floated the pavers on top flush with the stem walls.

I want to grout my joints, and have all the raw materials to make a mortar/grout. It would be easier to get a mortar mix, I know... But I love to make my life more difficult.

I have:
300# of Portland cement
160# of type S
Lime chat
Blow sand (it's brown, tho)
White screened sand from a sand mine
Stucco sand mixed with type 2 (I have screens, tho)

Did you old schoolers use a certain grit of sand that you know of? I did a small batch of mortar that was roughly 1 cement, 1 type S, and 2-3 parts blow sand. But it looked like the blow sand wasn't binding very well to the cement/lime.

I have about 42'x42" of walkway to grout. My first batch did 6'+/-.

I have more questions, but I'll wait for answers to this one first.