re: Borrowing money from 401k...how does this work usually?

If this is an "investment" and not your primary residence you going to be looking at 15-25% down not 3.5%

Also is your 401k pretax or Roth?

If its pretax your withdrawal is going to be taxed heavily.

My tax guy was telling me one of his clients pulled out 100k to build a pool and a pool house. When tax time came he owed Uncle Sam 30k