Not directly taxed, but when you borrow against your 401k, you are basically taking out pretax dollars and replacing them with after tax dollars when you pay it back. Then when you withdraw for retirement, you pay taxes on it again.
I get what you are saying, since I used to see it the same way.
However, the key to understanding a 401k is to look at independently in its two distinct components.
1) The 401k is essentially investing in a fixed return fund at the loan's interest rate. So if you can do better this return with other funds in your 401k, you are giving up value. ETA: Apparently other plans work differently, but mine definitely deducts the loan from the investable 401k balance.
2) The cost of the loan can be compared against other borrowing options. The interest is not tax deductible, but is better in that you pay the interest to yourself. It is not collateralized in the exact sense, but you cannot skip payments if payroll-deducted, and there is the risk of a big tax hit if you leave the company and can't repay the balance.
This post was edited on 1/7 at 12:18 pm