iPhone user since 2007, help me choose between Nexus 6 and Note 4

I have been a apple smartphone/laptop/tablet fan since 2006. I had the first ever iPhone and upgraded every other iteration since, my last one being the iPhone 5.

I am a network administrator at a hospital. I used android early in it's inception on the first ever tablets and was not impressed. Then last year i got a nexus 7 tablet off craigslist for $100 and started to enjoy android and rooted it, etc and got comfortable with it as a mobile OS.

I use all google services from play music to gmail to google keep for notes. I also own a macbook air 2012 model and have an apple tv and chromecast, im looking into getting a fire tv as well.

I also own an android based Ouya game console I use for playing emulation on my TV.

I once said I would never deviate from the iPhone because I love the polish and a jailbroken phone has been the coolest tech I've ever owned.

I wanted a bigger phone and when the phablets started coming out I was starting to learn towards the Note 4 for my next phone. Then the iPhone 6+ was announced and my brother and wife have one, it is amazing piece of tech.

But I think I'm finally going android this time around because comparing the specs between the Nexus 6, Note 4, and 6+ it looks like a no brainer to go android this time around.

So between the Note 4 and the Nexus 6 what do you guys think? I heard TouchWiz gimps the Note 4. I'm also wondering why google wont allow a SD card expansion on the Nexus 6.

Main use will be personal but will be using citrix and exchange for work.

Thanks in advance.

ETA: Also talk me into staying with iPhone ^+ if you can, I just dont see it, and this coming from a former apple die hard fanboy.