re: I have the disquieting feeling that the GoT TV series is starting to drift

Truthfully, the series is handling the material far better than the books, which meander, completely abandon characters, and sprawl out in far too many directions.

Be grateful that the showrunners understood that dilemma and are actually acting to address it by diverging wildly from the books, including foregoing the introduction of many characters (at least as yet). The fact that the show has to follow Arya, Jaime, Cersei, Dany, Jon, Sansa, soon to be Stannis and Littlefinger, they are doing a pretty damn good job of weaving those stories together without completely losing the audience. Because even the bookreaders got lost around this time.