Well not entirely. It's actually not immune to all kinds of cunning attacks.
Today at Ruxcon (the computer security conference), Vincent Berg of scurity company IOactive demonstrated how so-called side-channel analysis can reveal far more than was intended.
What Berg found was that even though encrypted, the 256x256 pixel subsections used to build the map image on the client browser do not radically change their size compared to their un-encrypted cousins. This meant that he was able to create a image-size map of potential locations and by capturing the data stream between Google and the user could identify the images and use nothing more than their size to re-create the map being viewed.
Currently Berg has built this as a simple proof-of-concept and has only pre-analysed a small number of European cities. However there is no reason why this couldn't be extrapolated to the entire world at all resolutions, given enough disk storage and processing power.
It would probably be wise to assume every major government has already done this.
The author is attending Ruxcon as a guest of the organisers.