contact lenses act like little "bandage". They block some of the air that would normally reach the eye, but also hold some moitsure against the cornea's surface, if I remember correctly.

Have you tried rinsing your eye with plain old saline after removing your lenses? This might take care of the itching by hydrating your cornea.