Removal of the cam towers will not cause a problem. A common cause of the camshaft binding in the cam towers is a warped head due to a blown head gasket. If the head is surface ground on the gasket surface it is essential that the top surface is also machined or the cam towers will be out of alignment. Alliteratively appropriate thickness shims can be placed under the towers to bring them back into alignment.
If the head is bent more than .005" it should be straightened to within .005" prior to machining.
Gordon Dobie
Datun Racing Team Chief Engineer (Retired)