Hmm, you're right. Then instead of just controlling them, just make them track your target. Not as accurate, usualy, but it'll take less energy and concentration that way.