The actor in this case will either be ourself (in which case users.Bot
already exists), or a services nick (which may not have a full hostmask,
and therefore cause hashing errors). Only try to add a user if we're
changing a channel mode. This may still break on chanmodes, needs more
testing in that regard.