I think it a better version for 来い would probably be "Bring it!!!" Or duke Nukem "Come Get Some", especially since he's being a bit crude about it all.
He's commanding Apes By Arms to come to him though rather than challenging his opponents. If he'd said 「掛かって来い!」, then yeah, that would be more like what you're saying.