Keep in mind that the shortest period for a rally is 5 minutes, and it can't be decreased.
You can activate the lone wolf skill, which can immediately end military rallies.

Note: This skill only applies to your own rally.