Can you use non-AP costing special abilities when it's not your turn?

[Berf]

You guess right, it is the symbol (hand or hourglass) that matters when you want to activate a special ability.
Hourglass : anytime, unless the opposite is stated and/or you don't have what is required to activate it (AP, SP, etc.) If two hourglass abilities should be triggered at the same time OR if the two players want to activate hourglass abilities at the same time, it's always the First player who decides in which order the abilities are triggered.
Hand : only when the fighter is activated. Note that you cannot use Hand abilities if you have at least one Fear marker (unless a fighter ignores Fear marker, of course). Another trick, if the Yellow Gaki is in its Deployment state, you cannot use more than one Hand ability as long as your within the area effect, and I mean ONE, not each hand ability once, only one Hand ability.