I have the checkbox that says "Mark this if you want guards to ignore aggressive creatures" marked. Under body -> manage -> settlement -> manage_settings (the checkbox under Aggressive Creatures.)
Shouldn't the spirit guard avoid aggressive creatures? I've tried removing spirit guards and resummoning them.