@saper never tried that but AFAICS, pf copes with configuration relative to absent network adapters:
pass in on foo0 all
You will have to see if the rule is applied when the network adapter is connected, and if not ask devd to reload the firewall configuration when the device is plugged / unplugged.