So apparently even if I have party audio enabled, it still won't record people in the party chat voices unless they have share audio enabled on their console.
Whenever I go to my broadcast settings and select include party chat audio, it tells me:
Playstation wrote:Players in the party who want to include their voices in the broadcast must set [Party Settings]>[Allow Your Voice to be Shared] to [Allow]
Wouldn't be a problem if it was a perm option but I don't think it is, at least I haven't checked. If my Intensity card would pick up the console would make life easier too. ;P