I thought I’d post this in case anyone runs into the same problem I had:
To remove invalid annotations, first create a new indirect array (this code is in ruby):
new_annots = doc.CreateIndirectArray()
then iterate through existing annotations, pushing them into your new_annots array
then set the annotation value for the page sdfobj to your new array
voila! have to do this per page, but it will let you remove invalid annotations where the page.RemoveAnnot() method will not.
Note: although the annotations SEEM invalid, some may not be! try exporting these annotations to XFDF and re-merging them, if you need to get them back.