It seems like if it was the Annie Altman accusations, they could have just paid her off. If they wanted him around and he was a creep, there are ways to make this stuff go away. AFAIK a lot of the sister's accusations were accompanied by being excluded from the father's will. Not saying she made it up, but it seems like if those grievances are bundled, there's a easy way to make that problem go away.

Having talked with Annie, she's not willing to be bought off, as Sam has tried to do before in the past with her.