Data and Models for the paper "Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment"