The fly in the ointment
The single thing or person that spoils what would otherwise be very enjoyable or positive
e.g. I’m really looking forward to the family celebration later this month. The only fly in the ointment is that an unpleasant uncle insists on staying with us for the whole weekend.
The first reference to the idiom is in the Bible where a fly in the ointment caused it to “go off” and smell bad.