Because a non-trivial amount of the population is colorblind, it's a best practice to never rely solely on color when indicating behavior or status of a UI element.
The filter-pane buttons are by default, I think, green for positive selection, red for exclusion, and grey for unselected.
It would be better if there was also a small icon in the button that would indicate checkmark if positive, "X" if negative, and blank if unselected.
Or perhaps, if Unselected (red), show a thin "X" over the whole body of the button.
Thank you.