Impossible is such a strong word. Here’s a model that lets you select/toggle a list of points. What you do with them after that is up to you. 
pt_select_2020Dec7b.gh (24.5 KB)
It uses a Data Recorder to keep a list of the value of ‘list idx’ each time the button (blue group) is pressed. If the total number of clicks for any ‘list idx’ value is odd, it is removed.