How do you avoid weed from taking over your garden? Do not plant it, and take it out when it shows up, anyway.
A Kanban board should only contain things you are going to do, not things you would like to get done.
So, firstly, only put things on from the first category, and don’t be afraid to take out things if it turns out you won’t be doing them (soon).
That doesn’t mean you have to totally forget about them. You (or your product manager) can keep them in another list.