School vouchers promote a two-tier education system.
If private schools perform better, the goal should be to raise public schools to that level, not allow a parent to stop paying taxes towards them because they happen to be able to afford a better alternative.
So not only would you be taking funding away from public schools, making them worse, but you'd be supporting the idea that a wealthy family should be able to have access to better education than a poor one.