Believe it or not I actually struggled with this for hours so figured id put the very simple solution here

apt-get purge phpmyadmin
apt-get purge mysql-server
apt-get purge mysql*