So in trying to figure out why my pavucontrol is missing an icon, I learned a lot about how icon themes work today!
First, it seems that if your system is using dconf, #GTK 4 will only observe the value you see when running:
dconf read /org/gnome/desktop/interface/icon-theme
You can change it using:
dconf write /org/gnome/desktop/interface/icon-theme "'name-of-theme'"
(That's single quotes inside double quotes!)
This even seems to work while the application is running! :O