How can I delete templates?

I have created duplicates of templates, and some that we don’t want to use anymore. 
 
I cannot figure out how to do the simple task of deleting templates from my “templates” section. 
 
When I click the “x” in the right hand corner of the template, it doesn’t do anything. Am I missing something? 
Is there a designated area to find and customize templates when you’re not in the editor itself?