The verb "ausfeilen" does exist, but is typically only used literally in specific contexts, when you're actually taking a file or rasp to a piece of work. "Ausgefeilt" in the figurative meaning you're asking about is derived from the verb "feilen".
In adjectives, the prefix "aus-" can mean something like "fully", "completely", "to the largest possible extent". Take for example "ausgewachsen". Here, "aus-" is combined with the verb "wachsen". But when something is "ausgewachsen", it actually isn't growing any more, because it already has grown as much as it possibly could (or as much as it ever will).
Another example would be "ausgebrannt". If something is "ausgebrannt", it isn't burning any more, because it already has completely burned down or burned out.
Note that this principle doesn't apply to every adjective starting with "aus-". An example where it doesn't work like this would be "ausgeschlossen", which actually is derived from "ausschließen".
So, if your speech is "ausgefeilt", you've already polished it as far as possible (or as far as you ever will), and you're not polishing it anymore. If you are still working on it, you just say
Ich feile (noch) an meiner Rede.
I'm (still) polishing my speech.
To give something "den letzten Schliff" refers to the last round of polishing you give to something. Because not every round of polishing will be the last one, you can't use this phrase for every "feilen" an etwas.