I'm pretty sure you're overthinking this, assuming I understand the question correctly. First, it's better not to use the term "direct object" for German grammar. German has "accusative objects" and "dative objects" according to case, and these don't always correspond to the direct and indirect objects in English. For example the verb "helfen"/"to help" take a dative object in German but a direct object in English.
The main point is that "letzte" is just an adjective, so it's declined just like any other adjective, with the exceptions that it not used as a predicate and it's not comparable. Declining adjectives in German is rather complicated, even by German standards, but there are very few irregularities. So you'd use the "-r" ending in the same gender/number/case combinations you use it for any other adjective. This works out to several combinations starting with the masculine nominative case either with no article/determiner or with "ein" or "ein-word" such as "mein" etc. Examples of this are "Mein letzter Job war ..."/"My last job was ...", "Mein letzter Mann sagte ..."/"My last husband said ...". Another combination is feminine dative with no article. Such is the case with "in letzter Zeit", a common phrase which translates roughly to "lately". Finally, it's used with feminine or plural genitive with no article. I don't think this would come up all that much so I won't bother with an example.
As for answers to questions, these are usually single phrases, not complete sentences, so the usual rules of grammar don't apply. I assume you'd use the same declination as if the answer was filled in to make a full sentence: "Welcher Mann sagte das? Mein letzter (Mann sagte das)."
Anyway, this question might be closed because technically it could be answered with a declension table, but a bit more explanation seems warranted.