After nearly three months of hard work on a project, now the outsourcer is asking all the fellows on the same project to send him the TM's we all developed in order to try some in-house modifications to the end-customer's project. ... He supplied an initial TM, though very poor in quantity and quality, and I contributed with nearly 75,000 words, mainly caring a lot after the quality.
Did the outsourcer know that you would be using a CAT tool? If so, then it may be considered a reasonable assumption on his part that you would be generating a TM while you're doing the translations. The question you need to answer is whether it would be reasonable for such a client (knowing that you're using CAT) to think that the TM is one of the deliverables. Personally I think it is a reasonable assumption.
The fact that the client's supplied TM was crappy, isn't relevant to your problem, but the fact that the client did supply a TM may be relevant to the question of whether he was expecting an updated back at the end of the project.
Most CAT tools create TMs on the fly, i.e. as a by-product of the translation. But if you have one of those CAT tools that work with TMs but that doesn't generate a TM while you translate unless you do something extra, then the situation is different. For if it requires (or required) *extra* work for you to generate a TM for the client, then you have a good case to demand payment for it.