I want it to fit into the following sentence:

I would ______ leave like a phantom... (It's not finished yet).

P.S.: "Attemptedly" did not work.

solicitously
exertly

try to
I would try to leave like a phantom.
or:
I would attempt to leave like a phantom.

I think your problem is that you're trying to turn a verb into an adverb. You don't make adverbs that way.
But if you just gotta have an adverb in that sentence:
I would quickly leave like a phantom.

Earnestly, determinedly, assiduously