LinkedHashMap vs LinkedHashSet for retrieving specific elements & retrieving in insertion order

I'm doing a problem that clearly requires usages of Set but I need to retrieve elements from a set in insertion order + retrieve specific elements.

But finding the specific element was too slow( I guess O(n) since I have to go through the entire set to find & return it ).

So I went with a LinkedHashMap<someClass,someClass> where the key-value mappings contained the same objects.

Although this was faster, it used up twice as much memory which is especially concerning if my key/value(both same anyways) happened to take up alot of space.

I'm hoping if anyone has a better solution to my problem or optimizations.

Edit: By the way the comments for this SO answer may help

Read more here: https://stackoverflow.com/questions/65702840/linkedhashmap-vs-linkedhashset-for-retrieving-specific-elements-retrieving-in