It is very hard for me to believe that God, who is supposed to be all powerful and all knowing, would obscure His existence from the ones who are supposed to worship Him.

It just doesnt make sense.
Think about it, if you KNEW God existed, beyond any doubt, you would NOT fuck that prostitute behind your wife’s back because you would KNOW that you would be in, what would be considered serious trouble with the entity called “God” (or whatever appellation you gave it).

So the point of all this suffering CANNOT be that every man should act in a civilized way and be “good". Because if that were truly the desire of a God who is all powerful, then IT WOULD BE. Simply because He is omnipotent, ergo what He wants will be.

“But it is a test of man.” you say.
Why would a God with infinite wisdom, give a bunch of savage monkeys the free will to kill each other, if He truly did not want suffering?

Why would He, in His infinite wisdom, create a humans with less than God-like intellect, and then expect them to behave as if they had God-like intellect?