This is not an easy question to answer, as it really depends on the authors themselves. Take the example of writers who are writing a series. The only way to ensure that their readers buy their next book is to make sure that they have given them enough to ponder over. Cliffhangers are one example. It’s a way to keep everyone hooked. Engagement is really a good thing and ensures vitality both for the author and their creation. 

The other reasons are a little more complex and may sound incomprehensible unless you write yourself. Sometimes, authors like to keep things open to make sure the readers engage with them. It is done just to make sure that readers discuss and try to understand what really happened or what may happen next. The discussion sparked from this will then help relate it with the society–if it is indeed a commentary. This, too, helps in ensuring vitality for the authors and their works. Sometimes, suspense helps maintain the idea that life is unpredictable. You cannot know everything, simple as that. Sometimes, it’s simply an artistic choice; the author has told you whatever they needed to tell you, now it’s your turn to mull over all that information and engage with it.

It is just how literature works. A good book will make you want to think it over, and a good writer will know how to do it. Yes, sometimes it is done poorly and can be annoying, but if it is done poorly and is annoying, then you will know the author’s actual intention: they just want you to buy their next book.