My first yellow book is Evelyn Waugh's Brideshead Revisited. So what if the author came to dislike his own work later on (he was "appalled" after re-reading it in 1950) - I still love it. Whenever I see Brideshead Revisited at a bookstore, I always feel the need to buy it. I have two in English and one in Finnish, and I will probably buy more of them if I find a cover I really like.
My second pick is The Man Who Mistook his Wife for a Hat by Oliver Sacks. It was recommended to me by my former flatmate Izumi, who studied neurology back in the day. The book is full of interesting (true) stories about our brain, about how it works, and how disturbances in our brain can change the way we see the world and ourselves.
Next up is Elfriede Jelinek's The Piano Teacher. This is one of those uncomfortable books that I am glad I read but would never want to re-visit.
My last yellow choice is István Örkény's One Minute Stories. It was one of the first books I was able to read in Hungarian, and it has a very special place in my heart. The stories are superbly humorous, sometimes absurd or grotesque. If you like short stories by Daniil Kharms, you will love Örkény. Speaking of Kharms, a collection of his short stories would have been a part of this post - it is yellow - but I couldn't find it.