To be fair I haven't played much of Oblivion (perhaps thirty hours) because of the silliness I mentioned earlier in that it felt too 'gamey'. It was made for an audience with a background in casual games and instant gratification games like World of Warcraft, and I have a background in unforgiving, open-ended and character-heavy RPGs like Dungeons and Dragons, Nethack and its variants, and Baldur's Gate. The silly number-crunching of Morrowind probably appealed more to my geekish tendencies.
But I can tell you that the biggest thing that ruins Oblivion is Fast Travel. If I ever decide to play it again I'm going to use a mod that removes Fast Travel. Also the absence of a clear and easy-to-use map system. The Oblivion map looks pretty, but it's way too difficult to use (the overworld map is on one tab, the area map is on another). With Morrowind I could simply zoom in or out to show as much detail as I needed, and I could even reduce the size of the map window and leave it open on the screen while I played, as a mini-map. That kind of thing alone put hours into the game because I'd spend days just walking around to fill in the fine details of the map (and hence discover secret caves etc.).