The reality is probably far more nuanced that you expect, but I'm guessing your experience is tied to specific users and their setup.
Also, the sheer number of permutations and combinations mean that the following is somewhat speculative - I have not exhaustively tested all the different ways, but is a somewhat educated guess.
First, a history lesson.
In the beginning, there was SMS - Short Message Service. Basic, bare-bones, text only (≤160 characters). SMS only needed a basic cellular connection since the message was stuffed within the cellular control network (and, incidentally, is the root of Twitter's original 140 character tweet length limit)
Then came MMS - Multimedia Messaging Service - an extension to SMS that supported images, audio, and other media. It requires a data connection in addition to the cellular control (the initial message is sent over the control channel, with the image/audio/long text sent via a corresponding data connection).
Then, some time later, Apple introduced iMessage. This added more features such as encryption and rich content, and uses an entire data connection (no cell service required), which enabled it to work on Macs as well as iPhones.
Then, some time after that, Google led the development of RCS, partly to compete with iMessage (which is Mac/iOS-only). RCS also uses a data-only connection, so works outside of the cellular network space.
Now, SMS had no concept of delivery confirmation, read receipts, etc. You send a SMS and hope. If you get a reply, you know the message was delivered :)
MMS added limited delivery confirmation, but only in that the recipient's cell provider has sent the message, not that it was received at the end user's device.
iMessage and RCS both include optional end-to-end delivery and read receipts.
So, to your specific question, you first have to understand HOW the message is being sent to the end user.
If the user is a Mac/iOS user and you're using iMessage (messages appear in blue), then you should get a confirmation that the message has been delivered.
There may be cases (I haven't tested) where the message is delivered to multiple devices (e.g. the recipient's iPhone, iPad and Mac). I suspect the first device that gets the message (let's say the user's phone) will respond with a Delivered response. If the user then replies from another device, it might, concievably not mark as Read since this is on a different device from the one that confirmed delivery. Just a hunch.
If the user isn't on an Apple Device then the message is either using SMS, MMS, or RCS.
If they're using RCS, and read receipts are enabled, then you should get a confirmation. I don't play much in the Android space, so haven't tested edge cases here, but I assume a similar set of actions as iMessage.
I'll assume for the purposes of this discussion that the message isn't using RCS, and they're not on an Apple device. That means the message will be sent via MMS or SMS.
If the message is short (≤160 characters), then it'll go via SMS and no delivery or read receipt will be sent.
If the message is longer (>160 characters) or includes images, audio, or other media, then you may get a delivery receipt when their carrier has sent the message to their phone, but you won't get a read receipt since MMS doesn't support read receipts.
<phew> got all that?
In summary, a lot depends on what system/device the end user you're sending to is running on. This determines what protocol the message will be sent via. Even then (assuming SMS/MMS), there will be differences based on the content of the message (short message - no receipt; long message - possible receipt).
So that, I think, explains why the perceived inconsistency. There are many variables at play and you're not even aware that the underlying message technology may vary between messages in the same thread which makes it harder to figure out.
In the case of getting a reply to a 'Delivered' but not 'Read' message, I suspect the message is being sent via MMS, since it's longer than the SMS protocol allows, and MMS doesn't tell you when it's read.