When I pause the music on my iPod touch, it starts playing again within a few seconds.

Thanks for clarifying, KatDeB.

Let's try force closing the Podcast app, restart your iPod touch, then test it again. That usually resolves issues like this. Here's how:

Close an app on your iPhone or iPod touch - Apple Support

Keep us updated on how it goes.

Best regards.

Hi KatDeB,

Thank you for using the Apple Support Communities! Is this occurring when using the Music app or another app? Are you paying from the Now Playing screen as shown in the guide below:

Play music on iPod touch

Thanks!

Have a great day.