No, Wallet app is like any other app and has an icon you are required to tap to open. Double clicking the side button activates biometrics and Apple Pay. It is not the wallet. Settings, allows some parameters to be adjusted, but again does not offer the full gamut of settings. Please find the icon for the Wallet app on your iphone.
Tap the icon > tap the virtual card you want to delete > tap the more button (3 dots in a circle in upper right corner of the screen) > Card Details > scroll down > Remove Card > follow any additional instructions.
The card will not be available in Settings > Wallet & Apple Pay, and will not be available when you double clicking the side button. However, if you also added it to your Apple ID Account for use with iTunes, App Store etc. it will still be in your account.