Your actual card number is not stored on your phone. Only fingerprints stored on your phone, a phone passcode, or Face ID can authorize a payment. A distinctive transaction code and device-specific number are used by Apple Pay when you make a transaction.
If you store multiple fingerprints on your device including those of additional persons, those persons will also be able to access your Apple Pay via fingerprint when fingerprint authentication is enabled. You should only enable fingerprint authentication if you're the only person who has registered fingerprints on your device.