Some caveats. I have oversimplified, not taking account of transaction fees. I have not tried to be faithful to the syntactic format of a transaction; I'm after maximum conceptual clarity.
As is traditional, Alice is going to pay Bob 12 milliBitcoins, which I estimate today is worth £6.36.
The idea of Bitcoin is that everything is transactions. For Alice to pay Bob, someone else must have already paid her. I'm going to make it real easy and assume Alice has already received a single payment (from someone else) of 12 mBTC - this is called an 'unspent transaction output - UTXO', and she is merely going to 'pass along' that 12 mBTC to Bob.
This is how Alice got her 12 milliBitcoins
The transaction I'm calling TX: Pre was set up by someone else to pay Alice 12 mBTC. The payment was to Alice's Bitcoin address, a designator (like a bank account), which was constructed by hashing one of Alice's Public Keys and encoding it into an alphanumeric format (Base58). There are no security issues here, Alice published this Bitcoin address freely to accept payments. It all worked.
Alice constructs a transaction to pay Bob 12 mBTC
So TX: A2B (my name for the purposes of this post, not a Bitcoin term) is a new transaction. Alice (Alice's wallet software in fact) first inserts a pointer to TX: Pre, to its location on the public Blockchain - recall this is the source of Alice's 12 mBTC.
Secondly, she combines (most of) the fields in TX: Pre along with Bob's Bitcoin address to produce a hash code. This is then encrypted with her Private Key (corresponding to the already mentioned Public Key) to produce a digital signature. Only Alice can produce this digital signature as only Alice knows her own secret private key.
She then adds Bob's Bitcoin address as the destination, the amount of 12 mBTC to pay, and some other stuff (of course!) and the transaction is now ready to be launched onto the Bitcoin peer-to-peer network. Every node that receives it will check the transaction for validity (there are many checks in fact).
How can the Bitcoin network ensure Alice is the only one who can spend that 12 mBTC on prominent public display on the Blockchain as TX: Pre? Note that like all transactions on the Blockchain, TX: Pre is read only and cannot be altered. To do so would constitute a successful attack on the Blockchain - the 'confirmations' mechanism is one way this risk is mitigated.
The transaction validation process model (carried out by all full nodes)
A validating node (including Bob, who has perhaps the greatest interest in determining that Alice is legitimately able to pay him) first repeats Alice's hash.
TX: A2B has a pointer to TX: Pre so this can be retrieved from the public Blockchain and concatenated with Bob's Bitcoin address which is stored in the 'output locking script' part of TX: A2B. So these two items of data are hashed.
At the same time, Alice's public key (in the 'input unlocking script' of TX: A2B) is used to decrypt Alice's digital signature, uncovering Alice's original hash. If the two hashes are equal, this proves that Alice originated this transaction.
We finally make sure that Alice's Public Key (which we know to be valid because it unlocked Alice's digital signature) matches Alice's Bitcoin address (the latter is a hash of the former) from TX: Pre. Once this is done we are assured that the Alice who was the recipient of 12 mBTC in TX: Pre is the same Alice as the one who has originated (and signed) TX: A2B.
Bob can breathe a sigh of relief.
Note that formally, the above validation procedure is a concatenation of the 'input unlocking script' from TX: A2B and the 'output locking script' from TX: Pre.
And once TX: A2B has been lodged on the public Blockchain, transaction TX: Pre will be 'chained' to it, so will now be considered 'spent'. No-one else can spend it.
If anyone sees an error in this account which is more than just sloppy over-simplification, I'd be grateful if you could put something in the comments and I will fix it.
You may also be interested in my review of "Mastering Bitcoin: Unlocking Digital Cryptocurrencies" by Andreas M. Antonopoulos, and the diagram (from Satoshi Nakamoto's original paper) there.