1998 Avenis 1.6l petrol starting problems

[Pneumo]

Guys,

I'm in need of some advice.
I have a trusty Avensis with practically no problems excepts for a few hiccup's.
A few months ago I had some starting problems, engine would turn, but didn't start.
After some searching it was a faulty petrol pump in the back.
Now a few months later I'm having some starting problems again.
I was on the road for a 20 minute drive, stopped at the side of the road, wanted to leave again after 5mins and it happened...
Engine starts, turns over, then dies...
I try again and again, always the same.
Starter engine turns, engine fire's up for a second, then dies.
I'm was thinking again at the petrol pump as it seems it's just dying as I would've turn of the ignition, but it has just been replaced (and in that case, it wouldn't fire up, just turn over)
Anyway, I wasn't in a hurry so I minded my own business for an hour, tried again, but to no avail...
My brother was in the neighborhood so he drove me home.
Next day, I'm calling in the troops to tow my trusty (ahem) steed back to my place.
I fetched my spare keys to be sure.
I enter the car, turn the ignition, it starts for a second, then dies.
I use my spare keys => car starts???
I shut it down, switch keys to my "primary keys" car starts, no problems.
Now, everything is dandy again, I've been driving it the past week.
But I need to find the culprit here, cause I can't trust my car anymore...
Anyone has any ideas?

Cheers