If you're looking for a reliable but cheap car, you might consider something like a Honda Logo.
One trick to getting cheap insurance is to pick a car few young people drive, that isn't powerful. This is a perfect example.
Of course, you might value how cool you look more than money, so in that case, don't do this.
For the record, my first car that I actually drove, was a 1994 clio 1.4, but I had the bonus of being 20.
edit: sorry about the massive picture I can't find a smaller one.