You keep going back further and further, and at some point, something has to spring from nothing. It, and I'm not exactly sure what it is, can't start if that's not the case. Because the question is "how did the universe start", and the universe is everything that exists. So the only way for the universe to start is if it sprang from nothing, otherwise whatever it sprang from would be part of the universe, and we'd have to ask where that sprang from.
If the link made your head hurt, then don't skip straight to Brave New Universe I think you would enjoy some of the Hawking, though. It's been years since I read it, but I remember it being pretty good at coming up with ways to describe the complicated physical processes. Give A Brief History of Time a shot.
That's my point, I don't think it ever "started" per se -- it just has always been here, forever. And that tilts me.