In HTTP/0.9 (not used anymore), each request uses a separate TCP connection, and the end of a response is signalled by closing the connection.
In HTTP/1.0, an unofficial but very widely supported "Connection: Keep-Alive" request header can be used to request a persistent connection if the server supports it.
In HTTP/1.1, persistent connections are the default, and the old single-request behavior has to be requested explicitly.
Optionally requests may be pipelined, though this still has some disadvantages, so multiple (2–5) connections are often used.
HTTP/2 (aka SPDY) was specifically designed to multiplex many requests at the same time.
Resources: Wikipedia article and RFC 2616 section 8.1.
