The idea behind connection pooling is to keep connections open, yes, but to
share them among many users. Multiplexing.
The cost associated to db connections is not due to keeping them open. The
prohibitive cost comes in when every app or every query demands its own db
connection. This results in hundreds or thousands of connections open at
one time, and high rate of opening and closing of connections. This doesn't
scale. A small pool of re-usable connections solves this.
Think of New York City (or any densely populated area) and taxicabs. Lots
of people in NYC want to move around from one arbitrary point to another.
If each person had to have his own car to do this, as is done in the
suburbs, Manhattan would be filled with cars. Where can you put 20 million
cars? Instead, the city relies on a shared resource - taxicabs. People use
a cab to move from A to B, then return the cab to "the pool" of available
cabs.
Just because the passenger is done using the taxi, that does not mean we
want the taxi off the street and parked in the garage. When the passenger
arrives at his destination, he releases the taxi, and it is then usable by
the next passenger. A single cab might serve 20 fares in an evening.
Taxis are db connections. Passengers are queries or statements. Just
because the query completes, doesn't mean we want to close the connection.
When the query is done, the app releases the connection, though the
connection itself is not closed. It is returned to the pool to be used
again. A single pooled connection might handle 20 distinct queries in a
single second. The fact that each connection in the pool is "always open"
reduces the cost even further. The open/close cost is paid only once,
rather than continuously and repeatedly.
-D