It really depends on the situation, and your own or teams preferences and what you want to achieve.

For readability and understandability it is good to clearly organize and label (comment, indent and space out) your styles as well as separate them into common groups. If you have a single file, it can become a bit long with lots of content, and this affects readability, and the ability to find a specific portion of the file.

So if the CSS file becomes too large, you might want to consider separating different sections into different files. This is largely based on your CSS editor also. Editors that do not manage multiple files well can be a hindrance. Screen size also plays a factor, as large screens are well suited for single files.

Too many files can make things more complex then necessary. Here, it makes more sense to make each file more readable, then to separate them.

It also helps to have good version control. For this you can use SVN, or GIT.
SVN - http://subversion.tigris.org/
GIT - http://git-scm.com/
This helps the management of the CSS files as well as other files.

For organizing CSS see:
http://www.smashingmagazine.com/2007/05/10/70-expert-ideas-for-better-css-coding/

To consider how single or multiple CSS files affect load time. The browser usually has to wait for all the files to load before rendering styles. Multiple files mean that there are more HTTP requests made, and this usually means it will take longer.
There are many factors affecting how the CSS files will load. Browser usually only have a 2 connection limit per domain (2 TCP sockets) concurrently. So multiple files may or may not be downloaded concurrently due to this limit. There are other factors such as HTTP version used, support HTTP features on both server and browser (eg: HTTP pipelining), intermediate or browser cache, etc.

For loading multiple CSS files see:
http://www.cherny.com/webdev/53/on-http-page-load-times-multiple-file-requests-and-deferred-javascript

I prefer to use one CSS file. I delineate each section (the reset section, layout, typography, etc...) using commented out lines of dashes/underscores, and navigate around it using whatever code editor I'm using's "find" functionality.
Having multiple files can get confusing, and requires more HTTP requests (even though it's only for the initial pageview).

The only reason to use multiple CSS files is if you are going to share them across different sections of a site or across multiple sites. Think of it like having common files in another computer language.