Your basic options are to take the bus or train. There are some private bus companies, but I don't have much experience with them and for the most part I think they only travel the major routes like along the highways.  The NJ transit website will give you all of the details you need in terms of routes, timetables, and prices.

When I lived in North Jersey and commuted into NYC I chose to take the bus for a few reasons.

1. The bus was cheaper.
2. I could walk to the bus station rather than have to drive to the train station.
3. The bus was quieter.
4. I didn't have to transfer.
5. I was working on 38th street and so port authority was closer than penn station and so I could avoid having to take a subway.

Obviously there are plenty of people that take the train because it is more convenient for them (arguably faster, no traffic, etc.). I found that the bus was 30 minutes late getting into the city one day out of every 2 months on average. I gave myself extra time in the morning so it was never an issue for me, but for you it might be. Trains aren't always on time either though so I'm not sure which you would consider to be more reliable.

As an extra note I also found the NJ transit bus drivers to be excellent drivers.

If you want to live near the transportation, Hoboken is good (very busy town). Rutherford and Lyndhurst have lots of city transportation and are very nice towns (a little more "suburb" than Hoboken). Pretty much anywhere in eastern Essex, Bergen and Hudson (maybe even Passaic) counties is a good commute to the city, as you can see on any map, they're rather close already. As for which towns, it really depends on what type of lifestyle you're looking for. Do a search on any particular town you choose (wikipedia gives good statistical info).