3D CAD definitely has it's place in the design process, for more-complex projects. I don't head to the CAD program, though, until I have a few sketches (pencil & paper) done. For me, 3D CAD replaces the 'first prototype' in the process (which sometimes was just paper and cardboard taped together--and sometimes was a quick test, in the same material as the project, of a specific joint that I had not used before).
For most projects, I find I can get away with 'a quick sketch on the back of a napkin.' For a project with more complexity, 3D CAD definitely has its place, once you are CLOSE to a final design. It can really help with visualization of complex joints and finalizing dimensions.
I actual enjoy the design process and using 3D CAD and I do this as a hobby. I was 'drafting' drawings of my projects long before there was 3D CAD. If you just want to build, don't waste a lot of time in the 3D CAD process. I look at myself as a designer who actually likes building the prototypes, so I don't mind taking the time to draw it first.
KDub