Do not look at the title twice! Yes, this post is about non-GIS open source software. However, these open-source programs make great companions to any analysis.
For example, you may find the need for a traditional statistical software package. R Statistical Software can aid in importing, analyzing, and cleaning your data. You can perform traditional statistical analyses. There's even a spatial package, although you will better off sticking with open source GIS programs like GRASS or QGIS. A good overview of its spatial package can be found here.
Want to examine social networks? Then, Gephi's great! I just analyzed my Facebook network in only a few minutes after following a tutorial. In addition, Gephi has features and plugins to help you map geographic data.
At some point you may also need Python. Editing and organizing code, then give Notepad ++ a try.
A some point you will have to compress files, then 7-zip is a sure thing. You may want to playback some videos or animations and VLC Player works great.
GIMP is a image maniupulation program similar to photoshop. You can see an example of combining GIS with GIMP on a great GIS blog.
You will probably want to type up your results or make a few "PowerPoint" slides...so there's Open Office and the Libre Office implementation. If you need a standalone pdf creator, then there's PDF creator.
Lastly, if you ever want to venture away from Windows or other operating systems, there's Ubuntu--an easy installation of Linux. Be sure the open-source or for-fee programs you want to run have a Linux version before making the switch. Naturally, many open source programs have a Linux version but some do not.