Try out gnu radio, you build flow graphs in a GUI and it executes them in real time.

You really should play with audio samples from your system to get a feel for things, before diving into the math. You can also use it with a $30 rtlsdr dongle.