Having fresh clean water will help make great beer. I had to filter the local water because it seemed so chlorinated. The bad water ruined a good light lager I made a few years ago.
Brew your own Magazine had a great article on making your own water filter:
The only trouble I had is finding the water filter. I spent a long time walking around a big box store and couldn’t find it. Let me save you some time. Amazon.com is a life saver. They have the water filter you are looking for