Karting is good advice, or look at one of the formula series... Formula Vee is an option and you won't have to sell a kidney to buy a second hand car.
It's a process and you need to be aware of that going in. An expensive process to be sure...
You'll need to start with regional races. Not everyone has the talent and budget to win at Nationals. That means a single-seat, open wheel car... That's the start and you work your way up... if... you have talent (or a *lot* of money).
Formula Libre -->
http://www.formulalibre.co.za
You need a club membership and you'll also need to get an MSA licence to compete.
MSA -->
http://www.motorsport.co.za
Side note:
http://www.motorsport.co.za/DisplayLink.aspx?group=Starting Out in Motorsport&name=Starting Out in Motorsport
And:
http://formulavee.co.za/index.php/getting-started/
You'll need storage space, a trailer and you'll probably need someone who can work spanners too... Fact is, you're going to bump into things and you need someone who can put them back together again. Your car goes through scrutineering before every race, so your guy needs to know his stuff.
Also, constantly buying new tyres is going to make you cry. You can quote me on that.
You'll need racing gear and a helmet. Nomex, heat to toe and a HANS device is worth every penny. Don't go cheap here, the life you save could be your own. Gear gets inspected before every race too.
There are folk that offer rentals and that might be an option before jumping in the deep end.
One thing I have found is that racing is community. If you head out to a regional, folk will happily talk you through their experiences... That helps a lot.