Hello, @
kefallon
Your new sub-domain should have exactly the same directory setup as your main domain, so your subdomain's directory should be /httpdocs/forums again. After correct setup, without installation our product, you should be able to open your sub-domain e.g.
http://subdomain.site.com and see your main site installation content. Once this is correctly setup the product can be installed on the site and you will be able to customize how each URL behaves and looks from central Admin. So the product this does not create a separate installation of XenForo, it just makes the same installation to look and behave different based on address used to access it.
Thank you!