how can I unlock my iphone if I forgot my passcode

If you forgot the passcode for your iPhone, iPad, or iPod touch, or your device is disabled - Apple Support --> http://support.apple.com/HT204306

If it isn't working then you are not successfully getting it into Recovery Mode. Try again. Hold the power and home buttons down for quite a while to get the connect to iTunes display on the device.