You'll need to have the full source code version and you'll need to rebuild the AspDotNetStorefrontCore project to create the debugging symbols. Then, within Visual Studio you can open the XSLTExtensionFunctions class, set your breakpoints, and debug.
In version 8.0.1.2, LookupProductImage has two signatures, one that takes 5 strings, and one that takes 6...but none that take an int:
Code:
LookupProductImage(string sProductID, String sImageFileNameOverride, String sSKU, String sDesiredSize, String sIncludeATag)
and
Code:
LookupProductImage(string sProductID, String sImageFileNameOverride, String sSKU, String sDesiredSize, String sIncludeATag, String sAltText)