Getting out of a sale if mold remediation costs are too high?

I have placed and offer to buy a foreclosure with an FHA loan, now the other realtor is claiming that there are signs of mold in the basement (very questionable in my opinion) And wants the deal to be a 203K. My question is what if the appraiser comes back and says that mold is a problem and the remediation cost are going to be astronomical, is there a way at that point to say not interested in property and move on?