Merchants are not affected by the exchange rate because bitpay, coinbase, btcchina etc. all offer to process payments for you and eat the exchange rate risk. You get paid in your choice of fiat currency.
Not enough consumers use it is the most accurate answer.
Bitcoin adoption among merchants...