Spock determines that since the gas grenade didn’t work, nothing is real, and nothing will hurt them as long as they are certain it isn’t real. He mind melds with everyone so they share his certainty. I think Spock is taking it really hard that his gas grenade didn’t work. Either that or Scotty just didn’t believe it would work and that’s why it didn’t put him out?