Radiation torque on nonspherical particles in the transition matrix formalism