Other-race faces are discriminated and recognized less accurately than own-race faces. Despite a wealth of research characterizing this other-race effect (ORE), little is known about the nature of the representations of own-race versus other-race faces. This is because traditional measures of this ORE provide a binary measure of discrimination or recognition (correct/incorrect), failing to capture potential variation in the quality of face representations. We applied a novel continuous-response paradigm to independently measure the number of own-race and other-race face representations stored in visual working memory (VWM) and the precision with which they are stored. Participants reported target own-race or other-race faces on a circular face space that smoothly varied along the dimension of identity. Using probabilistic mixture modeling, we found that following ample encoding time, the ORE is attributable to differences in the probability of a face being maintained in VWM. Reducing encoding time, a manipulation that is more sensitive to encoding limitations, caused a loss of precision or an increase in variability of VWM for other-race but not own-race faces. These results suggest that the ORE is driven by the inefficiency with which other-race faces are rapidly encoded in VWM and provide novel insights about how perceptual experience influences the representation of own-race and other-race faces in VWM.