Replica
One complete serving instance (a single GPU or a TP/PP group); the unit you data-parallelize and autoscale.
One complete serving instance (a single GPU or a TP/PP group); the unit you data-parallelize and autoscale.
One complete serving instance (a single GPU or a TP/PP group); the unit you data-parallelize and autoscale.
One complete serving instance (a single GPU or a TP/PP group); the unit you data-parallelize and autoscale.