OpenAI Reportedly Finds New Optimization Method That Can Reduce Inference Costs by Over 50%

Odaily Planet Daily News: OpenAI's engineering team recently informed some colleagues that the company has found a new system optimization method that can reduce the "inference" cost of AI models by more than half. Inference cost refers to the computing resource cost consumed when a model is actually running and responding to user requests. This optimization mainly comes from improving the utilization efficiency of existing server resources, rather than relying on new computing chip investments. This development reflects that while AI companies continue to compete for computing resources, they are also improving the usage efficiency of existing infrastructure through software and system-level optimizations, in order to alleviate the pressure of rapidly growing model operating costs. (The Information)