Machine-learning (ML) models could enhance climate simulations by accurately representing small-scale processes like turbulence and convection. However, it’s unclear if better standalone (offline) performance leads to better integrated (online) performance in climate models. In this preprint, researchers, including Pierre Gentine, conducted extensive experiments with 2,970 hybrid simulations, finding that reducing offline error generally improves online accuracy, but some decisions can destabilize the model. Key improvements include incorporating memory, training on diverse climates, converting moisture input to relative humidity, and avoiding certain error metrics. This study answers crucial questions about ML design for parameterizations in climate modeling, paving the way for more reliable and efficient simulations.