CoRA-NAS: Coarse Ranking and Anchor-Residual Refinement for Neural Architecture Search

Abstract

Training-free neural architecture search ranks candidate networks from a few forward/backward passes at initialization, avoiding the cost of full training. Yet zero-cost ranking has saturated: under one protocol, no single proxy robustly beats the trivial #Params/FLOPs baseline across both structure-varying and size-varying spaces. We start from the observation that a trained network’s accuracy is driven by three complementary signals. Two are static and near-zero-cost: its capacity, and how well its structure uses that capacity at initialization. The third is its learning dynamics, visible only once training begins and read here at a small (≈1% of full training) cost. We realize this as CoRA-NAS, a single-configuration, coarse-to-fine method that does not use fully trained architecture-accuracy labels to fit its ranker. Stage 1 forms an equal-weight rank consensus over existing zero-cost proxies spanning the two static axes, with a consensus gate that adapts the bank per space without ever reading accuracy. Stage 2 spends ≈1% of a full-training budget on a stratified anchor set and corrects the static prior with a target-free early-curve residual. Across four vision spaces, CoRA-Refine has no weak regime on any of them: its per-space Spearman is 0.946 on NB201, 0.715 on NB101, 0.786 on TransNAS-Bench-101, and 0.894 on NATS-SSS. The refinement also selects strong architectures, reaching CIFAR-100 best-found accuracy 73.32, near the ground-truth best of 73.37. On the pure size space the unsupervised prior does not beat #Params, and the low-cost curve residual corrects this prior, yielding one configuration robust across all four vision spaces.