Bài báo quốc tế
Kho tri thức
/
Bài báo quốc tế
/
Training dynamics and state taxonomy in deep visual recognition networks
Training dynamics and state taxonomy in deep visual recognition networks
Lã Quang Hải
Training deep visual recognition networks is usually monitored using loss and accuracy. These measures show whether a model is improving, but they say little about how representations inside the network change during training. This paper proposes a dynamical systems framework for studying this internal evolution through layer activations. At each epoch, we compute three quantities from the activation trajectories. These form an effective integration measure based on detrended fluctuation analysis, a metastability measure based on the Kuramoto order parameter, and a composite stability index that combines the two. We apply the framework to nine architecture dataset configurations, including ResNet variants, DenseNet-121, MobileNetV2, VGG-16, and a pretrained Vision Transformer trained on CIFAR-10 and CIFAR-100. Under the present parameterisation, three retrospective patterns emerge. First, the integration measure separates CIFAR-10 from CIFAR-100 across the architectures studied and remains stable under broad hyperparameter variation. Second, the rolling volatility of the composite index often decreases before the accuracy curve plateaus, suggesting a candidate convergence signal that requires prospective validation. Third, the coupling between integration and metastability helps distinguish models that settle into richer representational regimes from those that remain more rigidly coupled. These observations motivate a retrospective four-state taxonomy of training regimes, namely Stable Convergent, Metastable High Integration, Partial Integration, and Rigidly Synchronised. The taxonomy is intended as an exploratory description of the configurations studied, rather than as a validated classifier. The results provide a foundation for future multi-seed, held-out, and prospective studies of dynamical diagnostics for deep visual model training.
Xuất bản trên:
Training dynamics and state taxonomy in deep visual recognition networks
Ngày đăng:
2026
Nhà xuất bản:
Discover Artificial Intelligence
Địa điểm:
Từ khoá:
Training dynamics; Dynamical stability; Hierarchical integration; Metastability; Convergence analysis; Deep visual models; Hurst exponent; Composite stability index; Image classification
Bài báo liên quan
MIST: A Multilingual Dataset and Benchmark for Fine-Grained Audio Inpainting Tampering Localization
Vũ Sơn TùngNF-DCL: Enhancing video anomaly detection with synthetic normal features and Debiased Contrastive Learning
Nguyễn Thu NgaA Workflow-Oriented Architecture Integrating Large Language Models for Automated Multi-Platform Content Management
Nguyễn Tất Thắng