Over the years, on-the-fly fine-grained sketch-based image retrieval (SBIR) has demonstrated significant potential for real-world applications. However, previous methods rely on weight-sharing and attention mechanisms, which increase the computational load and reduce accuracy. This study enhances feature extraction and task differentiation by eliminating weight sharing, thereby preserving strong feature discriminative power. Our model achieves 82.04% Acc@5 and 92.57% Acc@10 on QMUL-Chair-V2, and 66.37% Acc@5 and 80.78% Acc@10 on QMUL-Shoe-V2, outperforming Bi-LSTM, Multi-granularity and LGRL models, demonstrating improved retrieval performance.