The orientation-aware interaction-based deep material network (ODMN) is an advanced surrogate model for polycrystalline materials, capable of predicting crystallographic texture evolution while satisfying the Hill-Mandel condition \cite{Wei2025}. The trainable parameters in ODMN are intrinsically linked to the geometric characteristics of the representative volume element (RVE). However, a major limitation of ODMN is its requirement for retraining whenever the microstructure changes. To address this challenge, we introduce a graph neural network (GNN)-based framework that encodes polycrystalline microstructure information and maps the extracted latent representations to the ODMN parameters. Our results demonstrate that the GNN-ODMN framework accurately predicts both mechanical responses and crystallographic texture evolution under previously unseen microstructures and varying loading conditions. Furthermore, inspired by the success of foundation models in natural language processing (NLP), we employ a masked autoencoder (MAE) as a foundation model for pretraining three-dimensional voxelized microstructures. By leveraging transfer learning, the pretrained model enables accurate prediction of ODMN parameters with significantly reduced data requirements, achieving robust mechanical response and texture evolution predictions under unseen crystallographic textures.