The deformation behavior of sustainable secondary wrought Al alloys is strongly influenced by brittle intermetallic phases, which arise from high impurity content, for example, due to the use of cast-scrap in the recycling process. The shape, size, and morphology of these phases significantly affect mechanical properties such as strength and formability. To enable a microstructure-centered approach to alloy design, efficient numerical models tailored for such heterogeneous microstructures are essential. In this contribution, a phase-field fracture model for ductile Al alloys containing brittle intermetallic inclusions is presented. The model is implemented within the mesh-free FFT-based micromechanical framework DAMASK (Düsseldorf Advanced Materials Simulation Kit), which has been extended with a fully implicit phase-field fracture formulation. To evaluate the capabilities of the proposed model, several numerical studies are performed, providing insights into the mechanics of these Al alloys. Particular emphasis is placed on the robustness and scalability of the calculations, recognizing the necessity of high-throughput simulations for (inverse) alloy design approaches. Furthermore, the potential integration of the presented method into a Bayesian optimization scheme for predicting advantageous microstructures is discussed. The results demonstrate the potential of the proposed method to address challenges such as meshing complex microstructures and the reduced robustness of large-scale phase-field simulations, which are common limitations in conventional finite-element approaches.