State-of-the-art machine learning methods for constitutive modeling typically require a significant number of tensorial stress-strain pairs, which can be challenging to acquire. Non-contact model identification techniques that leverage only displacement or force data could be convenient for experimental mechanics. Nevertheless, these inverse problems often require that the constitutive model’s form is known a priori or follows a predefined material library and, hence, is difficult to generalize. In this work, we employ neural networks to parametrize components of plasticity theory, eliminating the need for pre-selected bases. By recasting the classical return mapping into a smooth approximation, our approach enables gradient-based optimization to discover constitutive relations from displacement data alone. Using a material point solver as a backbone, the presented framework can handle dynamic problems and large deformations, making it suitable for various experimental setups. Numerical examples showcase the algorithm’s ability to recover well-known yield functions and hardening mechanisms from observed material responses without relying on stress or force measurements.