The general fatigue behavior of metallic materials from low cycle fatigue (LCF) to very high cycle fatigue (VHCF) was analyzed and predicted based on three possible work energies which could affect to the fatigue behavior. It is well known that the fatigue crack initiation and propagation in metals are essentially influenced by to the accumulation of microscale material behaviors, such as irreversible deformations by the gliding of dislocations. However, a reasonable prediction of the fatigue behaviors covering both the LCF and the VHCF has been unattainable so far even with recent advanced theoretical models. Most of current theories for the fatigue life are focused on the significance of plastic deformations due to significant dislocation motions and related plastic work energy accumulation. Whereas these theories cannot be directly applied to VHCF case having fully elastic loading cycles. In addition, current theories cannot explain the wide variation of fatigue life in S-N curves especially for the VHCF case without introducing the uncertain heterogeneity of the target material. In this work, in addition to the plastic work energy as a representative origin affecting the fatigue behavior especially for LCF scenario, two more possible work energy sources based on anelasticity [1] which affect the fatigue behavior especially for VHCF condition were introduced in order to explain the distinctive fatigue behaviors observed only at VHCF conditions. The consideration of the three possible work energies may explain the difference between LCF and VHCF behaviors regarding the position of crack initiation (surface for LCF and internal for VHCF), the variation of S-N curves (relatively small during LCF and very wide during VHCF), and the effect of pre-deformation (weak for LCF, significant for VHCF [2]).