We study the influence of the large scale energy distribution on the long term dynamics of unstably stratified homogeneous turbulence at high Reynolds number $Re = 10^6$, using a statistical two-point spectral model based on the eddy-damped quasi-normal closure. We consider several initial spectral scalings ks in the infrared range with $s \in [1; 5] $ and we establish that the resulting kinetic energy growth rates are controlled by $s$, with the appearance of backscatter effects for s ≳ 3.5. We then assess that only for $s \le 4$ do we observe self-similarity in the infrared and in the inertial ranges, but not in the dissipative range. Compensated energy and buoyancy spectra exhibit the expected Kolmogorov-Obukhov $k^{−5/3}$ scaling at long time, and a trend to the theoretically predicted $k^{−7/3}$ scaling for velocity-buoyancy cross-correlation spectrum thanks to the very large Reynolds number. We also show a direct link between the late-time anisotropy of the flows and the infrared spectrum, thus demonstrating long-lasting effect of initial conditions on unstably stratified turbulence. We show that, in addition to the Kolmogorov $k^{−5/3}$ scaling, the kinetic energy spectrum inertial range includes a $k^{−3}$ zone due to polarization anisotropy, and we confirm the clear $\sin^2\theta $dependence of the velocity-buoyancy spectrum in the inertial range, where $\theta$ is the orientation of the wave vector to the axis of gravity. However, an unexpected quick return to isotropy of the scalar spectra has been identified, which cannot be explained by a standard dimensional analysis.