Gene expression is a stochastic process. Despite the increase of protein numbers in growing cells, the protein concentrations are often found to be confined within small ranges throughout the cell cycle. Considering the time trajectory of protein concentration as a random walker in the concentration space, an effective restoring force (with a corresponding "spring constant") must exist to prevent the divergence of concentration due to random fluctuations. In this work, we prove that the magnitude of the effective spring constant is directly related to the fraction of intrinsic noise in the total protein concentration noise. We show that one can infer the magnitude of intrinsic, extrinsic, and measurement noises of gene expression solely based on time-resolved data of protein concentration, without any a priori knowledge of the underlying gene expression dynamics. We apply this method to experimental data of single-cell bacterial gene expression. The results allow us to estimate the average protein number and the translation burst parameter.