Gene expression is a stochastic process. Despite the increase of protein numbers in growing cells, the protein concentrations are often found to be confined within small ranges throughout the cell cycle. Generally, the noise in protein concentration can be decomposed into an intrinsic and an extrinsic component, where the former vanishes for high expression levels. Considering the time trajectory of protein concentration as a random walker in the concentration space, an effective restoring force (with a corresponding “spring constant”) must exist to prevent the divergence of concentration due to random fluctuations. In this work, we prove that the magnitude of the effective spring constant is directly related to the fraction of intrinsic noise in the total protein concentration noise. We show that one can infer the magnitude of intrinsic, extrinsic, and measurement noises of gene expression solely based on time-resolved data of protein concentration, without any a priori knowledge of the underlying gene expression dynamics. We apply this method to experimental data of single-cell bacterial gene expression. The results allow us to estimate the average copy numbers and the translation burst parameters of the studied proteins.