In the research of the dynamic performance of the soil-structure system, input ground motion characteristics are key factors that must be considered. In this study, the shaking table test was carried out on the structure system model of the tunnel and aboveground building under different seismic inputs, including two seismic acceleration records with different frequency spectra and an artificial ground motion time-history. The influence of the spectra and the velocity pulse characteristics of earthquake ground motion on the dynamic response of the tunnel and the aboveground building structure was analyzed. The results indicate that for the seismic response of the site-tunnel-aboveground structure system, the interaction effect between the tunnel and the aboveground building is significant, and the existence of the aboveground building has a more significant impact on the higher frequency component of the seismic response of the underground tunnel; the interaction effect is significantly affectedby frequency spectra of the input ground motion; the influence of the velocity pulse characteristics of the input ground motion on the tunnel response is mainly in the higher frequency range, while the influence on the response of the aboveground building is not only in the higher frequency range, but also significant to increase the response in the frequency range of period 1.0 s to 2.0 s.