In this study, a soundscape restoration model was proposed in terms of audio-visual interaction based on psycho-physiological response by natural soundscape experience. To this end, audio-visual stimuli were collected in 6 sites of 2 types of natural environment (greenery and water) and 3 sites of urban environment (as control), and a laboratory evaluation environment was implemented using virtual reality technology. Sixty subjects participated in the experiment for two days, and the individual’s health status, personality and temperament, and sensitivity were investigated before the evaluation. In the experiment on the first day, a stress test (mental arithmetic) was performed before the stimulation experience, and questionnaires related to psychological recovery was answered after the stimulation experience. In all the experimental procedures, physiological responses such as heart rate, electroencephalogram (EEG), and eye-tracking were measured so that before/during/after response from experience could be compared. In the experiment on the second day, they were asked to respond to a questionnaire related to the soundscape and landscape of environment. Finally, the relationship between the audio-visual environment (soundscape and landscape perception) and the psycho-physiological response was investigated using structural equation model, and based on this, design guidelines for healthy urban city were proposed.