[Background] An unreasonable land use structure causes severe soil erosion and increases runoff-sediment yield or even major environment problems, especially brings huge negative effects on ecologically fragile arid and semiarid area. The objectives of this study were to model the effects of the changes of land use structure on reducing runoff-sediment yield in the Luoyugou watershed, a typical case of gully region of the Loess Plateau. [Methods] The Soil and Water Assessment Tool (SWAT) was used to model daily runoff and sediment data from this basin collected from 1986 to 2000. First, the model was used to determine acceptable values of the P-factor, R-factor, coefficient of determination (R²), and the Nash-Sutcliffe coefficient (E_ns) during both the calibration and validation periods. Then, the model was used to determine which kind of major land use in Luoyugou watershed can be chosen with the least runoff and sediment yield. Next, the land use structure of Luoyugou Watershed were adjusted to six types, and the model was used to simulate the six scenarios with 1986—2000 meteorological data of this basin. [Results] 1) Compared with those in 1995, the effect on reducing runoff and sediment yield of forest and terraced fields were better than the grassland and slope farmlands. Under extreme scenario simulation, runoff under forest and terraced fields decreased about 56.86% and 11.97% respectively, while the sediment decreased about 80% of corresponding runoff-sediment yield in 1995. 2) The runoff-sediment yield was influenced by the concentration of forest land. Compared with the low concentration of forest, runoff amount and sediment under high concentrated forest decreased at most of 4% and 27% respectively. However, there was probably a threshold of concentration, and if the concentration degree was greater than the threshold value, the watershed runoff-sediment yield was almost invariable, but it remains to be further validated. 3) The runoff and sediment yield under various land use types with the same area proportion were related to their distributive location. Different structures resulted in varying reduction of sediment yield and almost undifferentiated runoff. The sediment yield in the drainage of the watershed reduced in a varied level in 1995, and maximum reduced proportion of sediment yield was 26.79%. Moreover, when forest land locates in upstream, runoff-sediment yield at upstream and midstream were the least. [Conclusions] Therefore, concentration degree of forest should be more considered in adjusting the land use structure. More attention should be paid to optimize the land use structure of upstream especially the source of the watershed. Vegetation measures should be implemented in the source areas producing runoff and sediment, and the terraced fields should be placed in the midstream and downstream as much as possible.