Lille Restrup Hovedgaard is an old noble manor house, whose history can be traced back to 1459, when squire Niels Pedersen Skadeland received a deed to the manor. Around 1510, county judge Niels Clemmensen acquires Lille Restrup Hovedgaard, but after several disputes with the crown, King Christian the Second seizes the farm in 1518 and includes it as crown property. However, Christian the Second only possessed Lille Restrup Hovedgaard for a short time and already in 1520 the farm was given as a gift to Niels Clemmensen's widow. His daughter and her husband Christian Harbou later take over the farm, and a tombstone for one of their descendants and later owners of Lille Restrup Hovedgaard, Niels Harbou and his wife Berete Munck, is preserved and installed in the choir of the 1000-year-old Hvam Church.

Another of Restrup Hovedgaard's famous owners was Mogens Kaas (Sparre Kaas), who was a councilor under King Christian the Fourth. In the middle of the 18th century, Lille Restrup Hovedgaard came into bourgeois hands for the first time, when it was bought by the bailiff Chresten Sørensen Testrup, a skilled owner who, over a short period of time, rebuilt the breeding buildings and let the main building be surrounded by beautiful stone dikes, several of which are still standing.

Around 1750, the Testrup family planted the Linden alley, which are now  the venue for outdoor weddings. Back then, the Linden alley was the fine entrance to Lille Restrup Hovedgaard, and the entrance port being use today was reserved for those working on the farm. The Testrup family owns the farm in cå. 90 years and after this period, changing owners follow the main farm. In 1866 it was bought by J. H. A. Kjeldsen, who, after a fire, rebuilt the main building from split boulders. The manor house was built with inspiration from old English country style, which is also clearly visible in the park, where the stone dykes frame the area.

In 1948, J.H.A. sells Kjeldsen's descendants Lille Restrup Hovedgaard to publishing director E. Winther, whose son K.A. Winther later takes over the main farm. The Winther family still owns the farm today and currently the 3rd and 4th generation of the Winther family live in the main building. It was only in 1988 that Lille Restrup Hovedgaard transitioned from being a farm to running modern tourism. Today, Lille Restrup Hovedgaard functions as a beautiful setting for events and parties, where many celebrate their anniversaries and brides and grooms celebrate their love.