Highclere Castle was used for the orphanage. It was also used as the home of Lord Crawley, the Fifth Earl of Grantham and family in Downton Abbey (2010).
Time for lessons children! Come along.
This film has been criticised for the fact that Mary and Colin have ended up marrying each other despite originally being cousins in the book. However, this is considered a sequel to the 1987 adaptation and they weren't related in that.