Kinneil House is well worth a visit if you can find a day it's open. If nothing else, the Renaissance murals within the 'palace' (sticky-out bit on the north) are really quite stunning. The towers apparently contain fine marble staircases, although Joe Public doesn't usually gain access to those parts of the building.
Also worth a look is nearby Kinneil Church- one legend has it that St Serf stood here when he lobbed his staff across the Forth. It duly landed in Culross and the rest is history. There's some fine early 16th century graveslabs. The big tussocky area of ground south of the church is the site of the original village of Kinneil, apparently depopulated by plague and then duly cleared by the Hamilton's who fancied a sweeping vista of open parkland rather than a bunch of Baldrick-a-likes scratching out a living. Said peasants were decamped to a new settlement suitably far away on a headland, which the natives called Borrowstouness (Burgh Town On The Headland).
Look carefully on molehills and you can still find fragments of pottery. The broad, gentle depression running westwards through the meadow is the ditch associated with the Antonine Wall (2nd C AD Roman frontier).
I confess to a) being a native and b) being a former archaeologist...