An archeological dig on Fox Island in 1995 uncovered fortifications from Queene Anne's War in the early 18th century. The lower half of Fox Island is a popular picnic spot, and locals say that you can still find cannon half-buried in the ground. I've never come across any, though.
You can park in the community of Champney's West, off route 230. There's a semi-improved trail to the top of Fox Island; you can check out Alltrails for more details.
The steep climb up Fox Island requires some hands-and-feet scrambling, and probably isn't suitable for smaller kids. Medium sized kids and dogs, however, seem to like it plenty.
Left in the cache initially (twenty years ago, as of 2022!) are a couple of small glow sticks, keychain Monopoly and Mousetrap games, and some other odds and ends.
2022 UPDATE: This cache is now twenty years old, whaddya know! Recent finds seem to indicate that it's still in pretty good shape, but please let me know if the description above should be further updated. I updated an outdated trail link to Alltrails, which as of 2022 is a good link.