Lyme Regis is probably most famous for two things: In the 19th century, a local woman, Mary Anning discovered many very important fossils. She discovered the first Plesiosaur and Ichthyosaur skeletons. These are world famous, and date from the Jurassic age. In recent years, the coastline either side of Lyme Regis has been classified as a World Heritage Site (which puts the "Jurassic Coast" as it is now known, alongside other notable places such as the Great Barrier Reef and the Grand Canyon).
The town is also famous for scenes from the film "The French Lieutenant's Woman", probably the most dramatic being the scene where Meryl Streep is out on The Cobb breakwater on a stormy day with waves crashing over all around her. While you are in Lyme Regis, if the sea isn't too rough, why not retrace Meryl's steps along to the end of the sea wall?
The stone steps set in to the side of the Cobb just to the west of the cache are called "Grannies Teeth". Have a go on them, it's a unique experience!
You will probably need to pay for parking in the town unless you are very lucky.