The fieldstone border helps the gravel stay in place ... You might think you need a concrete or paver patio underneath your pergola, but as seen here in this outdoor space from @myhousemylifeandme ...

Clean the edges of the patio where you'll later glue down the border pavers. Clean concrete means a stronger glue bond. Scrub the border area with a concrete cleaner. Any stiff brush will do ...