If there’s a unique kind or file type for version 5 documents you can identify them that way. Otherwise, you’d have to open the file and get it’s “flow version” with Quark.
set testFile to (choose file)
tell application "QuarkXPress"
open testFile use doc prefs yes remap fonts no do auto picture import no without reflow
return flow version of document 1
-->13 (Quark 7)
-->9 (Quark 4)
end tell
(NOTE: If ‘suppress all warnings’ is used in Quark’s open command, the flow version is always converted to the version of the application opening the document.)