It is nice that it is numbers matching, but I don't think it really adds any big value because the car is not "rare" enough, ie a Ram Air car, or something unique like a 4.30 rear gear, radio/heater delete, manual steering -that type of thing. Sure, its a convertible and that has value, but I don't feel that the '64 GTO's are as desirable as the years that followed.
I think 20K is way too much considering all you have said about the car and if it needs a frame, well, there goes original numbers matching. Sad to say with it needing a frame replacement, its probably worth 1/4 of the asking price. Once you begin to strip off the paint, you might be opening a can of worms. I mean, frame, floors, trunk, quarters? lower doors?, lower front fenders? windshield channels? and what else? Looks like it needs an interior. The drive train may seem good, but are you honestly going to restore the body & interior and not rebuild the entire driveline, suspension,& brake system? (You could probably drop in a 455CI & 5-speed or 461CI and 4-speed with a 1965-66 tri-power and up the value over numbers matching in my opinion)
So what exactly are you buying again for 20K????
Now if it is a car you have to have and you have the finances AND know you will never reclaim your investment should you sell it, then go for it and don't look back. It is a nice looking car from 50 feet and perhaps you can give it the just due it deserves and restore it to its former glory.