Remote API actually completely simulates like you were running on GAE. You can do almost everything you can with a deployed app. Sure performance wise it’s slower (hence the remote nature), but other than that it just works. You can simulate whole requests, and do whatever you please with the data in the datastore. No idea why did this never came up with Google support (or never bumped into it on the net) though.
I haven’t mentioned it yet, but we are a startup as well, and I see that as no justification to ignore optimization. Obviously every improvement requires an analysis if it worth it or not… but it’s no different from any other company. Also to be honest it seems highly unlikely to me, that moving/migrating to a completely new solution — with every additional cost it generates — is cheaper than actually fixing the code.
I get that you needed something that could be argued as a basic “musthave” feature that GAE should provide, but that doesn’t change the fact, that as far as I know it never promised to have that at all. Which means you should have known that at the very beginning when you choose to use GAE. So from Google supports’ point of view, they are actually right. They charge exactly for the amount of calls you do, and it’s not their task to give you statistics about it if they never said they can/would. Sure it would be nice, but I can tell you a lot of things that “would be nice”, yet I don’t expect them to do them for me. When we had high datastore costs as well, we started doing our own analysis and it worked.
My problem is that from all of your problems the only still valid I can see is that Google doesn’t provide a proper datastore cost analysis. It is only major if you can’t afford the time to implement your own solution, and you are spending a lot of money unexpectedly. Those are two very big ifs.