The third, winning way was presented by serverless-apig-s3 plugin. The plugin does two things. It puts web content on S3, as pointed in
./web in my case) and enables the web access. It also configures API Gateway to serve the static content from the same domain, sidestepping any CORS issues. There are some other plugins for handling the Web Frontend but to me sidestepping CORS wins.