Lies, damned lies, and LiquidSky

Note: This little post has gotten bigger over time as I added stuff. Right now it’s not really well organized and somewhat messy. I haven’t had time yet to rewrite it to be more clear. Many of the criticisms found within still apply though some parts will need updating.

LiquidSky relaunched their service late last week and the rocky release and apparent misinformation in pre-release marketing has left many people disappointed and wondering what is happening.

If you have no idea what LiquidSky is, to make it short: It’s yet another “Cloud Gaming” service, using IBM Softlayer/Bluemix Virtual Servers as a base [1][2][3]. They got some publicity by advertising as the first “Free” gaming PC in the cloud using an ad-supported model.

The issues I will be adressing in this little rant are things that were promised or announced in the lead up to the launch of “LiquidSky 2.0" but turned out to be false or misleading as well as issues and problems that arose in the three days after the launch.

Little update (March 2017)

There has been a response explaining that the release was rushed due to a “leak” of the new client. This would explain some of the issues related to server capacity and backend. However this also means there would have been yet another delay that should have been announced earlier instead of leaving the community in the belief that the release was still going to happen that week. After the “We are in the final moments before our Official Launch!” update and the release of the launch day FAQ on thursday there was no attempt to dispute this belief before the alleged leak on friday. In fact, a post by Justin a few hours before the “leak” heavily hints at a release later that day. If they did in fact plan to release on monday these misleading posts were definetly not helping.

Since they claimed earlier that their “client is ready to be released”, and it’s unlikely that any outstanding issues would have been fixed during the weekend, I still consider those complaints to be valid. The main issue about misleading information prior to launch that remain uncorrected also still stands.

Another small update (August 2017)

Since I’ve written and published this post there have been numerous additional examples of LiquidSky lying or making changes that have had a negative effect on the user base. They also have limited the forums to users with accounts, and made the bug report forum private. I will update this post later but some of the changes I’m referring to are:

  • Shutting down 6 PoPs without any announcement which lead to data loss and customers no longer being able to use the service
  • Removing “Elite” and downgrading the performance plans…
  • … then “upgrading” them again (still lower than originally)
  • Removing the cheapest packages (9,99 subscription and 4,99 Pay-as-you-go)
  • First saying that Vega is ready on several occasions, and then claiming it takes more time and will be deployed in fall
  • Massive storage performance issues for over one month (from SSD to worse-than-HDD performance)
  • Introduced framerate limit at 67 fps (might have been there before)
  • Somehow framerate is now messed up, 30 works, 60 is 55 and “unlimited” (real value is 100) actually gives you 60. On Android 60 equals 45.
  • Limited ad supported users to the “Gamer” plan

They also still haven’t implemented USB passthrough, the Windows Store (though the CEO claimed to have talked with people at Microsoft, including their CEO…), and some other features that were initially advertised. Availability issues for paying users also remain.

Of course they also did not fix any of the security or privacy issues as well as continuing to violate open source licenses.

At this point I would recommend against giving LiquidSky any money.

If you really need graphics accelerated VMs or Servers there are cheaper alternatives such as the OVH public cloud offerings with GTX 1060 or 1070 cards and Hetzner’s GTX 1080 dedicated server for 99€/month (excl. VAT). Both of those also have CUDA and OpenCL available, which is not the case on LiquidSky.

And if you’re considering it as an alternative to a gaming PC: don’t.

I’m sorry that this post has gotten a bit messy. I plan to update it soon with sources for the claims above as well as restructuring it a bit to make it more clear. I will also remove or update issues that have been fixed.

Yet another, even smaller update (December 2017)

So it’s now December and almost 2017, Vega has been out and in consumer’s hands for a while and even a prosumer Volta GPU is available for purchase. So where is LiquidSky at?

The good news is that storage issues have been resolved and you now get 200 GB for PAYG plans instead of 100.

The bad news is everything else.

The Free plan has been removed with no plans of it ever returning.

With this one of the biggest selling points of LiquidSky is gone. Turns out that ads can’t really pay for the “World’s first free gaming PC” in the cloud. Free plan customers had their accounts deleted, regardless of any credits that might still be on that account. The only way to keep those credits and the account was to pay $10 for the cheapest package (at a 50% discount).

LiquidSky is still running Maxwell GPUs with no ETA or update on the Vega upgrade.

10 months after the announcement it’s still not here. It also appears that the post I was referring to in my last update, mentioning that upgrades are planned for fall and that they are in talks with nvidia and AMD, has been removed from the forums.

Security and open source licensing issues have not been fixed either.

And on another note for altneratives: Paperspace and Parsec now collaborate which enables you to launch a Paperspace VM through the Parsec website. For $7/month you will get 250 GB of persistent storage, add $0.40/hour for a Quadro M4000 (US) or Quadro P4000 (EU) GPU or $0.65/hour for a Quadro P5000 GPU. The performance of these GPUs should be similar to the GTX 970 (M4000), GTX 1070 (P4000) and GTX 1080 (P5000). When charging your account with $20 and then using the P4000 option in the EU + 250 GB of storage you will get a much better GPU, much better CPU, more RAM and VRAM as well as more storage and play time than the $20 PAYG LiquidSky option. It’s just *slightly* less convenient to set up and they do not offer a good android app.


The BF1 Demo

This is one of the first things some people noticed when jumping into the 2.0 Beta. LiquidSky 2.0 was demoed with BF1 at CES and the AMDs press event. Yet you cannot run the game on their Platform because of outdated Nvidia drivers. It also does not seem possible for the user to update those drivers. The same seems to be true for the new Mass Effect game, another title used to advertise the platform.

GPU performance

In an earlier AMA with the CEO (Ian) he was asked about the GPU used on LiquidSky 2.0 [1][2], his reply was:

Hello,
It is a Tesla GPU which is comparable to a GTX 1080ti.
-Ian

The card currently used by LiquidSky in all performance packages is the Nvidia Tesla M60, this model has two Nvidia GM204 GPUs which is the same chip used in the GTX 980/970. They are definetly not comparable to a 1080 Ti or even the 1070. It is quite hard to find any benchmarks using this GPU, a GFXBench OpenGL comparison is about the only thing I found and it indicates a performance slightly worse than the GTX 970. This is not too surprising considering lower clock speeds and power profiles of the Tesla cards compared to consumer GPUs. Performance on LiquidSky is probably also a bit lower than that due to virtualization overhead and more limited vRAM (in lower plans).

It’s also not entirely clear to me how GPUs are split between users; in the lowest plan (“Gamer”) the GPU is visible in the OS as a “Tesla M60–8Q” which according to Nvidia vGPU guide means one GPU per user with 8 GB of vRAM each, however only 2 GB are available. It seems that they will still assign a full GPU to each user but limit vRAM and core clock.

It is also worth noting that the answer in the AMA (Jan 20th) was given before the 1080 Ti was officially announced (Feb 28th). Nvidia also does not currently offer any Pascal based Tesla cards that support GRID vGPU. This does leave a bit of a suspicion that he was not being truthful, intentionally or not. And there was no attempt made to correct the wrong information before the launch.

Another announcement sometime later was that LiquidSky will also be using AMD Vega GPUs (AMD presentation at GDC, AMD press release), this does not yet seem to be the case. However there is no clear statement on when those GPUs will be available, though it was implied that they will be available at launch.

Update:

It looks like CUDA, OpenCL and PhysX are not supported in any of the plans. I also ran some of the Nvidia Vulkan samples to see if Vulkan is supported. Notably the “Vulkan Threaded Rendering” sample ran 12x faster on my GTX 980 (stock clocks) compared to the Elite plan. It also appears that the Fire Strike results vary a bit even though hardware across regions should be the same. It appears that all plans have very similar results in the graphics tests and that the main difference is due to the physics test’s result being much better on Pro/Elite. In conclusion there seems to be little reason to go above Pro for most games unless Elite receives a higher power GPU which might happen with the integration of Vega.

Update 2: Someone on the LiquidSky forums has linked this video in which Ian says in relation to the Elite package:

It’s for the most part gonna be at least you know a 1080 on the backbone, but a higher clock so it’s gonna be a pretty big GPU running that one

There also have been two more threads (#1, #2) talking about the lacking performance. The only response so far has been that “BENCHMARKS DO NOT WORK ON LIQUIDSKY!” which, given the similar results to what I expected (GTX 970), seems to be not entirely true.

Unlimited Plan grandfathering and delay compensation

In the aforementioned AMA, roughly one month before the official launch, the CEO (Ian) mentioned that users who were subscribed to the older “Unlimited” plan would be able to keep a grandfathered unlimited plan [1][2][3], this didn’t happen and the FAQ states you will have to pick a new plan[4]. Furthermore it was announced that as a compensation for the delay, active subscribers would be granted 2400 “skycredits”, this was later clarified to only be awarded if you resubscribed [1][2].

Skylounge not implemented

One of the features announced at CES and later also advertised is the “Skylouge”, essentiall a chatroom with some HTML5 games. In the current Client this was replaced with a “coming soon” image.

Delays

The delays were blamed on the server provider (IBM/Softlayer) for not getting the hardware ready in time, how true this is I cannot say. At the very least the state of the client and server software suggests that late hardware provision might not be the only issue that resulted in delays…

No two-factor auth

This was named as a feature of included in the release here: https://blog.liquidsky.tv/2017/02/24/preparing-launch-get-ready-fly-new-liquidsky/

No Windows Store

This was named as one of the benefits of switching to Server 2016 during their CES presentation (see the PDF on page 18), but is not available at this point. (Also mentioned here: https://blog.liquidsky.tv/2017/02/24/preparing-launch-get-ready-fly-new-liquidsky/)

Missing/Broken USB passthrough

This does not seem to be avilable yet, also announced here: https://blog.liquidsky.tv/2017/02/24/preparing-launch-get-ready-fly-new-liquidsky/

Servers not available for paying customers

People who purchased a subscription to skip the line cannot use the service due to lack of available servers. See this thread (amongst others).

Update: According to Jason’s post not all servers are live due to the rushed release and more are going live this week.

Lots of bugs in the Client/Server

There’s a comprehensive list here: https://community.liquidsky.tv/t/missing-features-errors-problems-with-liquidsky-2-0/22319

Worth mentioning are:

  • Payments not working for many payment methods
  • The client crashes before even starting
  • The LiquidSky server shuts itself down randomly
  • Monthly plans doesn’t have 500 GB storage space — only 100 GB is shown / usable

This also begs the question if their client was actually “ready to be released” as claimed in this thread or if the release was rushed because they had set themselves and unrealistic ETA.


Criticisms not related to the launch itself

Changed Skycredit rollover behaviour

Another change that was received negatively by myself and others was the changed rollover policy for Skycredits, there is a thread here that explains the issue: Instead of rolling over until a cap is hit they will only roll over for 90 days and then expire. This is arguably worse since you might not want to use the service for a while and only on-demand. Having your skycredits expire (i.e. your spent money vanishing) is definetly disappointing.

Changes in plans and pricing

Originally the higher subscription plans were supposed to come with 1 TB of storage, this has been reduced to 500 GB for all subscription plans. The “Pay as you go” plans have all been reduced to 100 GB instead of the 100/250/500GB tiering shown in Marketing material (e.g. during the “State of the Sky” stream on February 16th [1][2]). The amount of credits for the individual plans was also slightly adjusted and “PrioritySupport” has been removed from the “Pay as you go” plans.

One positive change is that the price of the “Elite” performance package was reduced from 240 credits/hour to 180 credits/hour. For some reason this change is not yet reflected on their website.

Misleading claims on their website

Let’s just go over what they say on their website, https://liquidsky.com/, right now:

  • “Play any game on ultra quality settings”/”Runs Any Game on Ultra Quality” — As discussed earlier, the lacking GPU performance on LiquidSky will mean that most games won’t run well at Ultra
  • “best graphics with ultra low and unnoticeable latency” — If you’ve ever played games locally, then it’s quite clearly noticeable. Also the video feed does look quite bad compared to uncompressed, raw video on a local system. That is the reality of fixed function video compression (they are using NVENC)
  • “ Through LiquidSky you experience up to 1GB/sec download and 100MB upload speeds” — The real speeds are 1 Gb/s download and 100 Mb/s upload, just a little error in units but I’m nitpicky since this has been on their site forever

On a positive note they removed the bogus claim that “Our proprietary compression algorithms deliver you the best quality graphics with the least amount of latency possible” that was present on the old website (Archive). As to why this was bogus: neither H.264 nor Opus nor the encoders used were created by LiquidSky. The only proprietary thing (afaik) is the protocol used for the handshake and transmission of mouse position/keypresses/etc.

Questionable claims in presentations

Also a little note on some claims made in their original LiquidSky 2.0 CES Press Deck (direct link to PDF):

The cost of electricity alone for a powerful gaming PC is $30+/month

$30/month with an average cost of about $0.12 per kWh in the US would translate to about 250 kWh/month. That equals about 340 W consumption 24 hours a day for an entire month. As should be obvious to anybody, this is a very unrealistic figure. If we actually assumed this kind of usage then a gaming PC that’s significantly faster than the “Elite” performance package and it’s power consumption would suddenly seem much more affordable compared to the ~$590/month you’d need to pay to get the same use out of LiquidSky.

4.4 Star Rating on Google Play with 10,000+ reviews

The reason I bring this up is this: https://blog.liquidsky.tv/2016/12/29/android-review-giveaway-november-winner-announced/. They held regular giveaways for Google Play reviews, this almost certainly lead to people making positive reviews in order to win which do not reflect the actual user experience. See the Google Play product page to get an impression yourself.

LiquidSky is already lined up to power some of the biggest Esports events with on-premise servers (1ms network latency), protecting the industry from cheating and unfair competitive advantages

No esports professional ever would accept the additional input lag and degradation in video quality/clarity caused by streaming the game instead of playing locally.

Privacy/Security concerns

Putting your data, including login information for several services, onto a third parties machines is always a risk. A data breach could expose stored passwords from browsers and other application. This should be considered and saving password should be avoided and two factor authentication used whereever possible

Additionally it seems that video data and keypresses are transferred without any type of encryption, enabling a potential man in the middle to log your keypresses and possibly even record your desktop video/audio if not entirely taking over the machine. More research is needed on that matter.

Update: I did some more research and it turned out that any communication between the “SkyComputer” and the client is indeed unencrypted. As a demo I wrote a little proof-of-concept keylogger that works by sniffing your (or anyone elses) network traffic for LiquidSky’s protobuf messages, you can find it on GitHub: https://github.com/derrod/liquidsky_keylogger

(Possible) Open-Source license violations

The LiquidSky Client currently includes:

IANAL but at the moment the client does not seem to comply with any of the aforementioned licenses.

The server software also uses live555/libcurl/openssl/ffmpeg libraries, though I’m not sure if it counts as distribution since it’s not offered for download but included within the VM made available to the user.

Update: added protobuf and gflags license violations.


Conclusion

LiquidSky overpromised and underdelivered. They are not the first cloud gaming service to do so and most likely not the last. At this point it seems like they overestimated their own capabilities, have lacking QA and tried to do too much with limited resources and unrealistic ETAs. And to top it all off they threw out bits of misinformation (e.g. GPU power), deliberate or not it didn’t help.

Communication is important, far too often companies that provide vague information and don’t clarify will receive backlash when the consumer gets to see the final product. The launch of No Man’s Sky being a recent example.

However, instead of communcating what is happening, what will and what won’t be available, and what changes have been made, LiquidSky kept being vaguet. And they are now receiving the backlash from their users for their failure to meet the expectations and hype they seeded.

All that said I’ll revisit them in a few weeks to see if they can deliver upon their (realistic) promises and possibly update this post, but at this point I’m skeptical.

P.S.: If I got anything wrong or I’m missing something important, please let me know!