Rails scales pretty well nowadays, yeah, but that does not mean that it magically becomes the best tool for every given application.
It’s slightly misleading to say that “Twitter switched to Java”. More specifically, they switched to Scala, which builds on Java. Scala is extraordinarily suited for handling real-time messaging on a large scale, in the way that Twitter does, and that is the reason for the switch.
So — was using Rails a mistake?
That Twitter switches technologies at this point is often mis-interpreted by outside observers. They think that this means that Twitter made a mistake by choosing Rails to develop Twitter in the first place. This is, of course, not the case — it simply means that Twitter is now at a different place, and a different technology is required.
Just like your college girlfriend might have been awesomely cool when you dated those years ago, you grew apart because your needs and priorities changed. That does NOT mean that the relationship was a “mistake”, it just meant that things changed.
When Twitter was created a few years ago, their needs were VERY different from now. They were a 2-man operation, and created their first version in a few days. They needed a platform that had a huge output per invested hour of development, and that allowed them to change the product very rapidly to the needs of users, because frankly, they had no idea what the fuck they were creating — it was something cool that people wanted, but where was it going? They had to roll with the punches at that time.
Fast forward a few years, and their needs are different. Now, the product is pretty stable, and not changing (a lot) compared to how much it changed in the beginning. Use of Twitter is enormous, and one of the biggest concerns is just handling the huge pressure that the service is under. Real-time updating of searching is now a big feature, which was probably not even thought of by anyone in the beginning.
While Scala/Java is a great fit for Twitter NOW, it would have been a spectacularly bad fit in the beginning. Partly because Scala was very, very new and immature, but also because Java in general is a bit of a battleship compared to Ruby, and would simply not give the flexibility and movement speed that Ruby on Rails provided.