Scala In 5 Years – My Prediction

It is no surprise that Scala is a hot topic these days from the business perspective and we’ve got two different investors asking me pretty much the same question: where Scala will be in 5 years?

Why 5 years? Well, that’s the typical exit horizon for the VCs and they all want to know our own take on what will Scala be then.

We’ve been using Scala at GridGain for almost two years by now and have been very active evangelist of it. In fact, our distributed computing DSL is second to none by a huge margin in Scala world (there’s simply nothing like that a all) and our commitment to Scala is larger than just technical curiosity or language enthusiast fetish – we’ve developed entire management and monitoring technology for GridGain using Scala as well.

We’ve been with Scala since when IDEA and Eclipse were unusable and time of daily compiler errors – but we’ve soldered on and gained invaluable experience and perspective on large enterprise project development using Scala.

50/20/20/10

In short – this is my undoubtedly highly approximated take on how JVM-based language eco-system will look in some 5 years (for enterprise development):

  • 50% of enterprise systems will be done in Java 7/8
  • 20% of enterprise systems will use Groovy
  • 20% of enterprise systems will use Scala
  • 10% of enterprise systems will use either Clojure, JRuby, Ceylon, or whatever else.

NOTE: I only assume JVM-based languages and only enterprise applications.

Let me explain my idea through several critical observations (everything IMHO, of course).

  • JVM is here to stay. There’s frankly very little that is wrong with JVM and replacing it will take a tectonic shift in computing. If you reading this blog – you will probably retire developing application that run on JVM
  • Despite my genuine love and deep appreciation of almost algebra-like simplicity and deep elegance of Scala as a language – I don’t see it having enough difference to displace Java or even pull even. 


    This is a hard statement to make… Scala is so much better than Java as a language that many of us in Scala circle tend to outright proclaim that it will take over Java in no time. Indeed, how can it fail when it’s so obviously superior in every regard?

    True – but it is only a language. There is no paradigm shift like it was with Java when I started with it in 1994. Combination of JVM and Java over predominant C++ back then was a tectonic shift – and Java never looked back.


    Scala for many is just a better, much better, Java, something that Java should have been matured to number of years ago but never did – preferring pragmatism of stability and compatibility over language changes. And this pragmatism only solidified its grip on the enterprise market over time.

    And so Scala is filling that gap for some of us who can blissfully ignore the language compatibility today. But tectonic change – it is not…

  • Despite being almost 10 years old project Scala hasn’t produce anything remarkable from framework perspective that would make Java or Groovy developers want to switch. Lift, akka, sbt, scalaz, ScalaTest – are all popular and great niche projects but generally result in yawning from Java audience. Done that, been there…

    

In Java we’ve had J2EE, Spring, JBoss, Tomcat, ESBs, AWT/Swing – and regardless of their eventual successes or failures they were pieces of technology that drove massive *and* wide-range Java adoption.



    I’ve argued recently that Scala doesn’t need killer app to succeed and I believe so deeply. In the same time a break through framework showing the best of what Scala could offer would be invaluable. ScalaTest and Specs, for example, can do that – but they a niche and rather specialized tools to gain massive following. So is GridGain’s DSL.

  • Scalaz – is a beautiful curse of Scala. It is its la femme fatale, if you will 🙂 You show it to someone who deeply appreciates language design, FP and have knowledge of Haskell – they will marvel it (I do). Show it to typical enterprise developer/architect – they will make sure to never touch Scala again.

    

The obvious problem with this is that the later outnumber the former like 10000 to 1.


  • There is little that attracts Groovy (2nd most popular language on JVM after Java itself) crowd to Scala… yet Groovy market is probably 10x of Scala at least today. It is really hard for Scala to gain more traction without “stealing” at least some of mindset of Groovy crowd – and I just don’t see it.



    With Grails, Groovy++ and VMWare commitment to Grails/Groovy overall I simply don’t see too many people migrating to Scala.


  • The entire pitch of Scala as a language for “new age” distributed programming is rather flawed and starts to rot in Scala community. Having earned authority on this subject I can clearly say that Scala has nothing special comparatively to Java to make it easier or simpler for distributed programming (this is NOT a negative statement but rather testament to Java original design):
    • Scala shares the same class loading and resulting distributed class semantics
    • Immutability is (can be) present in both languages (Scala is obviously supporting it more cleanly)
    • Local concurrency and concurrency unification is largely the same – or equally facing the same fundamental problems



    In the end – understanding of CAS-based concurrency or knowing all your primitives in java.util.concurrent just as equally important in Scala as in Java – for any serious programmer.

DSL – The Reason for Those 20%

I think there is one overriding reasons (a localized tectonic change, if you will) that will drive (some if not most of) adoption and growth of Scala short term – its ability to produce extremely elegant, powerfully expressive, and statically type safe DSLs.

DSL will play a role of a new kind of programming abstraction that will further simplify development of applications for complex domains.

Just look at the latest crop of Scala-based DSLs to see the future:

  • Specs2
  • GridGain’s Scalar
  • ScalaTest
  • Casbah

just to name a few…

All of them offer powerful abstractions over a complex underlying domain problem. All of them dramatically simplify the usage and application of that underlying model. The DSLs with these qualities (expressiveness, safety of static typing, and uncompromised performance and usability) are simply impossible in other language that I know of.

And that’s worth 20% in my book going forward.

Scala Documentation Rant…

There was a recent discussion on Scala group list about using students and volunteers (yes, coming from $3M funded company) to work on some less sexy stuff like… missing documentation for the core Scala libraries. You can see full discussion here: http://groups.google.com/group/scala-language/browse_thread/thread/f9144b6771a83dd6

Let me show what I have to deal with almost on the daily basis when working with Scala. This is rather trivial example – but very telling as towards dismal quality of Scala documentation (or practically non-existence of such in many cases).

I was looking at the code like that:

val num = str.toInt

There was an obviously missing error handling (what if str is not well formatted integer).

So I looked at StringLike‘s definition of toInt method. It was empty. Nothing.

OK… I opened the source code for StringLike and saw that toInt simply delegates to java.lang.Integer.parseInt(...) method (without a shred of Scaladoc on it):

def toInt: Int = java.lang.Integer.parseInt(toString)

That was enough but I had to double check my memory and I opened Javadoc for java.lang.Integer.parseInt(...) to see that it indeed throws NumberFormatException when string cannot be converted to integer.

So, instead of simple glance to Scaladoc, I had to look at Scala’s source code, then to Javadoc to see that method toInt indeed throws exception (which in Scala a runtime exception and compiler doesn’t even warn me about it).

Nice.

Scala Actors and Scala Adoption in Enterprise

Replied to the post on one of the Scala user groups:

In the long run I believe actors are irrelevant as it is just a specific technique for expressing certain types of algos. It’s simply too trivial to really matter when companies contemplate the move to a new general purpose programming language and its eco-system. Other (much more important) properties of that Scala eco-system will drive its adoption.

It’s like saying that someone will pick Java over C# because of java.util.concurrent.

Further more, the distributed world is infinitely more complex than a share-nothing message passing paradigm. If that would be the case – we’d all be using RMI and MDBs today. But we are not…

When it comes to STMs, Supervisors, Flows, Transactors… I think even Akka folks would admit that these are rather esoteric things. Not every one writes telco switches in FP-only language that doesn’t support mutability. Copying 25 year old technology from Erlang is not a guaranteed bullet-proof solution. Technology has really, really moved on since then. Concurrency, parallelism and determinism have different meaning and properties in a distributed world of today.

I’m not bashing Akka/Actors here – love them both. It’s just the reality of the world in my opinion.

Kind of sums up my feeling towards Scala adoption and actors.

Code from my presentation at Scala Days 2011, Stanford June 2nd.

I was asked by several folks to post the code from my presentation – so here it goes (without any changes):

import java.util.concurrent.CountDownLatch
import java.util._
import org.gridgain.scalar._
import scalar._
import org.gridgain.grid.GridClosureCallMode._
import org.gridgain.grid._

object ScalaDays {
    def main(args: Array[String]): Unit = scalar {
        pingPong()
    }

    def pingPong()  {
        val g = grid$

        if (g.remoteNodes().size < 2)
            sys.error("I need more than 2 nodes")
        else {
            val n1 = g.remoteNodes$().head
            val n2 = g.remoteNodes$().tail.head

            n1.remoteListenAsync(n2, new GridListenActor[String] {
                def receive(nid: UUID, msg: String) {
                    println(msg)

                    val latch: CountDownLatch = g.nodeLocal.get("latch")

                    latch.getCount match {
                        case 1 => stop("STOP")
                        case _ => respond("PING")
                    }

                    latch.countDown
                }
            }).get

            n2.remoteListenAsync(n1, new GridListenActor[String] {
                def receive(nid: UUID, msg: String) {
                    println(msg)

                    msg match {
                        case "PING" => respond("PONG")
                        case "STOP" => stop
                    }
                }
            }).get

            n1 *< (UNICAST, () => {
                val latch = new CountDownLatch(10)

                g.nodeLocal[String, CountDownLatch].put("latch", latch)

                n2 !< "PING"

                latch.await()
            })
        }
    }

    def sayHi() = println("I'm in this grid!")

    def mapReduce(msg: String): Int =
        grid$.remoteProjection() @< (
            SPREAD,
            for (w <- msg.split(" ")) yield () => {
                println("Calculating for: "  + w)
                w.length
            },
            (s: Seq[Int]) => {
                println("Aggregating: " + s)
                s.sum
            }
        )

    def helloWorld(msg: String) =
        grid$.remoteProjection() *<
            (SPREAD, for (w <- msg.split(" ")) yield () => println(w.toUpperCase))
}

Enjoy!