I wanted to put our two cents when Yammers’ personal email was leaked week or so ago but kind of waited to let the dust settle. We are active Scala developers with several years and production code behind us – and so we have this rare “real live” experience with Scala…
In two years we went from having Scala as a very strategic choice to have it simply as an interesting (and mostly enjoyable) technology that we use to develop some (and rather small) parts of our platform. I still think, as I always did, that Scala is wonderfully elegant and relatively simple language. For the first time someone has combined OOP and FP in a extremely practical general purpose programming language.
But our team’s enthusiasm significantly decreased over the last 12 months as it became obviously clear that Scala won’t be the eventual Java replacement for most of the enterprise Java development. Moreover, more and more people are starting to painfully realize that Scala won’t even make any significant dent on Java eco-system at all – remaining forever in 1-2% territory for very specialized projects where FP focus or DSL capabilities override any other requirements. With Java 8 coming up quickly, with Kotlin, Ceylon and Clojure nibbling on the edges (even if some of them don’t yet exist!), Groovy going strong – Scala simply doesn’t have much room to grow or maneuver.
Not being widely used means little interest for us, independent ISVs, to provide support and integration for it.
2011 was yet another clear indication that Scala eco-system is experience growth pains:
- Typesafe perceived troubles and lack of direction
- Ridicule about parallel collections in 2.9 (one of its major new features)
- Public “I’m fed up” email leaks
- SBT vs. Maven, Lift vs. Play, Scalaz vs. common sense…
- Scrutiny of few larger projects exposing rather alarming ineptitude in: scala.io, scala.xml, scala.actor, parallel collections, scaladoc implementation
- Glaring performance holes in the compiler known for years yet still ignored
- Compilation speed, binary backward compatibility woes continued (year after year)
- Utter lack of attention to API documentation – “read the source code instead” mantra
- IDEs integration
Some of this can be, of course, explained by the fact that most of the Scala eco-system was originally developed by University postdocs and students that lack non-academia experience in building commercial software (of that size and magnitude). Be it as it may – most of us would still expect the better outcome… On the other hand Typesafe seems to be pushing the language development and maintenance onto the community again and that scares me (read above) – preferring rather to concentrate on the Stack.
And then – there is a community… Oh – how I wish sometimes that Scalaz would never exist (along with some of the idiot savants behind it)! This Scala-wannabe-Haskel library is probably single-handedly responsible for a big share of “Scala is too complex” syndrome that sealed the fate of Scala at least for now. Having almost zero engineering applicability in the real world – it somehow inexplicably defined the public’s perception on Scala’s usability. While being interesting academically (and written by very smart guys indeed) – I wished it would never exist. Really.
Having said all that – I still love Scala and that’s what I use almost exclusively when I write code for GridGain (still do!). Heck – we even use one trait from Scalaz! Despite the numerous growth problems – I still can’t resist Scala elegance and almost an algebraic simplicity and pragmatism that is so rare these days. It’s the combination of simplicity of the language and the power it gives to developer that attracts me to Scala.
My wish list for Scala in 2012 thus is pretty simple: stop the bleeding, no new boneheaded features, and plenty of bug fixes and core hardening.
GridGain - In-Memory Computing 
Seriously, what’s the point of bashing Scalaz? If you don’t understand it, simply don’t use it.
I’m not sure there is even a few dozen people worldwide who fully understand Scalaz through and through… The point is that it garnered so much attention and publicity that it has define the Scala usage somehow – which is totally false. Scalaz has almost zero real-live applicability in actual coding.
I actually like its cleverness and pushing envelop on type-based programming. But there’s clear misconception about its applicability in the public eye.
> Scalaz has almost zero real-live applicability in actual coding
This is not my experience. Scalaz can really help in coding actual problems, like this question on StackOverflow: http://bit.ly/r5mQPA. It provides a very high-reuse value, where we don’t have to rewrite, again and again, code to do traversals or aggregations (and those are frequent use cases, right?).
I grant you one thing. It takes some time to learn those type-classes and to see how to use them to the best effect. But once you do that, the payoff in your daily coding is high because your toolbox expands a lot.
Eventually it becomes a trade-off programmers need to be aware of: either they don’t learn all that stuff or they have to accept some code duplication for lack of abstraction. At a lower-level this is the same trade-off involved in choosing the “complexity” of folds and maps with collections over using simple for loops.
Anyway, I really appreciate those tools and I’ll do my best to de-mystify them so that they can become part of others’ people toolbox. And this goes beyond Scalaz: parser combinators, functional reactive programming, continuations,… can all be immensely helpful in our daily jobs, even if they seem hard at first.
Eric,
You are obviously one of those few dozen around the globe who not only understand Scalaz fully but likely can use it effectively in a commercial setting. We’ve spent significant amount of time try to adopt Scalaz (initially even just for sake of adopting it) but came away with a simple conclusion: in most cases few extra lines of code and a bit less reusability provided for dramatically easier to understand code.
Some of us can be individually motivated to digest the Scalaz and get proficient in it – but for the team of non-trivial size it’s becoming a unnecessary and unwise hurdle.
Everything IMHO, of course.
Nikita.
If I get you correctly you say that you don’t fully understand Scalaz, but still you can say that is has “almost zero real-live applicability in actual coding” (whatever that means). Granted, many of the things in Scalaz already informally exists in the Scala library, monads, functors etc., but it’s useful to properly encode them into abstract typeclasses. And there is definitely some other useful stuff in there, for example Validation and Applicative. Yes, even in “actual coding”.
Hi Nikita –
My $0.02 on your statement “I’m not sure there is even a few dozen people worldwide who fully understand Scalaz through and through”. I am not sure if that’s the correct approach to use a deep framework like Scalaz. It’s indeed difficult to master the whole library upfront – I doubt if anybody has attempted like that. You can start learning *piecemeal*. Believe me there are tonnes of small abstractions in Scalaz that can work wonders in reducing boilerplates in your Scala code. I started using Scalaz like that only. Abstractions like Monoids, Equals, Validation etc. give you the composability and help you think in terms of higher order functions. As an example you can have a look at Chris Marshall’s Practical Scalaz talk (http://skillsmatter.com/podcast/scala/practical-scalaz-2518) where he describes how he started with smaller abstractions in his production Scala code.
Thanks.
Chris’s talk opened my eyes to the practical applicability of scalaz and is well worth watching – as debasish says, it’s little conveniences that I use day to day rather than huge chunks of it.
Jesper,
What I was trying to say is that we tried (hard) to use Scalaz constructs (Validation specifically, btw) and came away with a strong feeling: if you move away from purposely minimalistic code examples in blogs and onto real live code base – things get really hairy and unreadable rather quickly.
I guess it’s Scala mostly as similar Haskel code looks a lot cleaner – but be it as it may.
I guess I’m another who has found a way to use Scalaz productively; what I’ve found is that more than anything, using scalaz (and Validation is a great example) has forced me to become much more aware of all the ways in which things can go wrong in my code – but has also provided me with excellent tools that allow me to mitigate those corner cases in the best way possible. In that respect, I see Validations as very much what checked exceptions should have been; learning to use the rest of Scalaz has been an ongoing process, and for each new thing I’ve learned, I’ve had that same “well, gee, yes this *is* how that should be done” realization eventually.
For me, I became productive with scalaz first by using it for examples like Eric gave above, then using Validation, then by starting to realize that I needed Applicative instances for some of my own classes, just to enable me to use List.sequence (via implicits). Each new step I’ve taken with Scalaz has been in response to trying to solve some business problem, thinking “there should be a standard way to do this” and then finding the solution in the Scalaz type signatures.
As far as the ability to understand the resulting code goes, this is just something that requires effective communication in a team. Once you’ve learned what |+| or |@| means, the resulting code is *easier* to understand. But it does take learning and communication to get there.
I respect your opinion a great deal as you have been working with scala professionally for much longer than I. I have been wrong about quite a few things over my career. Coming to the JVM stack only recently after working with other languages for the last 6 years or so, I feel that scala fills a huge void left by Sun and Oracle. Java is extremely out of date and as long as jobs are available, like they are now, I would probably choose not to use it.
In regards to scalaz, I had quite the opposite experience (Validation specifically, btw). We were able to dramatically simplify some really nasty validation code with it. We ended up with 7-8 calls to functions returning Validation in a for comprehension. We have done some similar code simplification with the “|” function on the scalaz Option extensions as well.
With that said, almost the same code could have been written by replacing Validation with Either from the standard scala libraries.
What are the:
Ridicule about parallel collections in 2.9 (one of its major new features)
and performance holes in compiler?
Just on top of my head: http://beust.com/weblog/2011/08/15/scalas-parallel-collections/
Hi Nikita,
Thanks for your feedback!
Scala is not without its issues and just like any other programming language represents a tradeoff. I am sorry to hear that it did not meet your expectations or you do not see scala’s future as bright as used to be, however, I can assure you that some of the problems you listed are high on Typesafe’s priorities (please see what we are working on here http://blog.typesafe.com/getting-down-to-work ). As for actual progress, please see a few very recent changesets:
https://github.com/scala/scala/commit/c352105fa082f88163869c0ba86f3e32aff57fc7
https://github.com/scala/scala/commit/5fb26c6a889cf1609823338df8783bf880769b3f
https://github.com/scala/scala/commit/1b98d1fa2a54dc1d8fafed6d270534d729420c21
https://github.com/scala/scala/commit/989c0d0693e27d06d1f70524b66527d1ef12f5a2
https://lampsvn.epfl.ch/trac/scala/changeset/26042
https://github.com/scala/scala/pull/16
As for scalaz, personally I can not program in that style(mainly a lack of deep understanding at my part), but it did help me to understand a few concepts that I did not know before. I guess what I am trying to say is that even as a non-user I am glad it exists and I know there are folks who find it extremely useful.
Also, while you listed our community efforts (presumably you meant http://docs.scala-lang.org/ and https://github.com/scala/scala) as something to be concerned about, I find the opposite is true. No open source technology can really grow without its community (related to this: http://code.technically.us/post/13548980134/the-gist-of-it )
so if you and your company are still interested in making scala better, please help us any way you can (report bugs, fix docs or just participate in the community).
If it’s not the case and you’d come to the conclusion that scala is not working out for you, I wish you good luck with the technology choice you’d end up making.
Best,
Peter
Hi Nikita,
let me just first say that I seem to live in a very different world.
I can neither see Kotlin nor Ceylon anywhere on the map for the next decade — completely ignoring that they will experience the same growing pains which Scala has already behind it — if they ever get released.
Clojure doesn’t really compete with Scala, in fact Clojure could be written in Scala in the futuere, like it is written in Java today. Groovy? Had its chance years ago. Didn’t get much done (Documentation? Specification?). Additionally they will probably break thousands of lines of code in 2.0 when they switch back to (optional) statical typing. Apart from that I’m not interested in any controversy between Scala and Clojure. The enemy is status quo.
Criticism a year ago was about the lack of any commercial backing for Scala. Now there is Typesafe and they have been impressively responsive: Better IDE, binary-compatible builds, better documentation, move to Git and GitHub, … While I’m normally a loud critic on things I don’t like, I have to be honest here and have to give credit to the people working at Typesafe. They did an impressive job moving Scala forward. Maybe you can give some real-world example what you disliked?
I’ll skip the criticism about parallel collections which was more or less the result of a complete lack of understanding, responsible research and verification of facts before the author wrote that article.
Seems to be a new hobby of Java “experts”: Making claims about languages they have neither used nor understood.
In my opinion the public “leak” was actually a great thing: Substantial and constructive criticism! In response Typesafe gave a clear vision and time frame when they will fix the issues mentioned. Hale Coda’s article was a breath of fresh air compared to the recent unsubstantiated flame baits.
Not much to comment on SBT … it was totally stupid that the new version with a completely new API was released without sufficient documentation. Still shaking my head about that. Still wondering why SBT even has separate documentation instead of integrating it with docs.scala-lang.org.
On Play vs. Lift … with the amount of Lift’s language abuse I can totally understand where David Pollak comes from with his criticism … but I don’t think this criticism can be applied to non-Lift Scala code.
The lack of a more extensive standard library annoys me, too. But I think the first priority should be quality and a low foot print. It took Java 15 years to come up with a usable IO library, I think the criticism is a bit over the top here, considering we can easily use the new APIs from Scala. Would I like to see more standard libraries? Sure! But first I want to see scala.dbc gone, scala.xml made pluggable and the parser combinators shipped as a separate JAR file. The scala-library.jar went down from 8.4 (2.9.1) to 6.8 MB (trunk) and I think we can still reduce our footprint here.
About compiler performance, if we interpret every recent improvement as a proof that the technology sucks, because technology has to be perfect from day one … I guess there won’t be much technological progress in the future of mankind. I have been very happy with compiler speed in recent releases and the integration with IDEs and SBT has substantially improved.
Typesafe delivered with regards to binary compatibility. Actually, with their work on the migration manager they have done much more than Sun or Oracle ever did in the last 15 years.
Sorry about the lack of documentation though, it is a very important issue. I’m pretty sad that I and the all the other unpaid volunteers didn’t get as much work done in their free time as you expected.
IDE integration … great improvements here. IDEA rocked for years already and Eclipse is catching up nicely as well.
In the end I think Scala has a great future and huge possibilities — if we all could agree to spend a bit less time with talking and a bit more time with getting work done.
My wish-list doesn’t look that much different from yours:
– Keep delivering stable, well-tested, binary compatible releases like it was done in 2011.
– Bugfixes everywhere: The new pattern matcher recently merged in trunk will have a great impact here.
– Addition of the database stuff, enabling the Java ecosystem to compete against .NET/C#/F# again.
– Documentation, documentation, documentation.
@Simon, Very well said. It’s very relieving to read more words like these. It very much calms things down and I wish I had read it before my original comment to Nikita.
> Clojure doesn’t really compete with Scala, in fact Clojure could be written in Scala in the futuere, like it is written in Java today.
Im sorry thats just bullshit.
Clojure does compete with Scala. Yes at the moment it is a little harder to get to the same speed with clojure then with scala but that does not mean there not competitors.
The Clojure Collections where written in Java but the where written befor the clojure existed. Nobody rewrote the code because it that would yield much benefits. The Clojure Compiler is still the bootstrap compiler writen in java again it has not been rewriten in Clojure because that would’t yield much benefit. The Java Parts of Clojure would be written in Clojure not in Scala.
Even Beust was challenged in his ridicule on Scala parallel’s. As even Odersky has said, you don’t simply add .par and magically any nub can do parallelism. You still have to know what you’re doing.
Also you need to give more details as all you’ve done is pull the small handful of naysayers posts and combined it here. At least Coda had a lot of details to why it doesn’t fit for him and a lot of his existing applications. You are just showing how you simply don’t really code in Scala.
It’s funny cause if I can (and do) use scalaz in my day to day code that is indeed running in full production as REAL WORLD applications then why can’t you? Sorry for a whole (if I can do it so can you) but really once you’ve coded in Scala long enough and used it in a Functional Programing mindset instead of Imperative then things like scalaz and the built in parallelism on collections work out really (REALLY!) good.
Also Scala has already made quite an impact in the Java community and over all Java developers. I mean look at blog posts like this? Obviously people that are to lazy to learn outside their comfortable realm of everyday Java are gonna instead waste the tiny little energy they have to do write blog posts making claims they obviously don’t know what they are talking about. Not only the cry for attention blogs, but look at actual popular names in Java that are also writing a lot of Scala and some cases completely moved to Scala.
I don’t even know why you’re trying to hop on this train. All it does is slow down people’s productivity.
meh back to coding Scala for me good luck with things.
Your tone now is so much different from your view in June: https://gridgaintech.wordpress.com/2011/06/28/scala-in-5-years-my-prediction/ in which you declare that 20% of the enterprises will go with Scala because:
“DSL – The Reason for Those 20%…And that’s worth 20% in my book going forward”
People can change their opinions over time, given new facts and time to learn new things. Surely you are smarter today than yesterday. If not, why not?
being smarter and being wiser are very much different things, Michael
Nikita,
We all have different opinions regarding the purpose of some tool although we have (and I’m not saying you haven’t) to accept that in a community variety will likely to happen. You may see disadvantages (heck I see a few ones too) but I think criticizing its existence/usage does not help! I’m glad for this kind of variety …. gives users a lot of choices. I would not put things this way “SBT vs. Maven, Lift vs. Play, Scalaz vs. common sense…”
At the end of your post I think you touched the sweet spot! (and I think this is what you were trying to cover at the beginning of your post) Yammer’s problem might be other’s too. They talked a lot of performance issues that I think were due to the not so good scala documentation (There is a good effort to tackle this http://docs.scala-lang.org/) so let’s improve on how we can let other scala users know what might not be a good idea to use and let the scala core team know about it.
To wrap up, I think we should focus on core problems and let the guys behind it know it. http://docs.scala-lang.org/sips/ this is waiting for more stuff!
I hope you can focus your strength/force/anger? 😛 on the last paragraph.
Thank you for the feedback. It is indeed important for the general community to be aware of the challenges that a sophisticated language/stack such as Scala would present to teams in IT.
A lot of the upset feedback regarding your post is issued from Scala enthusiasts with a tin ear: It is fully understood that there will always be a group of developers that can do X in (ecosystem) Y. And it is further fully understood that Scala’s innovations are not without practical benefits. (That’s never been a question). Yes, Scala is a tour de force in language design. (clap clap).
The question is this: Is Scala the replacement for Java in IT as a general purpose commercial language. How this would have ever been answered “yes” in the theoretical has always bugged me. Let’s not shoot the (commercial but certainly not “IT” type) messengers who are now tellings us, based on empirical results, that the answer is “No, not_yet/probably not, because …”.
Java hits the sweet spot of development at scale. That’s one of the reasons, in fact, for the propensity of Java projects to balloon: because it is very easy, even for fools, to manage complexity and barriers to entry for new team members is quite reasonable. One may think that amusing, but it is a huge positive. Because in the hands of experts, it is bullet proof, yet /even relative fools/ can be asked to contribute to a Java project. That really is a huge + in the IT world. Simply consider the distribution of programming ability across the board. Scala is designed for and appeals to a select sub-set of that distribution, and quite obviously (doh!) presents difficulties for the general community. The original Scala answer to this issue was “Java interop”, but based on feedback to date, apparently that is not a solution.
A sophisticated language that can do wonderful miracles at the hands of the experts is really not that interesting — think about it — & the challenge (of Java) remains: design a language that is good enough, is comprehensible across the range of skills (in your expected teams), has excellent tooling, has very good performance, has a proven record of utility, and exposes enough hooks (e.g. bytecode transformation) to allow experts to create ‘magic’ (e.g. AOP) if magic is required, without over burdening (and thus negatively impacting the productivity of) those who are still trying to comprehend OO and grok DI (!).
That is the context of this (internet) conversation about Scala: can it replace Java? It is an /economic/ issue with a technical component, and not the other way around. And Scala+community does not get that.
Because Java got it all right the first time…
Seriously, though, I think it’s a bit premature to predict the demise of Scala. The amount of buzz and public discussion around it these days is a sign in itself that a lot of people are paying attention, and that Scala is maturing as a language and platform as people identify its weaknesses. For how many years did we suffer through the pronouncements of Java’s downfall as it was too slow for production use or didn’t have the capabilities of native code? And then came EJB.
I think Scala will be just fine and continue to be adopted by Java developers seeking more powerful means of expression. I, for one, am still excited about Scala.
I think this phrase says a lot: “We’ve spent significant amount of time try to adopt Scalaz (initially even just for sake of adopting it)”
Why would you adopt a library you don’t understand just for the sake of adopting it? Moreover, why would you then blame the library? That’s just bad technical decision making and is your fault.
The post sounds to me like Yammer heard “Scala is the next Ruby” and then rushed out to use every shiny and buzzworthy bit of it they could as opposed to actually assessing their needs and picking the appropriate toolset.
That should read, “GridGain heard”, not Yammer.
Another reason why Scala might not work in the developer world is that people are sick and tired of language wars… People are not really interested in the language itself, they are more interested in the ecosystem and the capable frameworks around the language which sort of explains why Groovy and Ruby are popular today. Now don’t get me wrong, I think Scala is a great language but without an ecosystem that simplifies getting things accomplished for people, Scala will not gain the popularity or the “next great JVM language” status that it has the potential to.
I can definitely understand the authors frustrations with Scalaz. I in fact like Scalaz. I feel like I’ve never had more fun learning Haskell idioms! I really enjoy reading posts explaining concepts in Scalaz like validation, trampolines, or iteratees. However, I’ve also seen a fair number of small code snippets that, when executed, result in stack overflows, enough to make me scared of ever trying to use it in production. When reading about trampolines, as an example, I saw a post from someone saying they were excited to see this feature because they were getting stack overflows using iteratees!
Also, I don’t think we realize sometimes just how far removed some of us are from the majority of programmers in terms of language expertise. It’s taken me over a year of studying in the Scheme world, for instance, to start feeling comfortable with the language, and I still have a hard time with some of its more advanced concepts like delimited continuations and advanced macros. Similarly, I’ve required almost as much time to finally feel like I’ve absorbed some of the basic concepts of category theory in Haskell like monads and applicative functors. The majority of programmers I work with, on the other hand, are neither 10 times smarter than me so that they can learn these ideas instantly, nor are they willing to spend months, perhaps years of effort, to better their skills and understand some of the PL concepts others of us take for granted.
As an example, not too long ago I had to fight to justify to a co-worker my use of closures and lambda expressions in my C# code. He had never seen closures before and the concept frankly scared him! I can even sympathize with him, because when I first learned about closures a long time ago it took a week or so before their meaning finally sank in to my brain.
If I have to fight use a language feature as basic as closures, how much more do you think I would have to fight to use the Maybe type to replace nulls, or the Either type to encode the possibility of errors? How kindly do you think this person would take to ad-hoc polymorphism via implicits? What about monad transformers? (I’m still trying to figure out the practical use of that one in Scala!)
While I like Scalaz, I also worry that it causes too much bad PR in the short term. There is already more than enough complexity FUD spread about Scala, and Scalaz just makes it that much easier for Scala’s detractors to scare people away. It’s great that people are pushing the boundaries of what Scala can do through its type system, but we shouldn’t forget that there’s great value in improving Scala’s ability to be just a better Java, and that includes beginners being able to jump in immediately and feel both comfortable and productive.
Pingback: Scala Report: Summer 2012 « GridGain – Real Time Big Data