Is Scala a Business Ready Language?

I’m a great (but not fantastic) developer and certainly no Scala expert so I approach this topic fearfully, knowing I’ll get something wrong.

And although I have decades of professional programming experience, I have zero Scala experience so I will be relying upon the experience of others. You will find a lot of quotes (with attribution) scattered throughout.

I’m both an opinionated and open-minded person so please express your own opinion below the article – you have a good chance of changing my mind.

A business is a team that takes ideas or products and brings them life.

Business Needs are Priority 1

When evaluating a technical choice in an organization, needs naturally fall into one of two categories: Business and Technical.

Even startup founders organize time and hat wearing between “businessy stuff” and technical execution.

The business needs of an organization must come before the technical needs for a simple reason. Teamwork.

A business is not just a bunch of developers and engineers sitting around building stuff. It’s tempting to draw a line of separation between tech and “everything else.” This is not teamwork – it’s silo building and any organization finding itself engaged in the business of building silos is doomed.

A business is more than a product, it’s more than a brilliant technology, it’s more than a team of 10x rockstars. A business is a team that takes ideas or products and brings them life through strategic partnerships, vision, mission, high-level planning, branding, marketing, sales, customer service, human resources and IT.

Business Needs

HR – Finding and Retaining qualified people.

From 2011:

“…it’s effectively impossible to hire people with prior Scala experience (of the hundreds of people we’ve interviewed perhaps three had Scala experience, of those three we hired one)” source

On May 1st, 2015 a quick trip to Stack Overflow Careers Candidate Search yields this startling metric. 56,000 java developers ( no interns or students) to 5,000 scala developers.

searching indeed.com for scala returns 3,733 results.

searching indeed.com for java returns 86,189 results.

Scala and Java are far apart on github.

Conclusion: Scala is the loser. Java is the clear winner with by far the most developers.
I could dig wide and far and come up with countless examples of the disparity between the availability and popularity of Java and Scala. Is this important? Only you and your organization can answer that.

Maturity and Stability of Scala

At 12 years old (internal vs general release), Scala is, without a doubt a mature language but does it’s maturity come with a hidden cost?

This release contains an important fix for serialization, which was broken in Scala 2.11.0 (SI-8549). The fix necessarily breaks serialization compatibility between 2.11.0 and 2.11.1 (this is separate from binary compatibility, which is maintained).Scala 2.11.1 Announcement

Breaking backward compatibility (yes I know it’s not binary compatibility) between minor versions is a negative.

Scala keeps breaking binary compatibility with every new release. In spite of previous promises, compatibility was broken in release 2.7, broken again in release 2.8 and broken yet again in 2.9. Ceki Gülcü – Is Scala worthy of your trust?

Yes, these are old versions and it appears that with the last few releases they have not broken backward compatibility. However, in September 2014, when asked in an interview with InfoWorld about an upcoming release (Don Giovanni – to be released in April, 2017 approximately) , Martin Odersky said: “That’s going to be a more fundamental rethink of what Scala is.”

To be intellectually honest, this is a good thing… technically. I admire the mindset of continuous improvement. This does not lead to stability however.

Conclusion: Should a language be considered mature and stable when these sorts of sweeping changes are planned or necessary?

Bringing Scala into Existing Businesses

There are many testimonials that can reasonably lead one to the conclusion that it is very viable to bring Scala into an existing JVM oriented business.

On Redfin, Alex Payne writes this about twitter

Scala is a lot of fun to work in; yes, you can write staid, Java-like code when you start. Later, you can write Scala code that almost looks like Haskell. It can be very idiomatic, very functional — there’s a lot of flexibility there.

However Julio Faerman writes

Although it is possible to invoke Java methods in Scala code and vice versa, the interaction between languages is not without complications.

Yakov Fain has an opinion about this:

If you have a team of several Java developers and want to introduce Scala, Groovy or any other exotic language, do not hire a person who knows this language. Hire an instructor to teach the entire team how to program in this languages and let one of the developers use it in your project.

Conclusion: Neutral.

Cost of Development

All positions are “senior” The location choice is an affluent community in the midwest which is cheaper than the coasts – adjust accordingly.

Position Location Cost
Scala Overland Park, KS $105,000
Ruby Overland Park, KS $105,000
Java Overland Park, KS $99,000
Python Overland Park, KS $105,000
Haskell Overland Park, KS $95,000
PHP Overland Park, KS $95,000

Image

Source

Conclusion: Irrelavent.


Technical Needs

Poor technical decisions have the potential to ruin a product or company completely. Consequently, appropriate care should be given to research and informed decision making.

I’ll be using mostly quotes for this section. Although I have read numerous articles and opinions and documentation pages about various libraries, I have no real-world experience.

Popularity (Google Trends)

Google Trends for Scala, Java, Ruby, PHP and C++

Google Trends for Scala, Java, Ruby, PHP and C++

 

Community

For a language to be worthy of a significant commitment by developers and businesses it must have a quality eco-system surrounding it. Topping the list is the community.

Language Meetup.com ( meetings/members) 2014 Conferences (lanyrd.com/)
Scala 398 / 109,873 32
Ruby 781 / 324,640 131
Java 1123 / 406,084 115
Python 1052 / 414,303 136
Haskell 182 / 44,821 8
PHP 1311 / 403,544 99

Conclusion: Scala clearly has a lot of work to do here. Only one of these three metrics is looking at numbers of people yet all of them have Scala at or near the bottom of the list.

Good Ecosystem

Node.js has an amazing library eco-system. One that I have yet to see duplicated elsewhere. If anyone involved with the Scala ecosystem is listening, you would do well to emulate npmjs.com

Nobody else so far has mentioned this, but the documentation for just about anything in Scala seems very slanted toward the academic rather than the practical. Most documentation reads like a professor is trying to justify a budget item; it’s all about “why Scala” and not much about “how to do things in Scala.” Reese Currie

Scala has methods to get similar results as npmjs.com but it is complicated and is part of the built tool – a product that some advise against using.

Comparing Open Source Projects By Language

Using GitHub as a data source reveals that the top two languages, by both project count and user count are Java and Javascript with the latter having more users than Java even with fewer projects.

The biggest surprise on this first dataset is the stunning disparity between the other languages and Scala. Scala is barely a blip on the radar.

The next GitHub dataset compares issues reported by language both as a number and as a rate per project. Without looking into the substance of the issues, there are two interesting conclusions that can be drawn. The first is that Java has the lowest defect rate (2.153 issues per project) and Scala has the highest (4.604 issues per project).

Conclusion: Between Java and Scala, Java is the clear winner.

Code Readability

Read the comments on this StackOverflow answer. Where is the community consensus?

Readability, readability, readability. Scala supports many programming paradigms(and has features to enable those), they combine into hundreds of idioms making it possible to write code in myriad of ways. Given the possibility, it’s hard to stop developers from going crazy and trying clever tricks, resulting in very unreadable code in the end.

It’s definitely possible to write beautiful code in Scala and then it’s the most beautiful language in the world but practically most people won’t do that. Scala libraries are written by very smart people but somehow even the best get carried away and go blind to code readability. If the best people write difficult to read code there is no way to control others. – Pankaj Gupta

Scala is a lot of fun to work in; yes, you can write staid, Java-like code when you start. Later, you can write Scala code that almost looks like Haskell. It can be very idiomatic, very functional — there’s a lot of flexibility there. –Alex Payne

“Scala is so flexible that you can write code in it that’s utterly impenetrable. You only have to look at Perl scripts to see the downside to that. Programmers who have to work together are happier if there’s only one or two clear ways to do something, not lots of different ways.” John Purcell

“My style changed gradually over time from object oriented to now completely functional” –Quora Review

Conclusion: Caution. Scala allows for code that is “utterly impenetrable.” Scala enables both functional and object oriented code. Without discipline and enforced standards a code base will become a mess – each developer contributing in their own evolving style.

Does Scala Lower the Defect Rate?

In spite of the conclusion drawn by the GitHub analysis that Scala projects have more issues per project, what is the conventional wisdom when it comes to answering this question? Perhaps in an enterprise environment things are different.

But most importantly, Scala taught me to program and reason about programming differently. I stopped thinking in terms of allocating buffers, structs, and objects, and of changing those pieces of memory. Instead, I learned to think about most of my programs as transforming input to output. This change in thinking has lead to lower defect rates, more modular code, and more testable code – David Pollak

This is a deep read on the subject of code quality as relates to programming language.

functional languages are generally less defect prone than procedural languages
A Large Scale Study of Programming Languages and Code Quality in Github

Conclusion: Yes. Probably. Temper this conclusion with the issue data from GitHub.

Performance & Scalability

Scala is named with the word scale implied in the name. While scala works well for large-scale projects, the intent of the name is more about your experience with it – it scales to meet you at your skill level.

“Scala is deep where other languages are broad.”
Martin Odersky – Slide 11

This means, scalable is meant in terms of flexibility and expressiveness. You can create your own control structures. E.g. the actors frameworks is a library, but it looks like it uses language features.

This means that Scala will scale according to the needed abstraction, not necessarily in terms of “answering 1.000.000 requests in a second” – Michael Kebe Programmers.StackExchange

Aleksey Shipilёv has an interesting article about benchmarking Scala vs. Java.

I should add that per amount of time spent programming, my Scala code is usually faster than my Java code since in Scala I can get the tedious not-performance-critical parts done with less effort, and spend more of my attention optimizing the algorithms and code for the performance-critical parts. – Rex Kerr

Java and Scala both compile down to JVM bytecode, so the difference isn’t that big. The best comparison you can get is probably on the computer language benchmarks game, which essentially says that Java and Scala both have the same memory usage. Scala is only slightly slower than Java on some of the benchmarks listed, but that could simply be because the implementation of the programs are different.

Really though, they’re both so close it’s not worth worrying about. The productivity increase you get by using a more expressive language like Scala is worth so much more than minimal (if any) performance hit – ryeguy

Intangibles

I think startups tend to underestimate the potential benefits of picking atypical technology stacks: learning new things is a powerful motivator for engineers—disproportionately so among those you’re most likely to want to hire—and having an interesting and thoughtful tech stack can help a lot in recruiting – Patrick Collison

Final Word

I’ll close with this series of quotes from Alex Payne of Twitter.

So what were our criteria for choosing Scala? Well first we asked, was it fast, and fun, and good for long-running process? Does it have advanced features? Can you be productive quickly? … And did Scala turn out to be fast? Well, what’s your definition of fast? About as fast as Java. It doesn’t have to be as fast as C or Assembly. …

Scala is a lot of fun to work in; yes, you can write staid, Java-like code when you start. Later, you can write Scala code that almost looks like Haskell. It can be very idiomatic, very functional — there’s a lot of flexibility there.

And it’s fast. The principal language developer at Scala worked on the JVM at Sun. When Java started, it was clearly a great language, but the VM was slow. The JVM has been brought to the modern age and we don’t think twice about using it.

Recent Relevance – New Site Launch

I’ve been doing a lot of thinking and reading recently about personal improvement and helping those around me.   I’ve been reading James Altucher’s book “The Choose Yourself Guide to Wealth” in which he advocates making a daily list of 10 ideas.

He suggests giving the ideas away for free, an “all boats lift in a rising tide” mentality.  I’ve been making multiple lists a day – some of them are too private to share (things like “how can I help Uncle Crime fight a lawsuit”) but the ones I can share will get posted.

I believe ideas are only relevant for a short period of time so the site is named  accordingly:

http://recentrelevance.com/

More thoughts on education

“I wish that our current educational approaches put much more emphasis on following one’s curiosity and asking ‘Why?’, rather than on memorization of facts and rote reproduction of processes. We can hardly blame students for not being curious when our educational system doesn’t generally reward that. So, sadly, it is surprising when a student asks ‘Why?'”

Steven Clarke

JSON Streaming Output

I recently put together a simple little bit of code to handle streaming large amounts of JSON formatted data to the browser.

In order for you to see any benefit from this, you will need to have the following scenario.

You have a response you’re sending back with some meta information about the response – maybe a response code, or error messages or a single record.

You also have a number of rows to send back as a member of the main JSON object.

These rows don’t all need to be the same object, but you can only have a single variable that you stream.

This allows you to fetch and send a single database row at a time instead of building up a huge response object in memory on the server.

It builds up a response that looks like this (the data in “rows” is streamed one line at a time):

{
    "success":true,
    "rows":
    [
        {"id":1,"random_data":"a6b1e15cc6523e87cb8d5a6a0ea39dbc61842b23"},
        {"id":2,"random_data":"12ae19125e2182294433e835ee8dc7e403154fdd"}
    ]
}