On programming languages
In the programming language design space there are so many interesting facets of language design and subsequent arguments about the various merits thereof that it can often bring about a paralysis of choice when deciding what to use or what to standardize on. Or, instead of paralysis, you’ll get entrenchment and an inability to form consensus: people love their language of choice and will argue its benefits to the death (or at least their exit). But what to do if you’ve got an application to write? A problem to solve? Another job to be done? What language should you use?
There are so many aspect to a programming language, that what you’re selecting is much more than just the syntax and language features. What is the community like? How long has the language been around? How painful are the changes between major versions? What’s the documentation like? What’s included in the standard library? What about the extended library and the greater ecosystem? Package management? What is the tooling like to do development? How do you model things in the language? What’s required to package and deploy/run? What kind of runtime behavior do you need and where is the application going to run? What other systems do you need to interact with and what languages are they written in? What is the core domain of your problem space? And, perhaps controversially, what does it feel like to write code in that language?
In short, I think that choice of programming language depends on team size and problem space—and then a dose of feel. If you’re an individual or a small team (and expect to stay small) and want to go far and fast pick a more sophisticated, niche language. If you’re working in a larger organization, standardize on a well known top-10 language widely used in your problem domain.
Powerful languages, often with steep learning curves, can be a secret weapon for individuals and small teams to move quickly with high quality and great flexibility. Once learned, and if well-matched to the domain, they are multipliers of effort. A common feature of these languages is that they allow programming language driven development in some form via meta programming (macros, domain specific languages, etc). This allows the host language to fit your domain like a glove and acts like a lever to make the most of your limited people power. Small teams can take on much larger competitors and move quickly to find product/market fit and innovate into new territory. See, for example, PG’s writing on LISP, Scheme, OCAML, Racket, or the Haskell communities. The critical insight here is that your constraint is not having a lot of humans working on the problem so: flexibility > standardization, advanced features > learning curve, agility > correctness.
Beyond being hard to learn, the downside is that these languages are usually niche enough that the pool of engineering talent with sufficient expertise is relatively limited and once you start creating languages within languages you have to train people to learn the base language, the domain, and then the domain specific language—which is no small task. Since skills in the derived language are much less transferable, it also means that unless you’re one of the engineers working on the meta level, it can be a dead end career-wise to know some random in-house DSL (I had a job once where said secondary language was affectionally named BobScript). Another downside is that you’ll often not be able to rely as much on the larger community so you’ll need to be competent and comfortable with quickly writing integrations into modern operational and business systems (e.g., you need to do billing but Stripe doesn’t have an out of the box Haskell client library). This usually means some combination of basic systems, net/http, security, and algorithmic programing is required. Additionally, editor tooling here can be spartan and basic. Developers tend to have high fidelity mental models of the inner workings of the language that they lean on instead of tooling support like autocomplete and a visual debugger.
Beyond language, what matters more is the other patterns you standardize on as there will be entire teams concerned about the meta health of all the code in your organization. Does it build consistently? Are security patches applied quickly and ubiquitously? Do you track open source license compliance? Is PII handled appropriately? Are you moving from one time series stats collection system to another? There’s always some project going on where you’re moving all the code from platform A to platform B (e.g., moving source control providers, cloud providers, runtime platforms, etc) and you’ll need teams and tools that can execute effectively across all projects. There are also issues of ownership and who to page when something goes wrong, catalogs of services and their SLOs and status pages. Having a uniform code base—one or a limited number of languages, identical formatting, single package system, standard scripts for setup/deploy, familiar interfaces, same RPC system—is a huge strategic advantage for that large organization. Small teams have most all of these same needs, but not the surface area and human level communication/synchronization problems.
So. Just you and a buddy or two? Reach for advanced languages that make you happy. Invest the time to learn them in depth and enjoy your freedom and agility. Building a large engineering team or working in a large organization? Standardization and ubiquity are your friends—stick with the usual suspects, mind your domain, focus on a single way of doing things but watch out because this is very much like the innovators dilemma: some LISP wielding high school kid from Hack Club is going out-innovate an entire subtree of your org chart one day.